The automation env is serving the tests of Nmstate. It may be used both locally and through CI.
Container specifications to be used for the tests are in the packaging
directory. The images are published on docker hub:
https://hub.docker.com/r/nmstate/
run-tests.sh: Execute the tests in a container using 'nmstate/fedora-nmstate-dev' container image.
The following steps are executed: - Run the container (defined in the Dockerfile) as a daemon. - Stop NetworkManager before adding additional networks (ifaces). - Add additonal networks (ifaces) to the container. - Start NetworkManager. - Execute all tests in the container.
It also handles the cleanup of the container and nets (stop,rm).
Assuming podman is installed on the host, just run:
./automation/run-tests.sh
By default, ./automation/run-tests.sh
will run all tests in the container
using 'nmstate/fedora-nmstate-dev' container image.
You may change the test type by specifying the --test-type
flag, for example:
./automation/run-tests.sh --test-type integ --el8
:
Integration tests (without slow test cases) using
'nmstate/centos8-nmstate-dev' container image.
./automation/run-tests.sh --test-type integ
:
Integration tests (without slow test cases) using
'nmstate/fedora-nmstate-dev' container image.
./automation/run-tests.sh --test-type integ_tier1
:
Integration tier1 test cases using nmstate/fedora-nmstate-dev
container
image.
./automation/run-tests.sh --test-type integ_tier2
:
Integration tier2 test cases using nmstate/fedora-nmstate-dev
container
image.
./automation/run-tests.sh --test-type integ_slow
:
Integration slow test cases using nmstate/fedora-nmstate-dev
container
image.
For a full list of command-line flags, run ./automation/run-tests.sh --help
.
For debugging, it is convenient to run the container and then connect to it in
order to run the tests. Setting the environment variable debug_exit_shell
will make the script start a shell instead of exiting the script after an error
or running the scripts:
debug_exit_shell=1 ./automation/run-tests.sh
After closing the shell, the container will be removed. Alternatively it is
possible to provide the --debug-shell
command-line option.
To specify a different container image for the tests, specify it with the
CONTAINER_IMAGE
variable:
CONTAINER_IMAGE=local/centos8-nmstate-dev debug_exit_shell=1 ./automation/run-tests.sh
It is also possible to pass extra arguments to PDB using the
nmstate_pytest_extra_args
variable or via --pytest-args
command-line
option, for example:
nmstate_pytest_extra_args="--pdb -x" ./automation/run-tests.sh
or:
./automation/run-tests.sh --pytest-args "--pdb -x"
../packaging/build-container.sh local/centos8-nmstate-dev ../packaging/build-container.sh local/fedora-nmstate-dev
To test the image, either specify it manually as described above or tag it locally:
podman tag local/centos8-nmstate-dev docker.io/nmstate/centos8-nmstate-dev:latest podman tag local/fedora-nmstate-dev docker.io/nmstate/fedora-nmstate-dev:latest
The container images are automatically rebuilt for new commits to the base branch or new tags. Therefore updates to the Docker Hub images should always happen with a pull request that is merged to ensure that the change is persistent. If this is not feasible, a new build could be pushed as follow to the Docker Hub:
podman login docker.io podman tag local/centos8-nmstate-dev nmstate/centos8-nmstate-dev:latest podman push nmstate/centos8-nmstate-dev:latest \ docker://docker.io/nmstate/centos8-nmstate-dev:latest podman tag local/fedora-nmstate-dev nmstate/fedora-nmstate-dev:latest podman push nmstate/fedora-nmstate-dev:latest \ docker://docker.io/nmstate/fedora-nmstate-dev:latest
It will be overwritten after the next commit to base, though.
In order to perform integration test cases against InfiniBand feature, running test in bare-metal OS is required.
Assuming the InfiniBand card is listed in ip link
as mlx5_ib0
and been
configured as datagram
mode.
sudo dnf install `./packaging/make_rpm.sh|tail -1` sudo ip netns add tmp sudo ip link add eth1 type veth peer name eth1peer sudo ip link add eth2 type veth peer name eth2peer sudo ip link set eth1 up sudo ip link set eth2 up sudo ip link set eth1peer netns tmp sudo ip link set eth2peer netns tmp sudo ip netns exec tmp ip link set eth1peer up sudo ip netns exec tmp ip link set eth2peer up sudo nmcli device set eth1 managed yes sudo nmcli device set eth2 managed yes cd tests/integration # Set TEST_IB_CONNECTED_MODE=1 when cards are configured as connected mode sudo env TEST_REAL_NIC=mlx5_ib0 pytest-3 -vv ./infiniband_test.py