This figure shows the complete testing architecture, including the different components. Everything is tested automatically (triggered by git commits) on a Jenkins setup, running at http://testing.confine-project.eu/
Different jobs test different situations, based on different hardware and/or different test subsets.
The minimal requirements to run the tests yourself are the following:
The first step is to checkout the
git clone http://git.confine-project.eu/confine/confine-testing.git confine-testing
This repository includes all the necessary scripts to create the VCT and researcher LXC containers.
cd into this directory:
test.sh script uses a few variables which can be set or the default values can be used. The variables are:
vct-container,vctffb4d14,controllerb55b35f,nodefwffb4d14.tar.xzby default. The list of available containers can be found at http://media.confine-project.eu/vct-container/test/.
vct_$(date -u +%s)
researcher_$(date -u +%s)
y, the script will only set up the VCT and researcher containers and then exit. This way,
lxc-consolefor the researcher can be used to manually start a test. Default
y, the script will not set up the VCT and researcher containers. This allows you to reuse currently running containers. Default
y, the script will not teardown the VCT and researcher containers. This allows for further investigation after the tests. Default
y, the script will install a new VCT inside the researcher container and use this for tests. Default
y, the script will copy the contents for the map
../researcher_overlayto the researcher. This allows customising e.g. network settings. Default
The only thing which you will want to change is the VCT_CONTAINER. A new VCT container is continually generated, triggered by new VCT, controller and node firmware releases. To limit disk usage, only the 20 last releases are kept. Better check quickly in the Jenkins installation for the latest generated VCT_CONTAINER:
This can be used as follows:
sudo bash -c "\ export VCT_CONTAINER=vct-container,vctffb4d14,controllerb55b35f,nodefwffb4d14.tar.xz; \ export VCT_LXC=vct; \ export RESEARCHER_LXC=researcher; \ bash test.sh"
The script will download, unpack, patch and start the VCT container provided by the
VCT_CONTAINER variable. It will then download the researcher container and unpack, patch and start it. The CONFINE tests and utilities are automatically donwloaded. It will then
ssh into the researcher and execute the tests. All this takes a while. After the tests are done, the VCT and researcher containers are stopped.
The IP address of the VCT container is default
fdf6:1e51:5f7b:b50c::2/64, for the researcher container
fdf6:1e51:5f7b:b50c::3/64 and for the bridge (
vmbridge) on the main host
Make sure that the testing framework is set up using:
sudo bash -c "\ export VCT_CONTAINER=vct-container,vctffb4d14,controllerb55b35f,nodefwffb4d14.tar.xz; \ export VCT_LXC=vct; \ export RESEARCHER_LXC=researcher; \ export SETUP_ONLY=y; \ export NO_SETUP=n; \ bash test.sh"
The SETUP_ONLY and NO_SETUP variables define respectively whether you want the test to be just set up (e.g. to inspect the LXC containers) or you do not want any setup of the containers (e.g. to test local applications or to reuse an existing configuration).
Entering the VCT and researcher LXC containers is done by using their name defined by
RESEARCHER_LXC. The login:“vct”, the password:“confine” without the quotes.
lxc-console -n $VCT_LXC lxc-console -n $RESEARCHER_LXC
When running the tests in the researcher container, the following command can be used to run all the tests:
env CONFINE_SERVER_API='http://vctserver/api' CONFINE_USER='vct' CONFINE_PASSWORD='vct' PYTHONPATH=confine-utils:confine-tests python -m unittest discover -s ./confine-tests/
For a specific test, the following command can be used:
env CONFINE_SERVER_API='http://vctserver/api' CONFINE_USER='vct' CONFINE_PASSWORD='vct' PYTHONPATH=confine-utils:confine-tests python ./confine-tests/confine/tests/test_group.py
For a specific test case, the following command can be used:
env CONFINE_SERVER_API='http://vctserver/api' CONFINE_USER='vct' CONFINE_PASSWORD='vct' PYTHONPATH=confine-utils:confine-tests python -m unittest confine.tests.test_node.GroupTests.test_get_groups
The testing framework can also help you test the installation of a new controller. It will start a new installation from scratch, inside the researcher LXC, and then run the tests against this controller. To trigger this, just run
sudo bash -c "\ export INSTALLATION_TEST=y; \ export TEST_API=1; \ export TEST_VCT=0; \ export TEST_INTEGRATION=0; \ export UPGRADE=$upgrade; \ bash test.sh;"
And enjoy the show.
As the tests are run automatically in different environments, some variables are defined to control which tests to run. They can be defined similar to the other variables, e.g. the CONFINE_USER variable.Currently the variables are:
In case you want to test a special feature, or you want to know how a VCT container is built, look at the vct_container template in the testing repository. It allows you to see everything required to build your own container. Example usage with your preferred hashes $ConfineDistHash $ControllerHash $NodeFirmwareHash:
sudo bash -c "source vct-container.sh; build_vct $ConfineDistHash $ControllerHash $NodeFirmwareHash"
The original test cases have been moved to a separate page for reference purposes.
The guidelines for writing tests have been moved to a separate page.