User Tools

Site Tools


testing:start

CONFINE testing

Architecture Overview

This figure shows the complete testing architecture, including the different components. Everything is tested automatically (triggered by git commits) on a Jenkins setup, running at http://testing.confine-project.eu/

Different jobs test different situations, based on different hardware and/or different test subsets.

Requirements

The minimal requirements to run the tests yourself are the following:

  • Internet access (for downloading containers)
  • quilt software (for patch management)
  • ssh software (to access containers)
  • IPv6 support (CONFINE uses IPv6)
  • conntrack support (for masquerading)
  • LXC enabled kernel (we heavily use LXC)
  • mounted cgroup (for LXC)
  • git software (for source code management)
  • a 64 bit system (the researcher container is 64 bit, contact Bart if you need a workaround)

How to

The first step is to checkout the confine-testing repository:

git clone http://git.confine-project.eu/confine/confine-testing.git confine-testing

This repository includes all the necessary scripts to create the VCT and researcher LXC containers.

Then cd into this directory:

cd confine-testing

The test.sh script uses a few variables which can be set or the default values can be used. The variables are:

  • VCT_CONTAINER: this defines the version of the VCT container to be used for testing: currently this is set to vct-container,vctffb4d14,controllerb55b35f,nodefwffb4d14.tar.xz by default. The list of available containers can be found at http://media.confine-project.eu/vct-container/test/.
  • VCT_LXC: this is the name of the VCT container. Default vct_$(date -u +%s)
  • RESEARCHER_LXC: this is the name of the researcher container. Default researcher_$(date -u +%s)
  • SETUP_ONLY: when set to y, the script will only set up the VCT and researcher containers and then exit. This way, lxc-console for the researcher can be used to manually start a test. Default n.
  • NO_SETUP: when set to y, the script will not set up the VCT and researcher containers. This allows you to reuse currently running containers. Default n.
  • NO_TEARDOWN: when set to y, the script will not teardown the VCT and researcher containers. This allows for further investigation after the tests. Default n.
  • INSTALLATION_TEST: when set to y, the script will install a new VCT inside the researcher container and use this for tests. Default n.
  • RESEARCHER_OVERLAY: when set to y, the script will copy the contents for the map ../researcher_overlay to the researcher. This allows customising e.g. network settings. Default n.

The only thing which you will want to change is the VCT_CONTAINER. A new VCT container is continually generated, triggered by new VCT, controller and node firmware releases. To limit disk usage, only the 20 last releases are kept. Better check quickly in the Jenkins installation for the latest generated VCT_CONTAINER:

  • click on the branches you want at the bottom of the output (e.g. ConfineDistBranch=master,origin/master,NodeFirmwareBranch=testing)
  • click on the last yellow build in the left column (i.e. the most recent build which has not failed)
  • click Console Output. There you should see a line describing the used container (e.g. Using vct: vct-container,vct3e32990,controller1726c74,nodefw6357d5a.tar.xz).

This can be used as follows:

sudo bash -c "\
export VCT_CONTAINER=vct-container,vctffb4d14,controllerb55b35f,nodefwffb4d14.tar.xz; \
export VCT_LXC=vct; \
export RESEARCHER_LXC=researcher; \
bash test.sh"

The script will download, unpack, patch and start the VCT container provided by the VCT_CONTAINER variable. It will then download the researcher container and unpack, patch and start it. The CONFINE tests and utilities are automatically donwloaded. It will then ssh into the researcher and execute the tests. All this takes a while. After the tests are done, the VCT and researcher containers are stopped.

The IP address of the VCT container is default fdf6:1e51:5f7b:b50c::2/64, for the researcher container fdf6:1e51:5f7b:b50c::3/64 and for the bridge (vmbridge) on the main host fdf6:1e51:5f7b:b50c::1/64.

Run tests manually

Make sure that the testing framework is set up using:

sudo bash -c "\
export VCT_CONTAINER=vct-container,vctffb4d14,controllerb55b35f,nodefwffb4d14.tar.xz; \
export VCT_LXC=vct; \
export RESEARCHER_LXC=researcher; \
export SETUP_ONLY=y; \
export NO_SETUP=n; \
bash test.sh"

The SETUP_ONLY and NO_SETUP variables define respectively whether you want the test to be just set up (e.g. to inspect the LXC containers) or you do not want any setup of the containers (e.g. to test local applications or to reuse an existing configuration).

Entering the VCT and researcher LXC containers is done by using their name defined by VCT_LXC and RESEARCHER_LXC. The login:“vct”, the password:“confine” without the quotes.

lxc-console -n $VCT_LXC
lxc-console -n $RESEARCHER_LXC

When running the tests in the researcher container, the following command can be used to run all the tests:

env CONFINE_SERVER_API='http://vctserver/api' CONFINE_USER='vct' CONFINE_PASSWORD='vct'  PYTHONPATH=confine-utils:confine-tests python -m unittest discover -s ./confine-tests/

For a specific test, the following command can be used:

env CONFINE_SERVER_API='http://vctserver/api' CONFINE_USER='vct' CONFINE_PASSWORD='vct'  PYTHONPATH=confine-utils:confine-tests python ./confine-tests/confine/tests/test_group.py

For a specific test case, the following command can be used:

env CONFINE_SERVER_API='http://vctserver/api' CONFINE_USER='vct' CONFINE_PASSWORD='vct'  PYTHONPATH=confine-utils:confine-tests python  -m unittest confine.tests.test_node.GroupTests.test_get_groups

Run an installation test

The testing framework can also help you test the installation of a new controller. It will start a new installation from scratch, inside the researcher LXC, and then run the tests against this controller. To trigger this, just run

sudo bash -c "\
export INSTALLATION_TEST=y; \
export TEST_API=1; \
export TEST_VCT=0; \
export TEST_INTEGRATION=0; \
export UPGRADE=$upgrade; \
bash test.sh;"

And enjoy the show.

Control which tests to run

As the tests are run automatically in different environments, some variables are defined to control which tests to run. They can be defined similar to the other variables, e.g. the CONFINE_USER variable.Currently the variables are:

  • TEST_VCT run inside VCT environment, so start creating nodes etc. You should turn this on, default value is 0 (no VCT).
  • TEST_API run basic REST API tests. You can leave this parameter, default value is 1 (do run API tests).
  • TEST_INTEGRATION run long integration tests, which might take more than 15 minutes. Default parameter: 0 (do not run integration tests).
  • TEST_EXISTINGNODES run tests with existing nodes, so do not create new nodes and try to reuse existing nodes. Useful for real hardware testing, probably leave this off. Default parameter: 0 (do not reuse existing nodes).

Build your own VCT container

In case you want to test a special feature, or you want to know how a VCT container is built, look at the vct_container template in the testing repository. It allows you to see everything required to build your own container. Example usage with your preferred hashes $ConfineDistHash $ControllerHash $NodeFirmwareHash:

sudo bash -c "source vct-container.sh; build_vct $ConfineDistHash $ControllerHash $NodeFirmwareHash"

Test Cases

The original test cases have been moved to a separate page for reference purposes.

Guide lines

The guidelines for writing tests have been moved to a separate page.

testing/start.txt · Last modified: 2014/02/24 13:24 by braem