In this tutorial you will learn how to use the CONFINE SDK and VCT (Virtual CONFINE Testbed) tool by deploying a real BitTorrent experiment, from start to finish. In this experiment, we will try to follow the same procedure that a new researcher would follow to deploy a real experiment.

We have split the tutorial into several self-contained parts. Each part introduces new steps and concepts, and provides everything an experimenter needs. Although each of the parts can be read independently, our recommendation is to complete the whole tutorial, from the begining to the end, before trying to deploy a CONFINE experiment. You can watch a video with the entire experiment at: http://www.youtube.com/watch?v=T0yn4NDHR7E.

During the explanations we will use some common vocabulary from SFA (Slice-based Federation Architecture). If you are not familiar with some concepts like slice, sliver or allocation vs. deployment you can read more about SFA on GENI SFA 2.0 document. Of course, some vocabulary will be used from CONFINE itself which you can look up in the Elements on the system.

We decided to create a BitTorrent experiment inspired on [????]'s work. It is not a very complex experiment, neither very imaginative, but it allows to explore most of the functionalities and processes related with CONFINE experiment deployment and the VCT tool.

The experiment's aim is to measure the latency between different geographical sparse nodes while they are sharing a small file using the BitTorrent protocol.

CONFINE provides a complete experimental research testbed built atop federated community wireless networks, currently used on production scenarios by their community users. It gives researchers an unique opportunity to work on real wireless networking/computational problems and use real community traffic from real scenarios. Moreover, it also implies that researchers have to take under their responsibility a community-friendly testbed usage when they deploy their experiments.

To avoid the saturation of the community network we encourage researchers to follow the TLBR (Test Local Before Real) approach. As you will see soon, CONFINE provides a virtualization tool (named VCT), which will help prepare and test your experiments without needing access to real CONFINE nodes.

So, our roadmap will be:

1. Local slice deployment. We will create a set of virtualized research devices using the VCT system, where we will deploy a virtual slice. It will contain a minimal debian32 image and a hello-world experiment that will do nothing.
2. A first BitTorrent experiment. We will manually install and configure all the experiment's software and dependencies. Afterwards, we will test our BitTorrent experiment.
3. Preparing and deploying experiments on a CONFINE testbed. Once the experiment is tested, we will create a Debian template image to deploy the experiment on hundreds of CONFINE nodes without extra configuration. Finally, we will show how to create a simple script that will execute the experiment and collect the necessary resulting data.

For a more complete explanation about Confine-Dist installation you can follow the Confine-Dist software tutorial or the node software overview.

First of all, make sure that you have a working Debian operating system. It can be a virtualized one, as suggests the Confine-Dist software tutorial, but is not mandatory. However, take into account that VCT scrips create several virtualized networking/hardware devices that may harm you machine's performance if it lacks enough free resources.

We will assume that you already have knowledge about bash scripting, LXC (Linux Containter utilities) and POSIX networking. However you don't need to have a deep knowledge of all the components used by the system. VCT is a complete tool that provides an abstraction layer for most of the *NIX commands (but not all).

You will of course need a simple text editor like vim, emacs or nano.

If you are going to build your own OpenWrt image, you will also need to have around 6 or 8 GiB free space on hard disk and a broadband connection to download around 500 MiB of data, as well as some libraries and tools that must be available on you system. The next command installs all of them:

~$ apt-get install git subversion g++ ncurses-dev zlib1g-dev gawk flex unzip bzip2 \
   gettext build-essential libncurses5-dev libncursesw5-dev \
   binutils cpp gcc make psmisc linux-headers-$(uname -r) docbook-to-man

Installation is very simple. Just clone Confine-Dist and compile it. The compiling process will take a few hours depending on your machine. You can skip this step if you are going to test VCT but you do not want to create your own OpenWrt image. The git checkout command ensures that you use the latest version this tutorial was checked against, but you may specify other versions or branches like testing.

~$ git clone http://git.confine-project.eu/confine.git confine-dist
~$ cd confine-dist
~/confine-dist$ git checkout c10955a27dc2f6c251ac6065492a3aea10b0a74d
~/confine-dist$ make all V=99 J=4

One Confine SDK task is to generate the customized OpenWrt image that we are going to use inside each research devices. This image contains the confine library and all the necessary specific-configuration that allows us to control and manage these nodes. At some point during the compilation process, the OpenWrt installer may ask you for the customization of such images.

For now, just leave the default configuration selecting the Exit button. If you are interested in OpenWrt insights, or in customizing an image for your physical research devices, you may take a look at the OpenWrt website.

Now that CONFINE SDK is correctly installed, it is time to take a look into the directory structure.

• /dl contains all the downloaded libraries necessary to compile the SDK.
• /files contains configuration files that will be copied to the OpenWrt image.
• /images contains the compiled OpenWrt image itself.
• /openwrt contains OpenWrt source, cloned from our Redmine git repository.
• /packages contains CONFINE-specific packages related with slices/slivers initialization and deployment.
• /utils contains some command line utilities for researchers and developers.
• /utils/vct contains the VCT itself.

The VCT environment is provided by a set of shell functions, scripts and bash-environment variables. The code is provided as a utility package in Confine-Dist (in directory utils/vct).

Currently, all VCT functionality is provided by the script vct.sh and configured via the default configuration vct.conf.defaults. To change the default configuration, copy vct.conf.defaults to vct.conf and perform changes there.

~/confine-dist$ cd utils/vct
~/confine-dist/utils/vct$ cp vct.conf.defaults vct.conf

Most of the file has self-explanatory comments that will help you customize your configuration file. However, next we will review the most important section, and make some changes. On vct.conf find the next line:

VCT_SUDO_ASK="YES"    # default:"YES" ask whether to execute sudo commands befor doing so"

The default VCT behavior is to ask for any command that needs sudo permissions. It may be useful to follow the code, and may prevent you from undesired changes. However if you trust us (and we hope it) this behavior can result annoying. So, we are going to tell VCT not to ask permission for sudo commands. The line will result as:

VCT_SUDO_ASK="NO"    # default:"YES" ask whether to execute sudo commands befor doing so"

The next line defines the path where VCT will be installed. We will review this directory structure later in this tutorial.

VCT_VIRT_DIR=/var/lib/vct

The line afterwards defines the URL to the OpenWrt image that CONFINE virtual nodes will run. As you can see, rather than using the recently generated file by Confine-Dist, we will be using a pre-generated image from our servers just to ensure that all CONFINE research devices have the same OS and confine.lib version (you can leave the value already specified in the file).

# the format must look like:
# <url-type><path/to/dir>/<file-to-download>.<image-type>.<compresson-type>
# valid url types: http://, https://, ftp://, file://, ssh:[port]:<user@domain>://
# valid image types: raw, img, or vmdk (vmdk only for virsh --version >= 0.9.9)
# valid compressions: tgz or gz
 VCT_TEMPLATE_URL="http://images.confine-project.eu/misc/openwrt-x86-generic-combined-ext4-merge_trunk_02.img.gz"

For the purpose of this tutorial, we are skiping the rest of the posible configurations that you can tune on this file, and we will focus on the experiments' definition. You can find extra information on node software overview. By default, two experiments are defined in order to show the common configuration options (we are going to create our own experiment later on this tutorial).

#################################################
# define your own experiments here and allocate them with: ./vct_sliver_allocate ...
 
 
vct_hello_openwrt() {
    cat <<EOF
config sliver $SLICE_ID
    option user_pubkey     "$( cat $VCT_KEYS_DIR/id_rsa.pub )"
#   option fs_template_url "http://downloads.openwrt.org/backfire/10.03.1-rc6/x86_generic/openwrt-x86-generic-rootfs.tar.gz"
    option fs_template_url "http://images.confine-project.eu/misc/CONFINE-sliver-openwrt-x86-generic-rootfs-070812_1023.tar.gz"
    option exp_data_url    "http://distro.confine-project.eu/misc/exp-data-hello-world-openwrt.tgz"
    option exp_name        "hello-openwrt-experiment"
    option vlan_nr         "f${SLICE_ID:10:2}"    # mandatory for if-types isolated
    option if00_type       internal
    option if00_name       priv
    option if01_type       public   # optional
    option if01_name       pub0
    option if01_ipv4_proto $VCT_NODE_SL_PUBLIC_IPV4_PROTO   # mandatory for if-type public
    option if02_type       isolated # optional
    option if02_name       iso0
    option if02_parent     eth1     # mandatory for if-types isolated
    option if03_type       isolated # optional
    option if03_name       iso1
    option if03_parent     eth2     # mandatory for if-types isolated
    option if04_type       isolated # optional
    option if04_name       iso2
    option if04_parent     wlan0     # mandatory for if-types isolated
    option if05_type       isolated # optional
    option if05_name       iso3
    option if05_parent     wlan1     # mandatory for if-types isolated
EOF
}
 
vct_hello_debian() {
    cat <<EOF
config sliver $SLICE_ID
    option user_pubkey     "$( cat $VCT_KEYS_DIR/id_rsa.pub )"
    option fs_template_url "http://distro.confine-project.eu/misc/debian32.tgz"
    option exp_data_url    "http://distro.confine-project.eu/misc/exp-data-hello-world-debian.tgz"
    option exp_name        "hello-debian-experiment"
    option vlan_nr         "f${SLICE_ID:10:2}"    # mandatory for if-types isolated
    option if00_type       internal
    option if00_name       priv
    option if01_type       public   # optional
    option if01_name       pub0
    option if01_ipv4_proto $VCT_NODE_SL_PUBLIC_IPV4_PROTO   # mandatory for if-type public
    option if02_type       isolated # optional
    option if02_name       iso0
    option if02_parent     eth1     # mandatory for if-types isolated
    option if03_type       isolated # optional
    option if03_name       iso1
    option if03_parent     eth2     # mandatory for if-types isolated
    option if04_type       isolated # optional
    option if04_name       iso2
    option if04_parent     wlan0     # mandatory for if-types isolated
    option if05_type       isolated # optional
    option if05_name       iso3
    option if05_parent     wlan1     # mandatory for if-types isolated
EOF
}
 
 
#################################################

Creating a new experiment is as simple as replicating a vct_hello_ function with a different name in the vct.conf file and tuning some parameters. The most important options are:

• fs_template_url defines the URL from where VCT will donwload the Linux container filesystem. By default, we provide two templates (one OpenWrt, one Debian) with 512 MiB of hard disk space. They will be enough for most experiments.
• exp_data_url defines the URL from where VCT will donwload the experiment's scripts and data. By default, we provide two templates for testing purposes. The OpenWrt template performs a simple ping between slivers, while the Debian template does nothing.
• exp_name defines the experiment's name.

It is recommended to leave the other options unchanged.

Take into account that “experiment” configuration is actually doing more than what it is supposed to do. It also defines the single Linux container template for all the slivers in your slice.

The second configuration file is vct.nodes, which contains the necessary information to use VCT with real nodes. Each one is defined on a single line.

# This file is ignored in favor of vct.nodes.local 
# Use it only for documenting the used NODE_IDs: 
 
#node-id  mac                                   comments:
 
f09c      00:0d:b9:24:b4:9c                     alix (for testing in Berlin)
f001      00:0d:b9:24:b5:00                     alix (for testing in Berlin)
 
de01      00:03:1d:0b:15:31                     commell (for testing in Wachtberg)
 
# Each following entry includes not only the MAC, but also the IPv4 address of
# the local interface of a research device *not in the local network*.  The IP
# adress must be reachable from the server and it will be used by VCT as a
# recovery addresses for node configuration.
f101      00:01:2E:2D:89:2B|10.228.201.72       Zotac Lab 104 UPC
f102      00:0D:B9:26:9D:AC|10.228.204.6        Alix C6 roof
f103      00:01:2E:39:9A:FA|10.228.201.81       Zotac Lab 104 UPC (black one)

Each line contains a local node identification number (coded with a 4-digit hexadecimal number), the MAC addres of the public node interface (and optionally its public IPv4 address) and a comment string, all of them separated by whitespace.

The pre-defined devices are part of the UPC Campus Nord testbed. For the purpose of this tutorial, we will comment all lines adding a # symbol at the begining of each line to deal only with virtual CONFINE devices.

Well! Now we are ready to setup our VCT and start playing with it! The setup process takes two independent steps: installation and initialization.

The first step installs all the VCT dependencies left on your OS, so it only has to be executed the first time. To start the installation, type the next command on you terminal (hereafter we will supose your working directory is ~/confine/utils/vct).

~$ ./vct_system_install

As a result, VCT downloads the research device template and other configuration files on the newly created VCT_VIRT_DIR folder structure.

• /downloads contains the CONFINE OpenWrt image defined in VCT_TEMPLATE_URL.
• /images contains all the KVM image files.
• /keys contains a set of private/public keys used by virtualized nodes to allow ssh access to virtualized nodes and slices.
• /mnt contains mounted filesystems (TODO VERIFY!).
• /node_prepare contains (TODO ???)
• /rpc_files contains (TODO ???)
• /tinc contains (TODO ???)
• /uci contains (TODO ???)

The next step is the initialization step, whose aim is to configure all virtual networking devices.

~$ ./vct_system_init

As a result, VCT configures the necessary networking bridges, public and private virtual links. Basically… (TODO brief explanation about interfaces). If you want to known more about the networking architecture you can take a look at the node architecture.

And… that's it. VCT ready to work!

As you can see, all ./vct_ commands are, in fact, symbolic links to the ./vct.sh bash script to make the usage of its functions more intuitive. They are divided in six (6) different categories. If you do not know how to use a given command, you can execute ./vct_help for a detailed list of options.

• ./vct_system_ commands configure and clean the VCT system on your current machine.
• ./vct_node_ commands provide direct interaction with real and virtualized CONFINE nodes (also known as research devices).
• ./vct_slice_ commands perform informational operations on slices/slivers.
• ./vct_sliver_ commands perform activity operations on slices/slivers.

Some times VCT treats slices, slivers and experiments as a single thing when clearly they are not. The important thing to remember is that a slice is composed by a set of slivers and that an experiment is allocated and deployed on a single sliver.

An Link Error Model for VCT based on Rician Fading can be used in combination with the VCT tools. However, it is not part of the VCT tools but independent scripts.

[1] https://redmine.confine-project.eu/projects/confine/repository/revisions/master/show/utils/vct