In this tutorial you will learn how to use the CONFINE SDK and VCT (Virtual CONFINE Testbed) tool by deploying a real BitTorrent experiment, from start to finish. In this experiment, we will try to follow the same procedure that a new researcher would follow to deploy a real experiment.
We have split the tutorial into several self-contained parts. Each part introduces new steps and concepts, and provides everything an experimenter needs. Although each of the parts can be read independently, our recommendation is to complete the whole tutorial, from the begining to the end, before trying to deploy a CONFINE experiment. You can watch a video with the entire experiment at: http://www.youtube.com/watch?v=T0yn4NDHR7E.
During the explanations we will use some common vocabulary from SFA (Slice-based Federation Architecture). If you are not familiar with some concepts like slice, sliver or allocation vs. deployment you can read more about SFA on GENI SFA 2.0 document. Of course, some vocabulary will be used from CONFINE itself which you can look up in the Elements on the system.
We decided to create a BitTorrent experiment inspired on [????]'s work. It is not a very complex experiment, neither very imaginative, but it allows to explore most of the functionalities and processes related with CONFINE experiment deployment and the VCT tool.
The experiment's aim is to measure the latency between different geographical sparse nodes while they are sharing a small file using the BitTorrent protocol.
CONFINE provides a complete experimental research testbed built atop federated community wireless networks, currently used on production scenarios by their community users. It gives researchers an unique opportunity to work on real wireless networking/computational problems and use real community traffic from real scenarios. Moreover, it also implies that researchers have to take under their responsibility a community-friendly testbed usage when they deploy their experiments.
To avoid the saturation of the community network we encourage researchers to follow the TLBR (Test Local Before Real) approach. As you will see soon, CONFINE provides a virtualization tool (named VCT), which will help prepare and test your experiments without needing access to real CONFINE nodes.
So, our roadmap will be:
For a more complete explanation about Confine-Dist installation you can follow the Confine-Dist software tutorial or the node software overview.
First of all, make sure that you have a working Debian operating system. It can be a virtualized one, as suggests the Confine-Dist software tutorial, but is not mandatory. However, take into account that VCT scrips create several virtualized networking/hardware devices that may harm you machine's performance if it lacks enough free resources.
We will assume that you already have knowledge about bash scripting, LXC (Linux Containter utilities) and POSIX networking. However you don't need to have a deep knowledge of all the components used by the system. VCT is a complete tool that provides an abstraction layer for most of the *NIX commands (but not all).
You will of course need a simple text editor like vim, emacs or nano.
If you are going to build your own OpenWrt image, you will also need to have around 6 or 8 GiB free space on hard disk and a broadband connection to download around 500 MiB of data, as well as some libraries and tools that must be available on you system. The next command installs all of them:
~$ apt-get install git subversion g++ ncurses-dev zlib1g-dev gawk flex unzip bzip2 \ gettext build-essential libncurses5-dev libncursesw5-dev \ binutils cpp gcc make psmisc linux-headers-$(uname -r) docbook-to-man
Installation is very simple. Just clone Confine-Dist and compile it. The compiling process will take a few hours depending on your machine. You can skip this step if you are going to test VCT but you do not want to create your own OpenWrt image. The git checkout
command ensures that you use the latest version this tutorial was checked against, but you may specify other versions or branches like testing
.
~$ git clone http://git.confine-project.eu/confine.git confine-dist ~$ cd confine-dist ~/confine-dist$ git checkout c10955a27dc2f6c251ac6065492a3aea10b0a74d ~/confine-dist$ make all V=99 J=4
One Confine SDK task is to generate the customized OpenWrt image that we are going to use inside each research devices. This image contains the confine library and all the necessary specific-configuration that allows us to control and manage these nodes. At some point during the compilation process, the OpenWrt installer may ask you for the customization of such images.
For now, just leave the default configuration selecting the Exit button. If you are interested in OpenWrt insights, or in customizing an image for your physical research devices, you may take a look at the OpenWrt website.
Now that CONFINE SDK is correctly installed, it is time to take a look into the directory structure.
/dl
contains all the downloaded libraries necessary to compile the SDK./files
contains configuration files that will be copied to the OpenWrt image./images
contains the compiled OpenWrt image itself./openwrt
contains OpenWrt source, cloned from our Redmine git repository./packages
contains CONFINE-specific packages related with slices/slivers initialization and deployment./utils
contains some command line utilities for researchers and developers./utils/vct
contains the VCT itself.
The VCT environment is provided by a set of shell functions, scripts and bash-environment variables.
The code is provided as a utility package in Confine-Dist (in directory utils/vct
).
Currently, all VCT functionality is provided by the script vct.sh
and configured via the default configuration vct.conf.defaults
. To change the default configuration, copy vct.conf.defaults
to vct.conf
and perform changes there.
~/confine-dist$ cd utils/vct ~/confine-dist/utils/vct$ cp vct.conf.defaults vct.conf
Most of the file has self-explanatory comments that will help you customize your configuration file. However, next we will review the most important section, and make some changes. On vct.conf
find the next line:
VCT_SUDO_ASK="YES" # default:"YES" ask whether to execute sudo commands befor doing so"
The default VCT behavior is to ask for any command that needs sudo
permissions. It may be useful to follow the code, and may prevent you from undesired changes. However if you trust us (and we hope it) this behavior can result annoying. So, we are going to tell VCT not to ask permission for sudo
commands. The line will result as:
VCT_SUDO_ASK="NO" # default:"YES" ask whether to execute sudo commands befor doing so"
The next line defines the path where VCT will be installed. We will review this directory structure later in this tutorial.
VCT_VIRT_DIR=/var/lib/vct
The line afterwards defines the URL to the OpenWrt image that CONFINE virtual nodes will run. As you can see, rather than using the recently generated file by Confine-Dist, we will be using a pre-generated image from our servers just to ensure that all CONFINE research devices have the same OS and confine.lib
version (you can leave the value already specified in the file).
# the format must look like: # <url-type><path/to/dir>/<file-to-download>.<image-type>.<compresson-type> # valid url types: http://, https://, ftp://, file://, ssh:[port]:<user@domain>:// # valid image types: raw, img, or vmdk (vmdk only for virsh --version >= 0.9.9) # valid compressions: tgz or gz VCT_TEMPLATE_URL="http://images.confine-project.eu/misc/openwrt-x86-generic-combined-ext4-merge_trunk_02.img.gz"
For the purpose of this tutorial, we are skiping the rest of the posible configurations that you can tune on this file, and we will focus on the experiments' definition. You can find extra information on node software overview. By default, two experiments are defined in order to show the common configuration options (we are going to create our own experiment later on this tutorial).
################################################# # define your own experiments here and allocate them with: ./vct_sliver_allocate ... vct_hello_openwrt() { cat <<EOF config sliver $SLICE_ID option user_pubkey "$( cat $VCT_KEYS_DIR/id_rsa.pub )" # option fs_template_url "http://downloads.openwrt.org/backfire/10.03.1-rc6/x86_generic/openwrt-x86-generic-rootfs.tar.gz" option fs_template_url "http://images.confine-project.eu/misc/CONFINE-sliver-openwrt-x86-generic-rootfs-070812_1023.tar.gz" option exp_data_url "http://distro.confine-project.eu/misc/exp-data-hello-world-openwrt.tgz" option exp_name "hello-openwrt-experiment" option vlan_nr "f${SLICE_ID:10:2}" # mandatory for if-types isolated option if00_type internal option if00_name priv option if01_type public # optional option if01_name pub0 option if01_ipv4_proto $VCT_NODE_SL_PUBLIC_IPV4_PROTO # mandatory for if-type public option if02_type isolated # optional option if02_name iso0 option if02_parent eth1 # mandatory for if-types isolated option if03_type isolated # optional option if03_name iso1 option if03_parent eth2 # mandatory for if-types isolated option if04_type isolated # optional option if04_name iso2 option if04_parent wlan0 # mandatory for if-types isolated option if05_type isolated # optional option if05_name iso3 option if05_parent wlan1 # mandatory for if-types isolated EOF } vct_hello_debian() { cat <<EOF config sliver $SLICE_ID option user_pubkey "$( cat $VCT_KEYS_DIR/id_rsa.pub )" option fs_template_url "http://distro.confine-project.eu/misc/debian32.tgz" option exp_data_url "http://distro.confine-project.eu/misc/exp-data-hello-world-debian.tgz" option exp_name "hello-debian-experiment" option vlan_nr "f${SLICE_ID:10:2}" # mandatory for if-types isolated option if00_type internal option if00_name priv option if01_type public # optional option if01_name pub0 option if01_ipv4_proto $VCT_NODE_SL_PUBLIC_IPV4_PROTO # mandatory for if-type public option if02_type isolated # optional option if02_name iso0 option if02_parent eth1 # mandatory for if-types isolated option if03_type isolated # optional option if03_name iso1 option if03_parent eth2 # mandatory for if-types isolated option if04_type isolated # optional option if04_name iso2 option if04_parent wlan0 # mandatory for if-types isolated option if05_type isolated # optional option if05_name iso3 option if05_parent wlan1 # mandatory for if-types isolated EOF } #################################################
Creating a new experiment is as simple as replicating a vct_hello_
function with a different name in the vct.conf
file and tuning some parameters. The most important options are:
fs_template_url
defines the URL from where VCT will donwload the Linux container filesystem. By default, we provide two templates (one OpenWrt, one Debian) with 512 MiB of hard disk space. They will be enough for most experiments.exp_data_url
defines the URL from where VCT will donwload the experiment's scripts and data. By default, we provide two templates for testing purposes. The OpenWrt template performs a simple ping between slivers, while the Debian template does nothing.exp_name
defines the experiment's name.It is recommended to leave the other options unchanged.
Take into account that “experiment” configuration is actually doing more than what it is supposed to do. It also defines the single Linux container template for all the slivers in your slice.
The second configuration file is vct.nodes
, which contains the necessary information to use VCT with real nodes. Each one is defined on a single line.
# This file is ignored in favor of vct.nodes.local # Use it only for documenting the used NODE_IDs: #node-id mac comments: f09c 00:0d:b9:24:b4:9c alix (for testing in Berlin) f001 00:0d:b9:24:b5:00 alix (for testing in Berlin) de01 00:03:1d:0b:15:31 commell (for testing in Wachtberg) # Each following entry includes not only the MAC, but also the IPv4 address of # the local interface of a research device *not in the local network*. The IP # adress must be reachable from the server and it will be used by VCT as a # recovery addresses for node configuration. f101 00:01:2E:2D:89:2B|10.228.201.72 Zotac Lab 104 UPC f102 00:0D:B9:26:9D:AC|10.228.204.6 Alix C6 roof f103 00:01:2E:39:9A:FA|10.228.201.81 Zotac Lab 104 UPC (black one)
Each line contains a local node identification number (coded with a 4-digit hexadecimal number), the MAC addres of the public node interface (and optionally its public IPv4 address) and a comment string, all of them separated by whitespace.
The pre-defined devices are part of the UPC Campus Nord testbed. For the purpose of this tutorial, we will comment all lines adding a #
symbol at the begining of each line to deal only with virtual CONFINE devices.
Well! Now we are ready to setup our VCT and start playing with it! The setup process takes two independent steps: installation and initialization.
The first step installs all the VCT dependencies left on your OS, so it only has to be executed the first time. To start the installation, type the next command on you terminal (hereafter we will supose your working directory is ~/confine/utils/vct
).
~$ ./vct_system_install
As a result, VCT downloads the research device template and other configuration files on the newly created VCT_VIRT_DIR
folder structure.
/downloads
contains the CONFINE OpenWrt image defined in VCT_TEMPLATE_URL
./images
contains all the KVM image files./keys
contains a set of private/public keys used by virtualized nodes to allow ssh access to virtualized nodes and slices./mnt
contains mounted filesystems (TODO VERIFY!)./node_prepare
contains (TODO ???)/rpc_files
contains (TODO ???)/tinc
contains (TODO ???)/uci
contains (TODO ???)The next step is the initialization step, whose aim is to configure all virtual networking devices.
~$ ./vct_system_init
As a result, VCT configures the necessary networking bridges, public and private virtual links. Basically… (TODO brief explanation about interfaces). If you want to known more about the networking architecture you can take a look at the node architecture.
And… that's it. VCT ready to work!
As you can see, all ./vct_
commands are, in fact, symbolic links to the ./vct.sh
bash script to make the usage of its functions more intuitive. They are divided in six (6) different categories. If you do not know how to use a given command, you can execute ./vct_help
for a detailed list of options.
./vct_system_
commands configure and clean the VCT system on your current machine../vct_node_
commands provide direct interaction with real and virtualized CONFINE nodes (also known as research devices)../vct_slice_
commands perform informational operations on slices/slivers../vct_sliver_
commands perform activity operations on slices/slivers.
Some times VCT treats slices, slivers and experiments as a single thing when clearly they are not. The important thing to remember is that a slice is composed by a set of slivers and that an experiment is allocated and deployed on a single sliver.
An Link Error Model for VCT based on Rician Fading can be used in combination with the VCT tools. However, it is not part of the VCT tools but independent scripts.