This guide describes how to register, install and maintain a node in a CONFINE testbed like Community-Lab. When interacting with a testbed controller, you will need to authenticate as a user with at least a node administrator role (also known as a technician) in a group which is allowed to manage nodes in the testbed. We recommend that you first get some experience by managing some virtual nodes under VCT before moving to real testbeds. See Using the Virtual CONFINE Testbed.
You can see some screenshots and more detailed information on the administration of nodes in Node installation (which is tailored for the installation of nodes belonging to Guifi.net), in CONFINE Research Device HOWTO (for Ninux) or in the tutorial section Creating Nodes (which covers the management of VCT nodes). For mode information on upgrading a node, see Node upgrade.
To register a new node for your group in the Community-Lab controller:
You shall fill the configuration items in the new node page:
That should be enough in most cases for indoor nodes. Other items you may want to change:
i686, you may change it under the Advanced section. Keep it unless your processor is very low-power (Via C7 or AMD Geode).
x86_64is not yet directly supported, so choose
It is also important to properly configure the network connectivity of the node under Firmware configuration:
#N, where N is the number of DHCP addresses reserved for slivers.
BASE_IP#N, indicating that N addresses are reserved for slivers after and including an initial base IP address in the local network.
Finally, if your node has additional network interfaces which can be used to reach other nodes at the raw link layer:
You may need to manually configure these interfaces in OpenWrt after installation.
Once the desired fields are filled, save the node. You are brought to the node list where it appears in set state “DEBUG”, which indicates that the node configuration is invalid or incomplete. If you visit the node page by clicking on its name you will see two warnings about the node missing some keys.
The CONFINE controller includes an application that lets you build a firmware image already customized for your node so that you may install and start using it straight away. Of course, this implies that the controller gets to know all private data about your node (keys, certificates, passwords…), but on the other hand you can avoid struggling with the manual configuration of OpenWrt and the CONFINE node system.
Once you have registered the new node in the controller (see Registering a node), go to the node page (e.g. via Nodes / Nodes in the main menu, then click on the node) and click on the Download firmware… button. In the firmware generation screen you may fine-tune some aspects of the process:
Please set the desired values, and remember to enable the USB image for the first node installation (not for node upgrades).
Now click on Build firmware, you will see a progress bar and after a while you will be presented with a generation summary screen. Check that the new firmware is available (e.g. that its build did not fail), and fetch it to your computer using the image download link.
The recommended way of installing a node for the first time is doing it via a USB flash drive (unless you have a virtual node, in which case please check vnode instead). Please have in mind that both the USB drive and the node's disk will be erased during the installation process.
To install the node for the first time:
NODE_IMAGE.img.gzbe the name of the image file.
dmesg | tail. Look for a line like
[sdb] Attached SCSI removable diskat the end: the
sdXname between brackets is that of the drive.
mount | cut -f1 -d' ' | grep ^/dev/sdX | xargs -L1 sudo umount(replace
sdXwith the drive's name).
zcat NODE_IMAGE.img.gz | sudo dd of=/dev/sdX bs=1M; sync(replace
sdXwith the drive's name).
/dev/sda(first internal hard disk) and reboot.
yand Enter, or wait for a minute.
nand Enter. You will be left in a console shell, from there run
sdXis the name of the right disk) and accept the selection.
The installation will proceed and the node will reboot. At this point you may remove the USB drive. The node will still reboot several times more, so give it around 10 minutes to ensure it has finished. If messages in the screen remain still for more than a couple of minutes, you may press Enter, run
mount and see that
/home is mounted to check that the installation went well.
/home is not mounted, maybe your controller did not configure the generated image to partition the node's disk automatically (VCT and Community-Lab controllers do); in this case run
confine.disk-parted to create the missing partitions and
reboot if the script asks you to. The node may reboot itself once more yet. Enter the console shell when the messages stop once more.
Finally, shut the node down by running
halt in the shell and wait for the message
System halted. to turn it off. Now you may move the node to its final location, connect it to the local network and turn it on again.
Once a node has been installed (see Installing a node via USB), it must be put into production for it to be able to deploy slivers.
OFFLINEthere may be problems with network connectivity.
DEBUGthe node may have an invalid configuration.
SAFEreported by the node (either in the node's state page or in the node list), go to the node's page, change its set state to
PRODUCTIONand click on Save. This enables the node to deploy slivers.
PRODUCTION, its current state may still read
SAFE. You may click on it to check in the node state whether it has already seen the change in the server.
Whenever you plan to perform some maintenance work on the node, or you plan to turn it off or expect bad connectivity for a while, it is strongly advised that you put the node's set state back to
SAFE until the incidence is over. If the node crashed due to some hardware failure, you may warn others by explicitly putting a
FAILURE set state. See Node states for more information.
To know the state of a node you may either:
The controller also offers a testbed status report summarizing node availability and software version per group. It can be accessed via Nodes / Summary in the main menu.
CONFINE nodes use to run an SSH service configured to accept some
root logins according to the settings provided when its firmware was configured (see Generating node firmware). Please note that this kind of access is used for managing the node itself; for entering a sliver in a node, see Logging into a sliver.
Usually as a node administrator in a node's group you should be able to use the SSH public keys configured as authentication tokens for your testbed account (see Changing user settings) for logging in nodes.
You need a
NODE_ADDRESS to provide to your SSH client to connect to the node as
root (see Addressing in CONFINE). For instance, with OpenSSH you may run
ssh root@NODE_ADDRESS. If you need to provide a different authentication key, you may use something like
ssh -i /path/to/id_rsa root@NODE_ADDRESS.
NODE_ADDRESS, as long as your computer is also configured as a host in that network (see Adding a host to the management network). The node's address is available under the Management network section in the node's page in the controller.
fdbd:e804:6aa9:1::2/64). First you need to add your computer to that network (e.g. with
ip addr add fdbd:e804:6aa9:1:2000::1234/64 dev eth0).
Please note that you can get the relevant addresses of a node by looking at the
/addrs member in the node's state (e.g. in the controller).
To add your computer to the unique recovery network:
MY_IFACE=eth0 UREC_PFX=fdbd:e804:6aa9:2 MY_SFX=$(ip a s dev eth0 | sed -rn 's#.* fe80::([^/]+)/64 .*#\1#p') ip addr add $UREC_PFX::$MY_SFX/64 dev $MY_IFACE
To detect the unique recovery addresses of nodes in your link (as long as their firmware is newer than 2014-05-20 and thus not affected by issue #444):
for host in $(ping6 -Lc2 ff02::1%$MY_IFACE | sed -ne "s/.* bytes from fe80::\([:0-9a-f]*\): .*/$UREC_PFX:\1/p" | sort -u); do ping6 -c2 -w2 $host > /dev/null && echo $host done
To remove your computer from the unique recovery network when done:
ip addr del $UREC_PFX::$MY_SFX/64 dev $MY_IFACE
You may use an SCP client to upload files to a node and download files from it, e.g. with OpenSSH you may run
scp FILES root@NODE_ADDRESS:/destination/path. However, please note that when using an IPv6
NODE_ADDRESS you should enclose it in brackets so that the colons do not confuse
scp. For instance, to upload a new firmware image to a node's temporary directory via its recovery address you may run
scp NODE_IMAGE.img.gz root@[fdbd:e804:6aa9:1::2]:/tmp.
The CONFINE node system software supports several ways of upgrading a node's firmware. Please remember that while the maintenance of the node is taking place, it is recommended to make its set state
SAFE in the controller (see Putting a node into production or maintenance). This will stop and probably undeploy slivers, so you may want to warn users running slivers in your node with anticipation.
The most common methods of upgrading a CONFINE node are summarized here, but you may want to see Node upgrade for further detail, additional possibilities and handling of special cases. Also, if you have a virtual node, please check vnode for alternative upgrade methods.
Te lightest upgrade method, it uses OpenWrt's package manager (OPKG) to install a newer version of the
confine-system package, available at the CONFINE package repository. Both procedures below will stop the CONFINE daemon, install the package and continue the daemon without losing already deployed slivers:
/tmp(see Exchanging files with a node).
confine_daemon_stop opkg --force-depends install /tmp/confine-system_VERSION_x86.ipk confine_daemon_continue
The simplest method, it downloads a full generic, non-customized node image from the CONFINE image repository, installs it over the old image while keeping the current configuration and deployed slivers, and reboots the node:
Shall you encounter any problems (e.g. lack of access to the repository or problems with partitions), try the manual method below.
A safer but slightly more complex method, it relies on having the new node image locally available in the node's file system.
GENERIC_IMAGEfile from the CONFINE image repository (usually
CONFINE-owrt-master-atom-current.img.gz) to your node's temporary directory
wget -P /tmp GENERIC_IMAGE_URLin the node.
GENERIC_IMAGE_URLto your computer and upload it to the node (see Exchanging files with a node).
confine.sysupgrade /tmp/GENERIC_IMAGE.img.gzin the node.
This should preserve all configuration and deployed slivers. If there are problems with the upgrade, you will need to perform a harder upgrade using a customized image (see Generating node firmware), which will remove all data in the node:
CUSTOM_IMAGEfile to the node's temporary directory
confine.sysupgrade -n /tmp/CUSTOM_IMAGE.img.gzin the node.
It is advisable to check that the
/overlay partitions have been created and mounted by running
mount in the node. Otherwise run
confine.disk-parted in the node and follow the instructions.
To keep testbed node clocks synchronized, we recommend that you run an NTP server in your management network and configure your node's
/etc/ntp.conf, if it has not been already done by the firmware generation process. Enter the node as
root (see Logging into a node) and run the following two commands:
cat > /etc/ntp.conf << EOF # Add as NTP server a host connected to the management network. restrict default ignore restrict 127.0.0.1 driftfile /var/lib/ntp/ntp.drift server NTP_SERVER_MGMT_ADDR prefer iburst EOF /etc/init.d/ntpd restart
NTP_SERVER_MGMT_ADDR is the management address of the NTP server (
fdf5:5351:1dfd:0:0:0:0:2 in Community-Lab).