User Tools

Site Tools


soft:sliver-deployment

Sliver deployment

This is a specification for how to proceed in a node when a sliver is to be deployed (or updated). All the steps described in it happen entirely in the target node, and they only apply to container slivers (e.g. LXC-based but not KVM-based).

Overall deployment procedure

  1. Resources requested by the sliver are checked for their availability and allocated. This must be an atomic operation. The allocation of disc space is accomplished by creating a SLIVER_OVERLAY.img non-sparse file of the requested or some default size (e.g. with fallocate or plain writes, to allow the correct computation of free space).
  2. A new number for the sliver is computed and kept as sliver.nr. This must be an atomic operation.
  3. Addresses for sliver interfaces are assigned.
  4. The root filesystem for the sliver is created and prepared.

Failures happening before or during step 3 set sliver.state to fail_allocate, then deallocate resources and abort the procedure. Failures happening after step 3 set sliver.state to fail_deploy then cleanup, deallocate resources and abort the procedure. In both cases descriptive messages are included in sliver.errors for the appropriate members (or the empty member).

See Slice and sliver states for more information on the meaning of the previous states.

Sliver filesystem preparation

The creation and preparation of the sliver's filesystem consists of the following steps. Please note that steps marked with (i) are only performed when the sliver is deployed, while the others are performed both when deploying and starting a sliver:

  1. If not yet done, make the sliver template available at TEMPLATE (possibly read-only, e.g. mount its SquashFS image on TEMPLATE).
  2. (i) Create a filesystem in SLIVER_OVERLAY.img.
  3. Loop-mount SLIVER_OVERLAY.img on SLIVER_OVERLAY.
  4. Mount an overlay filesystem of TEMPLATE (ro) and SLIVER_OVERLAY (rw) on SLIVER_ROOT.
  5. (i) If the sliver includes a data file (possibly as a sliver default), download it by its URI to SLIVER_ROOT/<safe_tmpdir>/data. If the download fails (missing, timeout, forged, too big…), fail and report on /exp_data_(uri|sha256).
  6. (i) Run the sliver image preparation script (e.g. for deploying application data, setting host name, network configuration files…) with SLIVER_ROOT, SLIVER_ROOT/<safe_tmpdir> and SLIVER_OVERLAY as arguments. This script may be different for each template.type. If the script fails, get from it descriptive errors (e.g. via stderr) and fail.
  7. Remove SLIVER_ROOT/<safe_tmpdir>.
  8. Create the SLIVER_ROOT/confine directory if missing (remove previously if not a directory).
  9. (i) Create a user slice-%(SLICE_ID)012x in the RD (not the sliver) as explained in Out-of-band remote access to CONFINE slivers (warning, may be a little outdated).

Preparation script example

As an example for the usual template.type == debian or template.type == openwrt, the sliver image preparation script may handle SLIVER_ROOT/<safe_tmpdir>/data as an archive file (e.g. a gzipped tarball, a zipfile or a SquashFS image) and extract it to SLIVER_OVERLAY (in contrast, an OpenFlow-only template may handle the sliver data file as a text file with a list of rules to configure a controller). If the extraction fails (corrupt, too big…), the script fails and reports on /overlay_uri.

The archive may contain init scripts to automate the preparation and execution of applications, additional program and data files and directories, or overlay whiteouts to hide files and directories from TEMPLATE in SLIVER_ROOT.

The preparation script may use lxc-execute to quickly run configuration commands inside the not-yet-started container.

Warning: Preparation scripts must be extra careful since they operate on the sliver's root filesystem after the user-supplied sliver data has been applied. Thus using the root namespace is open to security risks (e.g. if the sliver's /etc/hostname has been replaced by a symbolic link to /etc/passwd or any other file), and using the sliver's namespace (e.g. with lxc-execute) does no longer guarantee that the executed program does what it is expected to. readlink -f NAME may be used to detect files which are actually outside of the sliver's root filesystem.

The "/confine" directory

This feature is not implemented as of the Confined milestone software release. Please take it as a specification.

For providing basic information to the sliver and a minimal, always available environment to users of Out-of-band remote access to CONFINE slivers, the RD prepares a read-only filesystem which is mounted read-only on SLIVER_ROOT/confine when the sliver is started. Its contents are:

/confine/node-id          # contains node ID
/confine/slice-id         # contains slice ID
/confine/bin/busybox      # statically linked
/confine/bin/scp          # statically linked
/confine/bin/sftp-server  # statically linked

The first two files help the sliver identify itself in order to gather more information from the node API served in the internal bridge (see Node architecture)) and therefore from the registry API.

The files under /bin provide the minimal environment for OoB access. Binaries are linked statically to avoid dependencies on libraries in the sliver filesystem which may get changed or deleted. The scp and sftp-server binaries provide support for SSH file transfer without depending on a working SSH installation in the sliver. With this, the shell for the slice-%(SLICE_ID)012x user in the RD could be similar to the following script:

#!/bin/sh
set -e
 
confine_bin=/confine/bin
exec_in_container() {
    local container
    container=$(container_from_user "$(whoami)")
    exec sudo -- lxc-attach -n "$container" -- "$confine_bin/busybox" sh "$@"
}
 
arg1="$1"
arg2dn="$(dirname "$2")"
arg2bn="$(basename "$2")"
if [ "$arg2dn" != "$confine_bin" -a "$arg1" = '-c' \
     -a \( "$arg2bn" = scp -o "$arg2bn" = sftp-server \) ]; then
    shift 2  # replace path of second argument to use CONFINE binary
    exec_in_container "$arg1" "$confine_bin/$arg2bn" "$@"
else
    exec_in_container "$@"
fi
soft/sliver-deployment.txt · Last modified: 2015/09/16 10:29 by ivilata