User Tools

Site Tools


soft:oob-access-slivers

Out-of-band remote access to CONFINE slivers

This feature is not implemented as of the Confined milestone software release. Please take it as a specification. See The "/confine" directory for more details on the implementation.

Introduction

A slice administrator running a CONFINE slice needs to have remote access to any of its slivers through the community network even when the sliver has no connection to it because none has been set up or the connection has been severed on purpose or accidentally. This access includes both being able to log into a terminal to run programs, and to transfer files to and from the sliver to upload applications and collect results, all with proper slice administrator authentication.

This document introduces an approach to out-of-band remote access to CONFINE slivers which lets the slice administrator access them via SSH through the research device (RD). The slice administrator connects to SLICE_ID@CONTROL_IP[:CONTROL_PORT], where SLICE_ID is an identifier for the slice in the node (preferably the same in all nodes running the slice) and CONTROL_IP[:CONTROL_PORT] is the IP address and TCP port for the SSH daemon providing out-of-band access to slivers. This daemon is considered to be part of the node's control software, thus the port is only needed if the research device is behind a community device performing NAT and not forwarding port 22.

The approach supports the following SSH-based interfaces:

$ ssh [-p CONTROL_PORT] SLICE_ID@CONTROL_IP
$ scp [-P CONTROL_PORT] SLICE_ID@CONTROL_IP:tmp/results.log .
$ sftp [-P CONTROL_PORT] SLICE_ID@CONTROL_IP:tmp
$ sshfs [-p CONTROL_PORT] SLICE_ID@CONTROL_IP:tmp /mount/point

Sliver users

The RD has a sliver group whose users can run certain LXC commands as root using sudo. When a sliver is deployed in the node:

  1. A new sliver user is created in the RD. The user is called SLICE_ID as described above and it belongs both to its own group SLICE_ID and to the sliver group, its shell is /bin/sh and its password is disabled.
  2. The sliver user's home directory consists of:
    • ./ owned by root:SLICE_ID with mode rwxr-x—.
    • .ssh/ owned by root:SLICE_ID with mode rwxr-x—.
    • .ssh/authorized_keys owned by root:SLICE_ID with mode rw-r—–.
    • tmp/ owned by root:SLICE_ID with mode rwxr-x—.
  3. SSH authorized keys for the sliver user are set up (see below).
  4. The sliver's temporary directory is bind-mounted on ~SLICE_ID/tmp. The directory is also mounted when the RD boots with the sliver already deployed.

When the sliver is removed from the node:

  1. The directory ~SLICE_ID/tmp is unmounted. The directory is also unmounted when the RD halts or reboots.
  2. The SLICE_ID user (and group) is removed together with its home directory.

Sliver access

The previous setup plus the proper SSH authorized keys (see below) allows slice administrators in the slice to read and write files (since the mounted tmp is usually rwxrwxrwt) on each sliver's temporary directory. At the same time, it forbids the sliver user and other users from tampering with their files, including storing files in the home directory or tmp (even when it is not mounted). Since it allows to write files when the sliver is not yet running, it may be used to upload different configuration files to be read on boot by each sliver.

When a sliver user is created, its ~/.ssh/authorized_keys file is populated with the SSH public keys of every slice administrator allowed to access the slice (thus supporting slice administrator authentication). When opening a plain SSH connection to the RD, slice administrators are only allowed to attach to the sliver's console. This is obtained by placing the following options at the beginning of each key:

command="sliver-console",no-port-forwarding

The sliver-console attaches to one of the container's virtual console using lxc-console:

#!/bin/sh
exec sudo -- lxc-console -n "$(whoami)"

The slice administrator can use the console until the detach combination is pressed (Ctrl-a q by default).

Conclusions

We've seen an approach to implement support for out-of-band access to slivers in a node using a familiar SSH interface, not requiring slivers to have network connection nor any specific service or support (in fact file transfer doesn't even need the sliver to be running). Also, the proposed solution has no additional requirements besides an SSH server and the LXC tools.

Some open questions common to both approaches remain, though:

  • The support of key options in authorized_keys files in the RD may require the use of OpenSSH or a recent or custom built version of the Dropbear SSH server.
  • Slivers should be careful not to mount anything over /tmp (anyway this requires additional container capabilities). The RD can ensure that slivers have a /tmp directory with adequate permissions. Otherwise, a dedicated directory can be used, or it can be configured per slice or per sliver.
  • Since SSH access to a sliver's temporary directory happens as the sliver user in the RD, files uploaded appear in the sliver as belonging to the user with its numerical ids. The slice administrator should be careful to adjust permissions so that these files are readable in the sliver.
  • A full-blown sliver console isn't available in any case. An option is to run lxc-start under a GNU Screen session (as in Linode's Lish). However Screen is a very flexible tool, so this could have security implications.
  • However, a sliver's boot messages can be captured by redirecting its console to a file using lxc-start's -c option, but the file can grow too much if many messages are produced. Alternatively, the sliver may be configured to run bootlogd or some similar solution.
  • Last but not least, SFTP access allows the slice administrator to browse the whole RD file system. Although restricted to its user's permissions, this still allows things like listing the SLICE_ID of the other slivers in the node and saving files in global temporary directories, possibly bypassing the sliver's disk quota. Thus permissions in the RD (especially under /var/lib/lxc) should be carefully selected, and the setfacl tool should be used to remove permissions from the sliver group on certain directories, but a file system supporting ACLs is needed. Alternatively, SSH access for sliver users can be chrooted. OpenSSH supports this (as explained here), but unfortunately the shell session is also chrooted. However, file transfers can be disabled for the sliver group in the SSH daemon and ProFTPD can be set up (on a different port) to provide chrooted SFTP support.
  • A more elegant solution to the previous problem is for sliver users' SSH connections to spawn a shell inside the container so that shell sessions, commands and file access operate directly in the sliver environment (in the likes of PlanetLab's vsh user shell). This requires that the sliver includes shell, scp and sftp-server binaries and that the sliver is running. Also, this uses lxc-attach, which requires a series of patches not yet into mainstream as of Linux 3.2. This solution does away with the command option in SSH keys and it relies on a user shell like this instead of /bin/sh:
#!/bin/sh
exec sudo -- lxc-attach -n "$(whoami)" -- /bin/bash "$@"
soft/oob-access-slivers.txt · Last modified: 2015/09/16 10:29 by ivilata