User Tools

Site Tools


requirements:network-isolation

Network isolation

Code SREM-17
Responsible Axel Neumann
Components testbed node

Description

In Confine networks, distributed experiments can be executed on the same facilities as being used in parallel by the main CN (production network). Therefore it must be ensured that experiments do not interfere seriously with each other nor with the production network.

On the other hand, since Confine is aiming to enable community integrated network experiments, allowing some (controlled) level of interaction between the experiments and the main CN is also important.

This document describes the requirements for isolating and/or controlling sliver interactions over the shared network infrastructure (thus, sliver actions that go beyond the scope of a node).

In contrast, Slice isolation is focusing on resource consumption that has only node-local effects (limited to the scope of the node in which the sliver is operating).

Analysis

Overview

The required isolation is tightly coupled to the types of interactions with the CN that are necessary for an experiment. Following a pessimistic apporoach, all experiments must be considered evil. Network isolation is responsible to ensure that such interactions can not disrupt the production network infrastructure or other experiments.

A number of experiments have been discussed during first months of confine project and (partly) outlined in Experiments. The following four Subsections identify _interaction_types_ that are discussed independently in the remainder of this document. An experiment will usually participate in several interaction types simultaneously.

Controlling Resource consumption

All experimental network interactions must be controlled with respect to the bandwidth and airtime consumption which they demand from the shared network infrastructure. This interaction is discussed separately because:

  • The controlling of resource consumption can be done _without_ knowledge about logically affected networking layers and transmitted content (payload).
  • It is only about interactions which _are_ actively related to a specific slice. Therefore the only required parameter is the slice in charge of the traffic that is consuming network resources.

Controlling data collection

All experiments must be controlled to protect privacy and sensitive data of CN users. This interaction is discussed separately because:

  • The prevention of privacy violations may require network-layer specific handling and knowledge about transmitted content (payload and sensitive data).
  • From slice point of view it is a passive interaction.
  • It is (also) about access to network data which _is_not_ actively related to the data-collecting slice.

Isolating conflicting experiments

The production network and other concurrently operating experiments must be protected from experiments that are affecting common protocols behavior and resources like standard addresses and ports. This interaction is discussed separately because:

  • The isolation must be handled with respect to lowest affected networking layer.
  • From slice point of view it is an active interaction.
  • It is ONLY about interactions which _are_ actively related to a specific slice.

Enabling participation of Community Network (CN) users

Experiments may target on the interactions with CN users. This interaction is discussed separately because:

  • It is be about user interactions which maybe _only_indirectly_ related to the slice interested in user participation.

Resource Consumption

This Section discusses limitation of resource consumption (in terms of used bandwidth and airtime) by an active experiment.

It must be ensured that the amount of link capacity consumed by an experiment at any link in the network remains within limits so that other concurrently operating slices and the production network have a fair share.

Specific slices or the production network should be configurable with a higher priority to ensure minimal operation as long as possible.

Use cases:
  • stress-test experiments, eg udp bandwidth test over a multi-hop path.
  • Active, potentially disturbing or conflicting experiments (see Experiments)

Derived Requirements and Challenges

In theory, this limitation must be re-enforced (on a per slice basis) at every hop. Therefore it must be possible to identify the originating slice at each router in the network. In contrary, the type of conveyed data inside the packets and related networking layer is not relevant here.

Another challenge lies in the dynamically changing link capacities of wireless networks. …

Technical Approaches and Options

  • Traffic shaping tools (eg tc, openVSwitch) may be used to enforce an upper limit of bandwidth consumption per slice and link in the network.
  • Src/Dst IP addresses, vlan tags, or flow labels could be used for network-wide identification of the slice in charge of a specific packet. Tools like iptables, ebtables can be used to identify originating slice of a packet.

Open Questions

  • Define degree of serious interference (unacceptable disruption) to others
  • Define fair share

Data Collection

Researchers might be interested in real-time data from network interfaces. Such data could be interesting for later analysis or as real-time input for network protocols to cope with changing network and traffic characteristics.

Controlling data collection must be handled on a per-layer basis. It must be decided which type of information is considered sensitive and must be modified for anonymization or filtered.

For example MAC and IP addresses could reveal the person that has initiated a communication flow and thereby reveal location or usage patterns of individuals. However, if address fields would be anonymized, capturing usage of port numbers might be acceptable. On the other hand, real-time access to MAC address fields might be the main requirement for an experimental protocol where such information is only needed to trigger instant reactions and discarded immediately.

Deciding on appropiate policies for controlling data collection is also related to legal issues and privacy regulations in CN licenses. Feasibility, to implement appropiate control mechanisms, depends on desired policies and available anonymization techniques.

Use cases:
  • See column for “Lowest accessed layer” in Experiments

Derived Requirements and Challenges

  • Capturing, filtering, and modifying (eg for anonymization) interface data in real-time is performance intensive.
  • Ideally, to conform with privacy regulations of different CN licenses (peering agreements) and on the other hand allow researchers to access as much information as possible, different filtering and anonymization policies should be applicable.
  • In case of slivers based on virtualization (LXC), capturing data at any layer is possible with standard Linux tools and APIs from the host but difficult to:
    • Provide phy-header information to slivers via standard APIs (radiotap). For example phy and 80211-mac-header information can not be forwarded over a layer-2 bridge. But access via APIs would be favorable (eg for link-layer feedback to routing protocols).
    • Protect sensitive data. An interface is either in promiscuous mode (all packets can be captured including third-party payload) or not (only own traffic can be captured).

Technical Approaches and Options

  • Most restrictive privacy regulations could be applied by default.
  • Full access to interface data is granted only for non-virtualized slivers (slivers with full device access) and in isolated areas where privacy of others is not an issue.
  • Researchers must agree to a “code of conduct” which prohibits them to collect or analyze sensitive data.
  • Data with sensitive information is pre-processed offline (for anonymization) by confine system before being made publicly available.

Open Questions

  • Find tradeoff between accessible information, technical complexity, and users privacy.

Conflicting experiments

Confine testbed must allow experimentation with key network service (routing, DNS) and other protocol resources (address, port, channel assignments).

Isolation from production network and other experiments must be ensured depending on lowest accessed networking layer and used protocol resources.

Use cases:
  • See column for “Lowest impacted layer” in Experiments
  • Active, potentially disturbing or conflicting experiments (as described in Experiments)

Application Layer Experiments

Application-layer experiments based on udp/tcp communication can be considered non-conflicting if it can be ensured that IP addresses and port numbers used for this experiment are not conflicting with already allocated numbers for other key services. This is because communication between protocol instances is happening end-to-end and transparent for lower layers.

Derived Requirements and Challenges
  • No strict isolation required.
  • Filtering of outgoing traffic for illegal address or port usage
Possible Technical Approaches

Firewall techniques can be used to filter usage of reserved addresses or port numbers

Open Questions

Further discussion about restrictiveness of firewall policies is required. For example: Would an experimental DNS service provided over standard ports but via a specific IP address be considered “potentially disruptive” because a CN user may accidentially use this DNS service for name resolution?

Use cases:

Overlay / Peer-to-peer networking, Application usability evaluation, Application optimization, Delay Tolerant Networking, Content distribution

Transport Layer Experiments

Similar as Application Layer Experiments such experiments can be considered transparent for lower layers and require only modification on end devices.

Derived Requirements and Challenges
  • No strict isolation required.
  • Filtering of outgoing traffic for illegal address or port usage
Possible Technical Approaches

Technically, New transport protocols in Linux can be implemented with raw sockets or by registering a new protocol in linux kernel as a kernel module. raw socket access can be controlled in LXC with the drop.capabilities option

Open Questions

Same as for Application Layer Experiments.

Use cases:

Transport-Protocol Optimization and Stitching

Network Layer Experiments

Experiments targeting on modification of core networking services, protocols, and allocated addresses.

Derived Requirements and Challenges
  • Logical isolation required to lower layers because DPI or filtering of unknown behavior is too complex/impossible
Possible Technical Approaches
  • Software Define Networking (SDN) like techniques (eg vlan tagging, openVSwitch can be applied to ensure isolation to production network and other experiments
Open Questions
  • Evaluate complexity and compatibility of vlan tagging or rather new concepts like openSwitch, openVSwitch.
Use cases:

(Hybrid) Routing Protocols, Multi-Topology Routing, Routing Security, Link layer Feedback to Routing Protocols

MAC Layer Experiments

Experiments targeting on modification of core link-layer services, protocols, and allocated addresses.

Derived Requirements and Challenges
  • Full and exclusive access or Logical isolation required to lower layers because DPI or filtering of unknown behavior is too complex/impossible
Possible Technical Approaches
  • Full and exclusive access to dedicated network interface for a particular sliver
  • Full and exclusive access to full research device for a particular sliver (to allow loading of special kernel modules)
  • Logical isolation, (like above) SDN techniques can be used to logically isolate experiments from production network.

However, in this case the SDN solution is less clean (more dirty) because isolation is happening in the same networking layer as the experiments.

Open Questions
  • Prevention of privacy violations
Use cases:

Network Coding, Layer-2 Routing

PHY Layer Experiments

Derived Requirements and Challenges
  • Full and exclusive access to dedicated network interface or even to full research device because logic isolation is not possible.
Possible Technical Approaches
  • TBD…
Use cases:

Tx-power adaption, Tx-rate adaption

Participation of CN Users

Researchers might be interested in the participation of CN users in their experiments to:

  • explore user-acceptance
  • identify pitfalls and bugs
  • evaluate user behavior

Interaction with CN users can be achieved with different approaches:

  • Community Network wide by enabling routing to slivers via CN public IPs and existing CN infrastructure
    • Use case: Experiments classified as “Active, good netizens”

(eg: Test user-acceptance for an experimental file-sharing or distributed web-caching protocol).

  • Community Network wide by forcing routing via an overlay network setup up by the researcher (eg via DNS rewrites or proxies)
  • Local, via direct link access to slice from nearby environment of confine node.

Such option would allow CN user participation in experiments where normal routing via common CN routing protocols is explicitly NOT wanted.

  • Use case: Test user-acceptance for an experimental routing protocol or overlay network.

Derived Requirements and Challenges

  • Since slivers have access to core CN infrastructure, it must be ensured that sliver operations (via non-isolated links) conform with CN address and port allocations and expected protocol behavior.

Possible Technical Approaches

  • Restrictive firewall rules can be applied to prohibit specific protocol services or address or port usage via non-isolated links.
  • Network wide: by assigning CN public IPs to sliver and announcing IPs via routing protocols used in existing CN infrastructure.
  • Local access: Linux mac80211 allows creation of several virtual interfaces on top of one physical wireless interface. Slivers could be granted exclusive access to such virtual interface. This way, slivers could spawn their own wireless AP (hot-spot) and offer dhcp, mobility, or other service to users in nearby environment.

Open Questions

  • Define to what extend CN-user participation is realistic, appreciated, considered useful, supported.

Open discussions

Several open questions have been addressed in related Sections respectively.

Additionally:

  • What about overlay networks to connect isolated experiments in separate confine clouds (could be discussed in “CN participation”)?

Comments

Terms and Acronyms

Production Network:

CN: Community Network

CN users: person actively using CN infrastructure for networking services. Could be a passenger using a CN hot-spot.

CN infrastructure: used for community, isolated Confine cloud, and mixed (community and Confine cloud) network infrastructures

requirements/network-isolation.txt · Last modified: 2012/05/31 09:26 by axel