User Tools

Site Tools


benchmarking:firstbrainstorm

First brainstorm

Some first concepts where brainstormed and are archived on this page for reference purposes.

General Concept

BB: compare to other benchmarking frameworks like the CREW testbed:

http://www.crew-project.eu/sites/default/files/CREW_D4%201_TCD_R_PU_2011-09-30_final.pdf http://www.crew-project.eu/sites/default/files/CREW_D4.3_JSI_R_PU_2012-03-01_v1.0_0.pdf http://www.crew-project.eu/sites/default/files/Benchmarking%20computers%20and%20computer%20networks.pdf

(Source: http://www.crew-project.eu/documents)

CB: If we want to use an OMF-based approach, much of the work already done and documented in D4.1

  • 5.2. OVERHEAD ANALYSIS OF EMBEDDED WIRELESS TESTBEDS WITH OMF
  • 5.3. DEVELOPMENT OF A TEST ENVIRONMENT FOR NETWORK MEASUREMENT TOOLS BASED ON OMF

can be the basis for the benchmarking architecture.

CB: Tools for modifying the settings of the radio devices can be based on DLEP. This control feature is already in our development plan. IETF 87 - Berlin, Germany July 28 - August 2, 2013 has jump-started the standardization process again (hopefully)…

CB: For topology control, we suggested the an iptables/ebtables based approach for the VCT. For a real testbed, the rules will become more complex and you have to do a distributed configuration. Here, SDN approaches might also be helpful. With the integration of ns3 and VCT, topology can be generated completely in the ns3. This may also include the emulation of link failures, etc..

Testbed Validation

CB: The coexistence of an emulated environment and the real testbed might also help to validate the emulation or the parameter set used for network emulation.

Repeatability

CB: Topology for research interfaces should be restrictable to some of the links physically available (e.g. per ip-/ebtables).

BB: this completely corresponds to the OMF efforts by iMinds

Reproducibility

CB: The work on isolation is - at least in my opinion - directly related to benchmarking in terms of reproducibility. There is a section on network isolation and slice isolation in the wiki.

Verifiability

CB: We have some first steps regarding a measurement infrastructure. I guess it is important to record traffic conditions during the experiments if perfect isolation can not be achieved.

BB: how does this relate to the monitoring by Navaneeth? AK: Monitoring by Navaneeth influences measurements.

CB: The spectrum monitors deployed in some nodes might also help in characterizing the environment the tests are performed in.

Similar Research

CB: Maybe the OMF collection mechanism for experiment data even fits into this topic. Some other tools e.g. from CREW are not reused, we will comment on this in the deliverables with a table of tools with their features and limits. Of course OMF will help and will be reused.

benchmarking/firstbrainstorm.txt · Last modified: 2014/02/18 17:31 by barz