Some first concepts where brainstormed and are archived on this page for reference purposes.
BB: compare to other benchmarking frameworks like the CREW testbed:
http://www.crew-project.eu/sites/default/files/CREW_D4%201_TCD_R_PU_2011-09-30_final.pdf http://www.crew-project.eu/sites/default/files/CREW_D4.3_JSI_R_PU_2012-03-01_v1.0_0.pdf http://www.crew-project.eu/sites/default/files/Benchmarking%20computers%20and%20computer%20networks.pdf
CB: If we want to use an OMF-based approach, much of the work already done and documented in D4.1
can be the basis for the benchmarking architecture.
CB: Tools for modifying the settings of the radio devices can be based on DLEP. This control feature is already in our development plan. IETF 87 - Berlin, Germany July 28 - August 2, 2013 has jump-started the standardization process again (hopefully)…
CB: For topology control, we suggested the an iptables/ebtables based approach for the VCT. For a real testbed, the rules will become more complex and you have to do a distributed configuration. Here, SDN approaches might also be helpful. With the integration of ns3 and VCT, topology can be generated completely in the ns3. This may also include the emulation of link failures, etc..
CB: The coexistence of an emulated environment and the real testbed might also help to validate the emulation or the parameter set used for network emulation.
CB: Topology for research interfaces should be restrictable to some of the links physically available (e.g. per ip-/ebtables).
BB: this completely corresponds to the OMF efforts by iMinds
CB: We have some first steps regarding a measurement infrastructure. I guess it is important to record traffic conditions during the experiments if perfect isolation can not be achieved.
BB: how does this relate to the monitoring by Navaneeth? AK: Monitoring by Navaneeth influences measurements.
CB: The spectrum monitors deployed in some nodes might also help in characterizing the environment the tests are performed in.
CB: Maybe the OMF collection mechanism for experiment data even fits into this topic. Some other tools e.g. from CREW are not reused, we will comment on this in the deliverables with a table of tools with their features and limits. Of course OMF will help and will be reused.