Experiments involving wireless networks are difficult to run. We have a choice between using specialized platforms (like ORBIT, w-iLab.t, IoT-LAB, and others) or setting up some ad hoc testbeds. Specialized platforms make the experimentation task easier and offer good support for experiment repeatability. However, they only offer operating conditions that are far away from real-world deployments: nodes are usually distributed on a regular grid in one large room, and reproducibility in varying environmental conditions is very limited. As the performance of wireless networks strongly depends on their topology, environment, and operating conditions, validating protocols in artificial conditions may not lead to meaningful results, and reproducibility is de facto limited. Moreover, researchers do not have physical access to the equipment so debugging is tedious and some measurements like fine-grain energy consumption may be impossible. Ad hoc testbeds can help validating a given protocol, but they do not scale and it is difficult to reproduce results since most of the time a precise specification is missing.
We have developed WalT, a reproducible platform to run reproducible experiments. Our goal was threefold: (i) easily setup a platform to develop, debug and make preliminary validation, (ii) allow deployment and control of experiments on a larger scale, (iii) provide a way for others to deploy exactly the same experiment in a different environment, to challenge reproducibility and also guarantee a suitable starting point and adequate repeatability when comparing with other proposals. It follows an approach that lies somewhere in between specialized platforms with rigid and limited topology, and ad hoc testbeds. WalT nodes are single-board computers on which users can deploy their OS (filesystem, kernel) packaged as a docker image for easy customization and sharing.
A WalT platform provides:
WalT presents the following advantages:
Besides supporting reproducible experiments, WalT also enables the emergence of reproducible platforms-researchers can set up their WalT platforms to validate the results of others at different places or environments.
All WalT resources are publicly available: forge project, sources, and docker hub.