Sponsored by the National Science Foundation Systematic Experimentation, Automation, and Scaling Up Sarah Edwards, GENI Project Office

Sponsored by the National Science Foundation 2 GEC22 – March 23, 2015 Next you might be asking… How do I build repeatable experiments of non-trivial size?

Sponsored by the National Science Foundation 3 GEC22 – March 23, 2015 Our Advice for Novice Experimenters Creating Repeatable Computer Science and Networking Experiments on Shared, Public Testbeds by S. Edwards, X. Liu, N. Riga In Operating Systems Review, Jan ‘15 Concise advice for novices on designing experiments to run on a testbed: 1.Formulate a clear plan 2.Automate the execution/analysis of your experiment using best practices 3.Build scalable experiments a.k.a. Start small. Then scale up Plus a case study created by a PhD student. Link to ACM Library For students designing their own experiments

Sponsored by the National Science Foundation 4 GEC22 – March 23, 2015 #1 Formulate a clear plan #2 Automate the execution and analysis of your experiment and analysis Know last working configuration … change one thing at a time … … automate as you go AND save your work … #3 Build scalable experiments Start small... then scale up.

Sponsored by the National Science Foundation 5 GEC22 – March 23, 2015 Systematic Experimentation on a Testbed Combine best practices and methodology from: Science Scientific Method Experiment Design Repeatability Software Engineering Programming Version Control Debugging System Administration Using Node Types to Scale Up Configuration Management Briefly, focus on a concept from system administration that will help us be better scientists

Sponsored by the National Science Foundation 6 GEC22 – March 23, 2015 Automate Any sufficiently advanced technology is indistinguishable from magic. -- Arthur C. Clarke automation virtual machine self-configuring software router Auto-configuring XORP software router = xen VM + XORP image + configuration script

Sponsored by the National Science Foundation 7 GEC22 – March 23, 2015 Scaling Demo

Sponsored by the National Science Foundation 8 GEC22 – March 23, 2015 In Conclusion … Hyperbole & A Half &