1. Sponsored by the National Science Foundation Systematic Experimentation in GENI

2. Sponsored by the National Science Foundation 2 Systematic Experimentation – GEC 20 – June 23, 2014 Agenda Motivation Sarah Edwards, Xuan Liu, Niky Riga, Vic Thomas, GPO Case Study: Virtual Router Failure Restoration Xuan Liu, UMKC/GPO Tools and Techniques Custom images and install scripts InstaGENI, Gary Wong, University of Utah ExoGENI, Paul Ruth, RENCI Porting ExoGENI Images to InstaGENI Ezra Kissel, Indiana University Manipulating RSpecs with geni-lib Nick Bastin, Barnstormer Softworks How to Collaborate Sarah Edwards, GPO Q&A Experimenter Drop-In (next session)

3. Sponsored by the National Science Foundation Systematic Experimentation in GENI: Motivation Sarah Edwards, Xuan Liu, Niky Riga, Vic Thomas GENI Project Office

4. Sponsored by the National Science Foundation 4 Systematic Experimentation – GEC 20 – June 23, 2014 GENI-FIRE 150 node topology Courtesy of Thierry Rakotoarivelo, NICTA, GEC 20 Demo Clear plan Automation Scale and Visualization

5. Sponsored by the National Science Foundation 5 Systematic Experimentation – GEC 20 – June 23, 2014 Virtual Computer Networking Lab Courtesy of Cong Wang and Divya Bhat, UMass Amherst, GEC 20 Demo Start small

6. Sponsored by the National Science Foundation 6 Systematic Experimentation – GEC 20 – June 23, 2014 Intelligent Data Movement System Courtesy of Ezra Kissel, Indiana University, GEC 20 Demo Long-lived slice for stitched, shared VLAN Long-lived slice for stitched, shared VLAN Dynamically add/remove nodes as needed

7. Sponsored by the National Science Foundation 7 Systematic Experimentation – GEC 20 – June 23, 2014 Why doesn’t this work? (Anonymous) Past Bug Reports repeatable incremental test controller in one AM? test controller in one AM? GRE works Stitch test controller in one AM test controller in one AM Multi-site Hadoop Used control plane to send high volume data between racks Interfere with rack ! OpenFlow multi-site OpenFlow multi-site OpenFlow multi-site OpenFlow multi-site

8. Sponsored by the National Science Foundation 8 Systematic Experimentation – GEC 20 – June 23, 2014 Systematic Experimentation Running an experiment on a testbed requires best practices and methodology from a combination of: –Science (Scientific method) –Programming –System Administration Today, talking about strategies and techniques for bringing up experiments in a systematic way.

9. Sponsored by the National Science Foundation 9 Systematic Experimentation – GEC 20 – June 23, 2014 Why experiment systematically? Debugging Easier to debug experiments. Easier to get help debugging. Especially for complex experiments. Scalability and Repeatability A well designed experiment will be repeatable and systematically scalable Valid Develop a solid understanding of and validate your experimental setup Rigorous Perform a scientifically rigorous (and publishable) study

10. Sponsored by the National Science Foundation 10 Systematic Experimentation – GEC 20 – June 23, 2014 Terminology for today’s talk Study –A series of experiments demonstrating a result. –A publishable result. Experiment –Multiple runs of a fixed topology, procedure, etc Run –A single iteration of an experimental procedure

11. Sponsored by the National Science Foundation 11 Systematic Experimentation – GEC 20 – June 23, 2014 General Principle #1: One thing at a time Experimentation 101 and Debugging 101 Only change one thing at a time Examples: Software/image Configuration Number of nodes/links Geographic distribution of nodes/links

12. Sponsored by the National Science Foundation 12 Systematic Experimentation – GEC 20 – June 23, 2014 General Principle #2: Start Small Start by building smallest possible topology by hand. Automate as needed. Test and measure as you go. clientserver host OVS switch host router worker master worker node

13. Sponsored by the National Science Foundation 13 Systematic Experimentation – GEC 20 – June 23, 2014 General Principle #3: Save what you do Log all of your experimental artifacts for every experiment that works –RSpec –image –install script –custom software –measurements –etc Use version control to store your artifacts Always know the last configuration that worked

14. Sponsored by the National Science Foundation 14 Systematic Experimentation – GEC 20 – June 23, 2014 Experiment Workflow Summary Workflow How often should I loop?

15. Sponsored by the National Science Foundation 15 Systematic Experimentation – GEC 20 – June 23, 2014 Process Recommendation A. Build (smallest possible) topology by hand at a single aggregate B. Automate topology creation at a single aggregate C. Orchestrate and Instrument D. Increase scale E. More nodes F. More aggregates Automate

16. Sponsored by the National Science Foundation 16 Systematic Experimentation – GEC 20 – June 23, 2014 Process: Start Small A. Build smallest possible topology by hand at a single aggregate B. Automate topology creation at a single aggregate –Store RSpecs, software, etc in version control –TOOLS Use install scripts and custom images –Determine how to update software on your nodes Repeat A/B as needed. C. Orchestrate and Instrument your experiment procedure TOOLS GENI Desktop and LabWiki Automate as needed. worker master worker node

17. Sponsored by the National Science Foundation 17 Systematic Experimentation – GEC 20 – June 23, 2014 Process: Scale D. Increase scale (number of nodes, volume of traffic, etc) TOOLS Flack for small scale. geni-lib for larger scale client server client server

18. Sponsored by the National Science Foundation 18 Systematic Experimentation – GEC 20 – June 23, 2014 Process: Split across aggregates E. Split an existing topology across second aggregate –How connect across aggregates matters (stitching vs OF etc) TOOLS stitcher for VLAN stitching and GRE tunnels. ExoSM for Intra-ExoGENI stitching. Flack for GRE tunnels. F. Split across multiple aggregates client server client Colors represent different aggregates Inter-aggregate link

19. Sponsored by the National Science Foundation 19 Systematic Experimentation – GEC 20 – June 23, 2014 Avoiding Common Pitfalls Don’t start with a large or inter-aggregate topology In general, if you are keeping your experiment up because you can’t recreate your setup, then something is wrong. GENI is not a simulator Common Issue: OpenFlow controllers used on an multi-aggregate topology with loop MUST be tested in a single aggregate topology with loop

20. Sponsored by the National Science Foundation 20 Systematic Experimentation – GEC 20 – June 23, 2014 Case Study

21. Sponsored by the National Science Foundation 21 Systematic Experimentation – GEC 20 – June 23, 2014 Process Summary A. Build (smallest possible) topology by hand at a single aggregate B. Automate topology creation at a single aggregate C. Orchestrate and Instrument D. Increase scale E. More nodes F. More aggregates Automate

22. Sponsored by the National Science Foundation 22 Systematic Experimentation – GEC 20 – June 23, 2014 Overview of Tools and Techniques Automation –Custom images on InstaGENI/ExoGENI –Converting ExoGENI images to InstaGENI –Install script on InstaGENI –Postboot script on ExoGENI –Measurement with LabWiki and GENI Desktop –Repeating and sharing experiments with apt Scaling Up –geni-lib Inter-aggregate experiments –stitcher –OpenFlow TutorialsNow Drop-InNowTutorial Now

23. Sponsored by the National Science Foundation 23 Systematic Experimentation – GEC 20 – June 23, 2014 Agenda Motivation Sarah Edwards, Xuan Liu, Niky Riga, Vic Thomas, GPO Case Study: Virtual Router Failure Restoration Xuan Liu, UMKC/GPO Tools and Techniques Custom images and install scripts InstaGENI, Gary Wong, University of Utah ExoGENI, Paul Ruth, RENCI Porting ExoGENI Images to InstaGENI Ezra Kissel, Indiana University Manipulating RSpecs with geni-lib Nick Bastin, Barnstormer Softworks How to Collaborate Sarah Edwards, GPO Q&A Experimenter Drop-In (next session)

24. Sponsored by the National Science Foundation How to Collaborate

25. Sponsored by the National Science Foundation 25 Systematic Experimentation – GEC 20 – June 23, 2014 Project Membership example Projects have 1 Lead and any number of Admins, Members, and Auditors http://groups.geni.net/geni/wiki/GENIConcepts#Project Typical Class Expiration Typical Research Project

26. Sponsored by the National Science Foundation 26 Systematic Experimentation – GEC 20 – June 23, 2014 Working with multiple members in a slice Research Asst Slice Lead Post-Doc Slice Member Professor Slice Admin Members of all slices in a project: Project Leads (Professor) Project Admins (Post-docs, researchers) Other can be added manually http://groups.geni.net/geni/wiki/GENIConcepts#Slice

27. Sponsored by the National Science Foundation 27 Systematic Experimentation – GEC 20 – June 23, 2014 Slice Access Being a member of a slice means you can act on a slice: –Add resources –Check status –Delete resources –Renew resources With any tool!

28. Sponsored by the National Science Foundation 28 Systematic Experimentation – GEC 20 – June 23, 2014 To ensure access in collaborator’s resources: Option 1: Make resource reservation from Portal or omni Option 3: Use a common public key Add slice member’s accounts to existing resources: $ omni -V 3 poa SLICE geni_update_users --useSliceAggregates –-useSliceMembers Alternatively… $ addMemberToSliceAndSlivers myslice username Slice Access: Logging in to resources Slice membership does not guarantee ability to login to resources! Only works on InstaGENI/ProtoGENI

29. Sponsored by the National Science Foundation 29 Systematic Experimentation – GEC 20 – June 23, 2014 Questions?

30. Sponsored by the National Science Foundation 30 Systematic Experimentation – GEC 20 – June 23, 2014 Special Topic Experimenter Drop-In In this room after the break Writing ExoGENI velocity scripts New ExoGENI image snapshotting feature Converting ExoGENI images to InstaGENI images Manipulating RSpecs with `geni-lib` Sharing experiments with `apt` Thanks to: –Paul Ruth, RENCI –Nick Bastin, Barnstormer Networks –Rob Ricci, University of Utah