1. Sponsored by the National Science Foundation 1 GEC16: March 20, 2013 Are you ready for the tutorial? 1.Did you do the pre-work? A.Are you able to login to the GENI Portal? Using your home institution username and password, or Using a GPO issued username and password, or A temporary account B.Have the GEC16 Tutorial VM including both: VirtualBox GEC16 Tutorial VM image 2.Sign In to GENI Portal 3.Join Project: GEC16AdvNetw 4.Grab a Worksheet and Instructions

2. Sponsored by the National Science Foundation Tutorial: Advanced Network Experiments Niky Riga GENI Project Office Jay Aikat University, North Carolina GEC16, 20 March2013 $ omni.py createsliver aliceslice myRSpec.xml INFO:omni:Loading config file omni_config INFO:omni:Using control framework pgeni INFO:omni:Slice urn:publicid:IDN+pgeni.gpolab. expires within 1 day on 2011-07-07 INFO:omni:Creating sliver(s) from rspec file INFO:omni:Writing result of createsliver for INFO:omni:Writing to ‘aliceslice-manifest-rspe INFO:omni: ----------------------------------- INFO:omni: Completed createsliver: Options as run: aggregate: https://www.emulab. framework: pgeni native: True Args: createsliver aliceslice myRSpec.xml Result Summary: Slice urn:publicid:IDN+pgeni Reserved resources on https://www.emulab.net/p Saved createsliver results to aliceslice-man INFO:omni: =================================== omniomni

3. Sponsored by the National Science Foundation 3 GEC16: March 20, 2013 Tutorial Outline Part I: Learn how to use Omni Presentation and Hands-on Part II: Presentation of Advanced Experiments Tmix traffic generation Click Software Routers Part III: Hands-on Experiments Split in two groups: – Tmix – Non-IP routing using Click

4. Sponsored by the National Science Foundation Part I: Introduction to Omni Niky Riga, Sarah Edwards GENI Project Office $ omni.py createsliver aliceslice myRSpec.xml INFO:omni:Loading config file omni_config INFO:omni:Using control framework pgeni INFO:omni:Slice urn:publicid:IDN+pgeni.gpolab. expires within 1 day on 2011-07-07 INFO:omni:Creating sliver(s) from rspec file INFO:omni:Writing result of createsliver for INFO:omni:Writing to ‘aliceslice-manifest-rspe INFO:omni: ----------------------------------- INFO:omni: Completed createsliver: Options as run: aggregate: https://www.emulab. framework: pgeni native: True Args: createsliver aliceslice myRSpec.xml Result Summary: Slice urn:publicid:IDN+pgeni Reserved resources on https://www.emulab.net/p Saved createsliver results to aliceslice-man INFO:omni: ===================================

5. Sponsored by the National Science Foundation 5 GEC16: March 20, 2013 Hands On Exercise Do a Simple Client-Server Experiment Reserve two hosts using Omni and exchange traffic between them

6. Sponsored by the National Science Foundation 6 GEC16: March 20, 2013 New GENI Terminology AM API sliver credentials certificate

7. Sponsored by the National Science Foundation 7 GEC16: March 20, 2013 Experimenter GENI Credentials, GENI Certificates, GENI Keys GENI Certificate : identifies user; created by the Portal and you can download it through the webUI, it is an ssl certificate GENI Credential : identifies user's permissions; you almost never have to handle; tools query if from Portal Services GENI Keys : Access to compute resources are through ssh-keys, you can use your own or use the one GENI created for you Passphrase : is the password that protects your private key

8. Sponsored by the National Science Foundation 8 GEC16: March 20, 2013 Omni: Resource Reservation tool A command line experimenter tool Create slices and slivers using the GENI AM API Written in and scriptable from Python Use existing accounts –GENI Portal –ProtoGENI –PlanetLab Works with aggregates that implement the GENI AM API –ProtoGENI, PlanetLab, OpenFlow, InstaGENI, ExoGENI $ omni.py createsliver aliceslice myRSpec.xml INFO:omni:Loading config file omni_config INFO:omni:Using control framework pgeni INFO:omni:Slice urn:publicid:IDN+pgeni.gpolab. expires within 1 day on 2011-07-07 INFO:omni:Creating sliver(s) from rspec file INFO:omni:Writing result of createsliver for INFO:omni:Writing to ‘aliceslice-manifest-rspe INFO:omni: ----------------------------------- INFO:omni: Completed createsliver: Options as run: aggregate: https://www.emulab. framework: pgeni native: True Args: createsliver aliceslice myRSpec.xml Result Summary: Slice urn:publicid:IDN+pgeni Reserved resources on https://www.emulab.net/p Saved createsliver results to aliceslice-man INFO:omni: =================================== http://trac.gpolab.bbn.com/gcf/wiki/Omni

9. Sponsored by the National Science Foundation 9 GEC16: March 20, 2013 Omni Command Workflow createslicelistmyslicesgetversionrenewslicecreatesliverlistresourcessliverstatusdeletesliver Repeat for each aggregate Repeat for each aggregate Create Slice Create Sliver Cleanup Legend: AM API command

10. Sponsored by the National Science Foundation 10 GEC16: March 20, 2013 omni_configure.py [omni] default_cf = portal users = gpousr21 # ---------- Users ---------- [gpousr21] urn = urn:publicid:IDN+pgeni.gpolab.bbn.com+u ser+gpousr21 keys = ~/.ssh/geni_key... You only need the Omni bundle from the portal Standard AM nicknames Run this script for standard configuration: 1.Download GENI bundle 2.Run the script

11. Sponsored by the National Science Foundation 11 GEC16: March 20, 2013 omni_config [omni] default_cf = pgeni # Tutorial accounts are on GPO’s PG users = gpousr21 # gpousr21’s keys loaded on the VM to allow login # ---------- Users ---------- [gpousr21] urn = urn:publicid:IDN+pgeni.gpolab.bbn.com+user+gpousr21 # Really important to get the keys correct!!! #key to load on VM keys = ~/Tutorials/Omni/gpousr20/ssh/gpousr20_key.pub # default aggregates to run omni commands on aggregates = http://emulab.net/protogeni/xmlrpc/am, https://pgeni.gpolab.bbn.com/protogeni/xmlrpc/am,...

12. Sponsored by the National Science Foundation 12 GEC16: March 20, 2013 Getting Help omni.py –h Lists all commands and their arguments Lists all command line options Lists Omni version Lists url to find out more information about Omni Omni Troubleshooting page: http://trac.gpolab.bbn.com/gcf/wiki/OmniTroubleShoot http://trac.gpolab.bbn.com/gcf/wiki/OmniTroubleShoot

13. Sponsored by the National Science Foundation 13 GEC16: March 20, 2013 Before we start Many people will be accessing the resources, so some calls might fail. Wait a bit and try again! Omni is a command line tool, copy-paste is your friend You can copy-paste between your computer and the VM.

14. Sponsored by the National Science Foundation 14 GEC16: March 20, 2013 Part I: Design/Setup –Step 1: Configure Omni –Step 2: Obtain Resources Part II: Execute –Execute Experiment –View results Part III: Finish –Teardown Experiment

15. Sponsored by the National Science Foundation 15 GEC16: March 20, 2013 Configure omni geni@NSDI13-Tutorials:~$> omni-configure.py –f portal

16. Sponsored by the National Science Foundation 16 GEC16: March 20, 2013 Part I: Design/Setup –Step 1: Configure Omni –Step 2: Obtain Resources Part II: Execute –Execute Experiment –View results Part III: Finish –Teardown Experiment

17. Sponsored by the National Science Foundation 17 GEC16: March 20, 2013 Experimenter For the Tutorial: Keep it simple Slice Sliver AM ProtoGENI Aggregate You Once you understand how to run this simple experiment, scaling up is easy! GENI Portal

18. Sponsored by the National Science Foundation 18 GEC16: March 20, 2013 Request Specifications Resource Specification Document (RSpec) XML document that describes resources –hosts, links, switches, etc GENI v3 AM extensions Client extensions Tool extensions

19. Sponsored by the National Science Foundation 19 GEC16: March 20, 2013 The Three Types of RSpecs Aggregate Manager Client Advertisement Request Manifest Advertisement RSpec : What does the AM have? Request RSpec : What does the Experimenter want? Manifest RSpec: What does the Experimenter have? omni.py listresources slicename omni.py listresources omni.py createsliver slicename reqRSpec

20. Sponsored by the National Science Foundation 20 GEC16: March 20, 2013 Omni Commands omni.py getversion omni.py createslice slicename omni.py renewslice slicename date omni.py listmyslices username omni.py createsliver slicename requestRSpec omni.py sliverstatus slicename omni.py listresources [slicename] omni.py deletesliver slicename

21. Sponsored by the National Science Foundation 21 GEC16: March 20, 2013 Experiment Workflow Part I: Design/Setup –Step 1: Establish Management Environment –Step 2: Obtain Resources Part II: Execute –Run Experiment Part III: Finish –Teardown Experiment

22. Sponsored by the National Science Foundation 22 GEC16: March 20, 2013 Scripting Omni You can write custom Python scripts –Call existing Omni functions –Parse the Output Example: readyToLogin.py –Calls getversion, sliverstatus, listresources –Parses output –Determines ssh command to log into node –Print ssh configuration blocks to place in your ssh config file More examples distributed with Omni –remote-execute.py

23. Sponsored by the National Science Foundation 23 GEC16: March 20, 2013 Run Experiment geni@NSDI13-Tutorials:~$> readyToLogin.py omni99 –a pg-utah xterm -e ssh -i /home/geni/.ssh/ geni_key nriga@pc204.emulab.net &

24. Sponsored by the National Science Foundation 24 GEC16: March 20, 2013 Experiment Workflow Part I: Design/Setup –Step 1: Establish Management Environment –Step 2: Obtain Resources Part II: Execute –Execute Experiment Part III: Finish –Teardown Experiment

25. Sponsored by the National Science Foundation 25 GEC16: March 20, 2013 Part III: Finish Experiment When your experiment is done, you should always release your resources. –Normally this is when you would archive your data –Delete your slivers at each aggregate slice project aggregate RSpec user resource sliver AM API sliver credentials certificate

26. Sponsored by the National Science Foundation 26 GEC16: March 20, 2013 Omni: Command Line Tool Written in Python Easy to automate experiments Helps with simple orchestration through omni scripting Talks to all AMs that speak the GENI AM API ✗ Very manual ✗ Needs RSpecs $ omni.py createsliver aliceslice myRSpec.xml INFO:omni:Loading config file omni_config INFO:omni:Using control framework pgeni INFO:omni:Slice urn:publicid:IDN+pgeni.gpolab. expires within 1 day on 2011-07-07 INFO:omni:Creating sliver(s) from rspec file INFO:omni:Writing result of createsliver for INFO:omni:Writing to ‘aliceslice-manifest-rspe INFO:omni: ----------------------------------- INFO:omni: Completed createsliver: Options as run: aggregate: https://www.emulab. framework: pgeni native: True Args: createsliver aliceslice myRSpec.xml Result Summary: Slice urn:publicid:IDN+pgeni Reserved resources on https://www.emulab.net/p Saved createsliver results to aliceslice-man INFO:omni: ===================================

27. Sponsored by the National Science Foundation Part II: TMix Traffic Generator Jay Aikat University of North Carolina, Chapel Hill

28. Sponsored by the National Science Foundation Part II: Click Software Routers Niky Riga GENI Project Office

29. Sponsored by the National Science Foundation 29 GEC16: March 20, 2013 Software Routers Not wire-speed Easy to deploy Provide ability to implement new routing protocols over a L2 network fast –MobilityFirst, Packet Cache, XIA An easy way to bandwidth limit your slice

30. Sponsored by the National Science Foundation 30 GEC16: March 20, 2013 Access of multiple users in resources Omni allows multiple users to access the same resources –Modification of omni_config file –Different ways for each compute aggregate Alice Bob Alice’s Slice ProtoGENI MyPLC

31. Sponsored by the National Science Foundation 31 GEC16: March 20, 2013 Install scripts Specify in the rspec –Where to download software from Usually comes with an install-script.sh –What commands to run at boot time It runs in the context of a the “geni” user –It is not your user account –It does have sudo priviledges

32. Sponsored by the National Science Foundation 32 GEC16: March 20, 2013 Install scripts Automatic installation of software

33. Sponsored by the National Science Foundation 33 GEC16: March 20, 2013 Click Example Experiment You will run your own non-IP multipath router –Odd checksums left –Even checksums right 6 VMs in Utah –4 routers –2 end hosts

34. Sponsored by the National Science Foundation Part III: Hands-on Experiments

35. Sponsored by the National Science Foundation Part III: Click Example Experiment Niky Riga, Vic Thomas GENI Project Office

36. Sponsored by the National Science Foundation 36 GEC16: March 20, 2013 Hands On Exercise Run a non-IP router over L2 Reserve 6 hosts: 4 will be click routers, 2 will be end hosts. Odd checksums left Even checksums right

37. Sponsored by the National Science Foundation 37 GEC16: March 20, 2013 Part I: Design/Setup –Setup: Omni with Multiple Users –Obtain Resources: Create a slice and reserve resources Part II: Execute –Configure and Initialize Services: Configure the Click Routers –Execute Experiment: Use custom routing to forward traffic over multi-path topology Part III: Finish –Teardown Experiment

38. Sponsored by the National Science Foundation 38 GEC16: March 20, 2013 Multiple Users [omni] default_cf = portal users = alice, bob # ---------- Users ---------- [alice] urn = urn:publicid:IDN+pgeni.gpolab.bbn.com+us er+alice keys = ~/.ssh/alice_key [bob] urn = urn:publicid:IDN+pgeni.gpolab.bbn.com+us er+bob keys = ~/.ssh/bob_key ProtoGENI AMs [omni] default_cf = portal users = alice, bob # ---------- Users ---------- [alice] urn = urn:publicid:IDN+pgeni.gpolab.bbn.com+us er+alice keys = ~/.ssh/alice_key, ~/.ssh/bob_key SFA (PlanetLab) AMs Multiple accounts, each one with each own key One account accessible by many keys

39. Sponsored by the National Science Foundation 39 GEC16: March 20, 2013 Part I: Design/Setup –Setup: Omni with Multiple Users –Obtain Resources: Create a slice and reserve resources Part II: Execute –Configure and Initialize Services: Configure the Click Routers –Execute Experiment: Use custom routing to forward traffic over multi-path topology Part III: Finish –Teardown Experiment

40. Sponsored by the National Science Foundation 40 GEC16: March 20, 2013 Experimenter For the Tutorial: Portal Account with Omni Slice : click Sliver AM ProtoGENI Aggregate You GENI Portal $ omni.py createsliver aliceslice myRSpec.xml INFO:omni:Loading config file omni_config INFO:omni:Using control framework pgeni INFO:omni:Slice urn:publicid:IDN+pgeni.gpolab. expires within 1 day on 2011-07-07 INFO:omni:Creating sliver(s) from rspec file INFO:omni:Writing result of createsliver for INFO:omni:Writing to ‘aliceslice-manifest-rspe INFO:omni: ----------------------------------- INFO:omni: Completed createsliver: Options as run: aggregate: https://www.emulab. framework: pgeni native: True Args: createsliver aliceslice myRSpec.xml Result Summary: Slice urn:publicid:IDN+pgeni Reserved resources on https://www.emulab.net/p Saved createsliver results to aliceslice-man INFO:omni: =================================== omniomni http://www.gpolab.bbn.com/experiment-support/ ClickExampleExperiment/rspecs/click-XX.rspec

41. Sponsored by the National Science Foundation 41 GEC16: March 20, 2013 … in the meantime … Install Scripts specified in the rspec –It needs to be available over http does not run as “you”,but has sudo it runs at every boot $PATH might not be set OS-dependent 1.Write script in language of choice 2.Upload script 3.Test script 4.Add to rspec

42. Sponsored by the National Science Foundation 42 GEC16: March 20, 2013 Install Scripts: Look inside # 1. Change to dir. with installed software cd /local # 2. Check if first time if [ ! -f "./installed.txt" ] then #### Create the file #### sudo touch "./installed.txt” #### Run one-time commands #### #2a. Install necessary packages #2b. Install custom software #2c. Reboot if needed fi #3. Run Boot-time commands #3a. Start common services hn=`echo $HOSTNAME | cut -d'.' -f 1` if [ $hn == "server" ] then #4. Host specific commands fi 1. Download install script 2. Identify the 4 sections

43. Sponsored by the National Science Foundation 43 GEC16: March 20, 2013 Experiment Workflow Part I: Design/Setup –Setup: Omni with Multiple Users –Obtain Resources: Create a slice and reserve resources Part II: Execute –Configure and Initialize Services: Configure the Click Routers –Execute Experiment: Use custom routing to forward traffic over multi-path topology Part III: Finish –Teardown Experiment

44. Sponsored by the National Science Foundation 44 GEC16: March 20, 2013 Scripting Omni: remote-execute.py Script that helps you execute commands in your nodes from your computer Makes it easy to orchestrate experiments –Write scripts to orchestrate experiments Uses readyToLogin.py to get login information remote-execute.py –m [-h host]

45. Sponsored by the National Science Foundation 45 GEC16: March 20, 2013 Remote configuration of Click Remotely invoke configuration script : extractClickConfig.py –parse manifest rspec –extract information about topology –create click.config Remotely turn off IP in the routers

46. Sponsored by the National Science Foundation 46 GEC16: March 20, 2013 Experiment Workflow Part I: Design/Setup –Setup: Omni with Multiple Users –Obtain Resources: Create a slice and reserve resources Part II: Execute –Configure and Initialize Services: Configure the Click Routers –Execute Experiment: Use custom routing to forward traffic over multi-path topology Part III: Finish –Teardown Experiment

47. Sponsored by the National Science Foundation 47 GEC16: March 20, 2013 Send Traffic Start routers Send packet using netcat HostA Top Bottom Left Right HostB

48. Sponsored by the National Science Foundation 48 GEC16: March 20, 2013 Click Config File route :: Classifier(27/01%01,-); modify :: Unstrip(2) -> StoreData(0, "AliceWasHere3546") -> route; FromDevice(eth3, PROMISC true) -> Classifier(12/0800) -> modify; route[0] -> left :: EtherEncap(0x7744, 00:04:23:b7:14:76, 00:04:23:b7:18:fa) -> SimpleQueue -> Print(outL) -> ToDevice(eth2); route[1] -> right :: EtherEncap(0x7744, 00:04:23:b7:1c:e0, 00:04:23:b7:19:2e) -> SimpleQueue -> Print(outR) -> ToDevice(eth4); restore :: SimpleQueue -> Strip(30) -> EtherEncap(0x800, 00:04:23:b7:14:77, 00:04:23:b7:20:00) -> ToDevice(eth3);

49. Sponsored by the National Science Foundation 49 GEC16: March 20, 2013 Experiment Workflow Part I: Design/Setup –Setup: Omni with Multiple Users –Obtain Resources: Create a slice and reserve resources Part II: Execute –Configure and Initialize Services: Configure the Click Routers –Execute Experiment: Use custom routing to forward traffic over multi-path topology Part III: Finish –Teardown Experiment

50. Sponsored by the National Science Foundation 50 GEC16: March 20, 2013 Part III: Finish Experiment When your experiment is done, you should always release your resources. –Normally this is when you would archive your data –Delete your slivers at each aggregate slice project aggregate RSpec user resource sliver AM API sliver credentials certificate