Sponsored by the National Science Foundation Tutorial: OpenFlow in GENI with Instrumentation and Monitoring Divya Bhat Shufeng Huang Niky Riga GENI Project Office GEC17

Sponsored by the National Science Foundation 2 GEC17 GENI Programmable Network Key GENI concept: slices & deep programmability –Internet: open innovation in application programs –GENI: open innovation deep into the network Good old Internet Slice 0 Slice 1 Slice 2 Slice 3 Slice 4 Slice 1 OpenFlow switches one of the ways GENI is providing deep programmability

Sponsored by the National Science Foundation 3 GEC17 Racks and Campuses GENI Rack projects are expanding available GENI infrastructure in the US. Racks provide reservable, sliceable compute and network resources using Aggregate Managers. GENI AM API compliance

Sponsored by the National Science Foundation 4 GEC17 GENI Rack Campuses 43 racks planned this year Each rack has an OpenFlow-enabled switch

Sponsored by the National Science Foundation 5 GEC17 Core Networks NLR committed to 2013 meso-scale expansion following reorganization Internet2 adding 10GbE paths to Advanced Layer 2 Services (AL2S) at 4 of 5 OpenFlow meso-scale/ProtoGENI Pops GENI Aggregate Manager in Internet2 AL2S and dynamic stitching with GENI coming in Spiral 5 Internet2 SDN networks

Sponsored by the National Science Foundation 6 GEC17 FOAM An OpenFlow Aggregate Manager It’s a GENI compliant reservation service –Helps experimenters reserve flowspace in the FlowVisor Speaks AM API v1 Rspecs GENI v3, OpenFlow v3 extension

Sponsored by the National Science Foundation 7 GEC17 OpenFlow Experiments Debugging OpenFlow experiments is hard: –Network configuration debugging requires coordination –Many networking elements in play –No console access to the switch Before deploying your OpenFlow experiment test your controller.

Sponsored by the National Science Foundation 8 GEC17 OpenFlow common PitFalls Controller is responsible for all traffic, not just your application! –ARP, DHCP, LLDP Reactive controllers –UDP Performance in hardware switches –Not all actions are supported in hardware No STP –Broadcast storms

Sponsored by the National Science Foundation 9 GEC17 Part I: Design/Setup –Network Load Balancing and OpenFlow –Design your experiment –Obtain Resources Part II: Execute –Configure and Initialize Services –Execute Experiment –Collect Measurements Part III: Finish –Archive results –Teardown Experiment

Sponsored by the National Science Foundation 10 GEC17 Balanced Service Picture from : Balance load between two or more server providers In networks Balance traffic between two or more paths Need to run load balancer in each decision point, i.e. within the network

Sponsored by the National Science Foundation 11 GEC17 OpenFlow Switch Data Path (Hardware) Control Path OpenFlow Any Host OpenFlow Controller OpenFlow Protocol (SSL/TCP) Modified slide from : The controller is responsible for populating forwarding table of the switch Controller can get statistics directly from each switch: –per port –per flow –per table

Sponsored by the National Science Foundation 12 GEC17 Today’s Experiment Two paths from source to destination Balance traffic between the two paths so that: –Both paths are equally utilized –All TCP flows have similar performance Picture from :

Sponsored by the National Science Foundation 13 GEC17 Part I: Design/Setup –Network Load Balancing and OpenFlow –Design your experiment –Obtain Resources Part II: Execute –Configure and Initialize Services –Execute Experiment –Collect Measurements Part III: Finish –Archive results –Teardown Experiment

Sponsored by the National Science Foundation 14 GEC17 Design Your Experiment Modified slide from : Start OpenFlow Load Balancing Controller Connect OpenFlow Switch to Controller Start several TCP flows from Sender to Receiver S SW VM R LB OF Ctl

Sponsored by the National Science Foundation 15 GEC17 Design Your Experiment in GIMI Modified slide from : #start Load Balancing Controller Group('Switch').exec("/opt/trema-trema-f995284/trema run /root/load-balancer.rb") #connect OpenFlow Switch to Controller Group('Switch').exec(“ovs-vsctl set-controller br0 tcp: ptcp:6634: ") #start a couple of new TCP flows group('Receiver').exec("iperf -s") for i in group('Sender').exec("iperf -c t 100") wait 5 end

Sponsored by the National Science Foundation 16 GEC17 Monitor Your Experiment Modified slide from : S SW VM R LB OF Ctl

Sponsored by the National Science Foundation 17 GEC17 Part I: Design/Setup –Network Load Balancing and OpenFlow –Design your experiment –Obtain Resources Part II: Execute –Configure and Initialize Services –Execute Experiment –Collect Measurements Part III: Finish –Archive results –Teardown Experiment

Sponsored by the National Science Foundation 18 GEC17 Resources Reserved!

Sponsored by the National Science Foundation 19 GEC17 Part I: Design/Setup –Network Load Balancing and OpenFlow –Design your experiment –Obtain Resources Part II: Execute –Configure and Initialize Services –Execute Experiment –Collect Measurements Part III: Finish –Archive results –Teardown Experiment

Sponsored by the National Science Foundation 20 GEC17 Configure and Initialize Services Figure out the interfaces to monitor Update GIMI Script to monitor the correct interface: left = ‘eth1’ right = ‘eth2’ start Load Balancing Controller

Sponsored by the National Science Foundation 21 GEC17 Part I: Design/Setup –Network Load Balancing and OpenFlow –Design your experiment –Obtain Resources Part II: Execute –Configure and Initialize Services –Execute Experiment –Collect Measurements Part III: Finish –Archive results –Teardown Experiment

Sponsored by the National Science Foundation 22 GEC17 Execute Experiment Follow Instructions on Tutorial page to execute your experiment on LabWiki Pay attention to the output of your OpenFlow Controller:

Sponsored by the National Science Foundation 23 GEC17 Trema Trema is NOT an OpenFlow Controller Trema is a Programming Framework for implementing OpenFlow Controllers Trema is a Platform to run OpenFlow Controllers You need to: –Write your own controller using Trema API –Run your controller with Trema

Sponsored by the National Science Foundation 24 GEC17 Trema API API for both C and Ruby Event-triggered Functions: –start() is called when controller starts up –switch_ready() is called when switch connects in –packet_in() is called when a packet is received –stats_reply() is called when a flow stats reply message is received – More information about Trema API: –

Sponsored by the National Science Foundation 25 GEC17 Trema HelloWorld Modified slide from : class HelloTrema < Controller def start info “Hello Trema!” end def switch_ready datapath_id info “Hello %#x!” % datapath_id end def packet_in datapath_id, message send_flow_mod_add( datapath_id, :match => Match.from( message ), :actions => Trema::ActionOutput.new( OFPP_FLOOD ) ) end

Sponsored by the National Science Foundation 26 GEC17 Run Trema./trema run [controller] –e.g.,./trema run hello_trema.rb Trema Advantage: –Simple, easy to write You will need to write your own OpenFlow Controller in this tutorial using Trema (in Ruby)!

Sponsored by the National Science Foundation 27 GEC17 Part I: Design/Setup –Network Load Balancing and OpenFlow –Design your experiment –Obtain Resources Part II: Execute –Configure and Initialize Services –Execute Experiment –Collect Measurements Part III: Finish –Archive results –Teardown Experiment

Sponsored by the National Science Foundation 28 GEC17 Labwiki and OML Developed by NICTA, used for GIMI (GENI Instrumentation and Measurement Infrastructure) 3 Simple steps –Plan –Prepare –Execute A video demonstration of LabWiki is available at

Sponsored by the National Science Foundation 29 GEC17

Sponsored by the National Science Foundation 30 GEC17 Plan Plan your Experiment –Write descriptions about steps to perform experiments, see and compare results –Lab notebook where graphs can be inserted –Editable for additional comments about results –Can be saved for future reference

Sponsored by the National Science Foundation 31 GEC17 Plan

Sponsored by the National Science Foundation 32 GEC17 Prepare Write OMF Scripts –Simple Ruby like scripts that call OML-ified applications like ping, nmetrics and many more –Select from Existing templates –Edit the template –Save it as your own copy –Click and drag to the execute window when the script is ready

Sponsored by the National Science Foundation 33 GEC17 Prepare

Sponsored by the National Science Foundation 34 GEC17 Execute Run your Experiment –Enter Experiment name, slice name from the GENI Portal and set graph to true –Scroll to the bottom and Start Experiment –Logs are displayed. When the resources are turned ON, experiment begins –Scroll to the bottom and see the live graph –Drag and Drop Graph to Plan window to see live visualization in the “lab notebook”

Sponsored by the National Science Foundation 35 GEC17 Execute

Sponsored by the National Science Foundation 36 GEC17 Part I: Design/Setup –Network Load Balancing and OpenFlow –Design your experiment –Obtain Resources Part II: Execute –Configure and Initialize Services –Execute Experiment –Collect Measurements Part III: Finish –Archive results –Teardown Experiment

Sponsored by the National Science Foundation 37 GEC17 GENI TestBed OML Server LabWiki iRODS 1.Instrument 2.Run 3.Collect 4.Plot 5.Save OML Client 6.Obtain Collect and Archive 0.Reserve Experimenter 2.Run 3.Collect 4.Plot 5.Save

Sponsored by the National Science Foundation 38 GEC17 Archive - iRODs Integrated Rule-Oriented Data System –LabWiki stores all experiment related data here. Your scripts, result data and the manifest Rspec is stored in a user-specific directory structure –Download iRODs client (already on VM) –Go to the GENI Portal, under Profile and Click iRODs account –Configure iRODs Environment –To view experiment related data, go to iDrop, web interface for viewing iRODs data (link on GENI Portal and enter user name and password as shown.

Sponsored by the National Science Foundation 39 GEC17 iRODs

Sponsored by the National Science Foundation 40 GEC17 Part I: Design/Setup –Network Load Balancing and OpenFlow –Design your experiment –Obtain Resources Part II: Execute –Configure and Initialize Services –Execute Experiment –Collect Measurements Part III: Finish –Archive results –Teardown Experiment

Sponsored by the National Science Foundation 41 GEC17 Part III: Finish Experiment When your experiment is done, you should always release your resources. –Normally this is when you would archive your data –Delete your slivers at each aggregate slice project aggregate RSpec user resource sliver AM API sliver credentials certificate