1. 1 Proposed future direction for CHEETAH Outline What's our goal for the network: eScience network or large-scale GP network? Book-Ahead (BA) or Immediate-Request (IR)? CHEETAH network evolution to support IR Software modules that need to be implemented Applications CHEETAH/DRAGON/HOPI interconnectivity Malathi Veeraraghavan University of Virginia August 9, 2006

2. 2 eScience or GP? BA or IR? BAIR eScience networks very large file transfers need high-BW and long holding time + remote viz. need to reserve timeslot for other resources None? general-purpose networks circuit service to service providers - users are only administrators - router-to-router circuits circuit/VC service for end users - host-to-host + router-to- router (E2E) - partial-path router-to-router circuits on congested links (called in by end user)

3. 3 Reasons for choosing IR Metcalfe's value statement SCALE, SCALE, SCALE IR seems more natural in data world Bandwidth not as scarce as airline seats that only BA can be supported; so IR seems like a must-have Cisco/Juniper/Sycamore control-plane implementation + community's standardization effort is "wasted" to a large extent with BA Shouldn't at least one testbed to study whether and, if so, how, IR bandwidth-management software built into switches can be useful

4. 4 Co-management of bandwidth on IR and BA basis But, BA and IR cannot coexist without bandwidth partitioning BA allows for high-BW, long-duration calls IR calls will suffer a high call blocking rate if supported through BA scheduler

5. 5 CHEETAH wide-area network Raleigh PoP (MCNC) Control card GbE/ 10GbE card ORNL PoP Control card GbE/ 10GbE card SN16000 Control card OC192 card OC-192 GbE/ 10GbE card End hosts Atlanta PoP (SOX/SLR) SN16000 GaTech End hosts ORNL OC192 card NCSU OC192 card OC-192 (via NLR/SLR/NCREN) via NCREN UVa CUNY Via Nysernet/HOPI Via Vortex/HOPI

6. 6 What CHEETAH s/w enables now. CHEETAH zelda2 Atlanta, GA zelda4 ORNL, TN wukong Releigh, NC mvstu6Cville, VA UGa UT Gatech duke NCSU UN C UVa VT GbE circuit Once zelda2's GbE card is LOCKED in a circuit headed to mvstu6, it cannot communicate with wukong or zelda4 via CHEETAH X. Fang R. Gisiger

7. 7 Network evolution to support IR Current CHEETAH network only supports 10 circuits per OC192 link IR mode inefficient if m, the link capacity in channels, is small (i.e., 10) Recoup OC1-crossconnect capability of the SN16Ks from its current 1Gbps use Has three advantages supports higher m; better for IR GMPLS standards based signaling (GbE-SONET signaling is Sycamore proprietary) Call setup delay: 166ms for two-hop instead of 1.5sec!

8. 8 Network evolution options Four options: NICs with VLAN support PLUS VLSR for SN16K with control-plane support for VLANs 15454 with VLSR IP router with VLSR Ethernet switch with VLSR

9. 9 NICs with VLAN support Useful for web caching application since a cache may need to talk simultaneously to two other caches In general useful for end-to-end circuits (whether router-to-router or host-to- host) BUT, since the SN16K GMPLS software does not support VLAN mapping to multiples of OC1s, this solution is insufficient; hence the next slide

10. 10 SN16K/VLSR at each PoP Can write a VLSR for SN16K to support GMPLS control of VLAN-OCxx circuits and then use VLAN-capable NICs Use hdb commands for flow-based Ethernet service, i.e., to map VLANs to OCxx circuits, but hdb software is unsupported Beats the reason for our choice of SN16Ks (GA support of GMPLS control plane)

11. 11 15454/VLSR at each PoP Make the 15454 serve as the intermediary between Ethernet NICs in hosts and SONET based SN16Ks at cheetah PoPs Useful for end-to-end circuits (whether router- to-router or host-to-host), e.g., webcaching. Two problems: Needs a heterogeneous signaling procedure, Intserv based Path message and VLAN IDs as labels to a SONET based Path message and SKULM labels VLAN ID continuity requirement Cannot support partial-path circuits

12. 12 IP router/VLSR at each PoP Use OCxx SONET interfaces to connect IP router to SN16K Connect web caches to router Have router initiate pure SONET circuit setup Use PBR or just ordinary routing table update to map flows to different OCxx circuits; support multiple circuits from one web cache

13. 13 IP router/VLSR at each PoP Can support end-to-end circuits web caching CDN servers router-to-router apps: administrator initiated increases SNMP log monitor initiated increases video apps at 10-15Mbps - map to one OC1 Has the potential to support PPCs (partial-path circuits) Place router with VLSR in enterprises at edge of GbE cheetah access link

14. 14 Ethernet switch/VLSR at each PoP Does not help with the problems noted in today's Gb/s circuit use of the SN16K long call setup delays: 1.5sec non-standard solution high per-circuit BW Using an Ethernet switch/VLSR at an enterprise (e.g. CUNY) requires all VLANs sharing 1Gbps CHEETAH access link to be switched to the same exit SN16K. Even worse, m=1 if whole 1Gb/s link used for a circuit

15. 15 Software modules required CVLSR for IP router Web caching & video apps CTCP code to support multiple simultaneous flows

16. 16 Applications In order of preference: web caching video apps at 10-15Mbps - map to one OC1 router-to-router apps: administrator initiated increases SNMP log monitor initiated increases storage Contact: Paul Sheldon, Vanderbilt, has NSF MRI to install data depots (600 TB) CDN servers IEEE, ACM, ComSoc web sites

17. 17 Equipment purchase IP routers with channelized SONET cards with GA GMPLS UNI implementation at each CHEETAH PoP IP routers with GbE blades at enterprises IP router in NLR McLean (fourth CHEETAH PoP) unless MAX/Dragon router is available - then buy GbE and OC192 blades for that router If NC 454 transponders are unavailable, purchase transponders for DC-Raleigh NLR link - since HOPI doesn't have this Colocation costs at NLR McLean and NLR Raleigh

18. 18 DRAGON How many WDM channels per link? m has to be small due to transponder costs makes it hard to support IR hence the interest in BA GMPLS in Movaz switches useful only for provisioning but not BW sharing Heterogeneous circuit setup not advisable in IR mode: should provision fat pipes based on aggregate usage observation, not per small-BW VLAN Does each DRAGON PoP have an Ethernet VLAN switch with VLSR?

19. 19 Interest in using CHEETAH as part of the Southern leg for HOPI HOPI is public-domain "commercial" Further HOPI uses VLAN switches - small granularity! CAC even w/o enforcement is fine if CTCP is the only conduit at end hosts - can enforce there

20. 20 HOPI and web caching Seems like a good match Rick's black cloud experiment - same as web caching Also Rick wants the "hybrid" part to be exercised - here's how with IP and VCs/circuits

21. 21 What am I asking Rick? Web caches at PoPs Permission to run router VLSR software on Abilene routers that will need to update routing tables! Will of course demo this first thoroughly on cheetah network

22. 22 What am I asking Jerry? CVLSR on HOPI switches - hop-by-hop IR mode IR support - GMPLS peer model call setup Path message between router VLSRs (in different domains) will carry final dest. IP address Router VLSR within cheetah (HOPI) domain finds exit router for that domain and issues SONET circuit Path message with that exit router as destination Web caches at DRAGON PoPs

23. 23 Interconnect CHEETAH, DRAGON HOPI Through IP routers of Cheetah & Abilene With our IP router/VLSR combo, setup router-to-route SONET OC1 circuits via cheetah and router-to-router VLAN virtual circuits through DRAGON/HOPI. At routers, do PBR mapping for flows or just update routing tables for destination IP addresses (data-plane This means packets go back to IP layer between two networks

24. 24 Figure HOPI + CHEETAH with routers in between

25. 25 Web caching example Example: web cache in CA to web-cache at ORNL VLAN from Sunnyvale Abilene router through five HOPI nodes to McLean router McLean router to ORNL router through SONET nodes