1 ESnet Update Joint Techs Meeting Minneapolis, MN Joe Burrescia ESnet General Manager 2/12/2007

2 Office of Science Collaborators (or “Why I am Here in the Frozen North”) The Office of Science (OC) supports (FY2008) the research of about 25,500 Ph.D.’s, Postdoctoral Research Associates, and Graduate Students Half of over 21,500 users of SC’s scientific facilities in FY 2008 will come from universities SC National User Facilities – User Affiliations From Dr Orbach’s FY2008 Budget Request for the Office of Science

3 Collaborative Effort: OSCARS On-demand Secure Circuit Advanced Reservation System (OSCARS) Collaborative effort status – Working with Internet2 and DRAGON to support interoperability between OSCARS/BRUW and DRAGON – Working with Internet2, DRAGON, and Terapaths to determine an appropriate interoperable AAL framework (this is in conjunction with GEANT2's JRA5) – Working with DICE (Dante, Internet2, CANARIE, ESnet) Control Plane group to determine schema and methods of distributing topology and reachability information. Completed porting OSCARS from Perl to Java to better support web-services. – This is now the common code base for OSCARS and Internet2's BRUW.

4 Collaborative Effort: perfSONAR perfSONAR is a global collaboration to design, implement and deploy a network measurement framework. Collaborators – ARNES, Belnet, CARnet, CESnet, Dante, University of Delaware, DFN, ESnet, FCCN, FNAL,GARR, GEANT2, Georga Tech, GRNET, Internet2, IST, POZNAN Supercomputing Center, Red IRIS, Renater, RNP, SLAC, SURFnet, SWITCH, Uninett, and others… ESnet Deployed Services – Link Utilization Measurement Archive – Virtual Circuit Status In Development – Active Latency and Bandwidth Tests – Topology Service – Additional Visualization capabilities

5 Current ESnet Network Status Since the July Joint Techs meeting… o ESnet transited LHC traffic between FNAL and, via USLHCnet, CERN for the first time this month o 10 GE NLR connection between Seattle and Chicago -In production currently carrying LHC traffic to FNAL -Backup for IP circuits o 10 GE NLR connection between Chicago and Washington DC -Was accepted -Will serve as one transition point between the current ESnet backbone and ESnet4 o Chicago MAN dark fiber physically in place o Direct peering between ESnet and Latin America on both Coasts -CUDI in San Diego -AMPATH at MANLAN

Current ESnet Topology 10 Gbps circuits Production IP core NLR core Metro Area Networks Lab supplied International connections Primary DOE Labs Backbone hubs Europe Asia-Pacific New York Chicago Washington, DC Atlanta CERN Seattle Albuquerque Aus. Australia Sunnyvale South America Latin America Canada Russia and China Canada CERN San Diego Major research and education (R&E) network peering points R&E Qwest supplied 10Gbps backbone NLR supplied 10Gbps circuits 2700 miles / 4300 km 1200 miles / 1900 km

7 ESnet Site Availability “5 nines” (>=99.995%) “3 nines (>99.5%)” “4 nines” (>99.95%) Dually connected sites

8 ESnet4 Status ESnet4 progress is being made: o All hardware needed to deploy Phase 1 has been ordered - At the cost of about $4M -This hardware continues to arrive at LBNL o A transition plan and schedule is in place, we are scheduled to be transitioned from Qwest backbone circuits totally by September ~30 new 10G circuits in 2007 (WAN and MAN) –1 new 10GE circuit every 12 days -~40 new 10G circuits in 2008 (WAN and MAN) –1 new 10GE circuit every 9 days o The first ESnet4 Science Data Network switch has just been installed in New York City, it is connected to the Level3 Infinera gear

9 ESnet4 IP + SDN Configuration, April 2007 Denver Seattle Sunnyvale LA San Diego Chicago Raleigh Jacksonville KC El Paso Albuq. Tulsa Clev. Boise Wash DC Salt Lake City Portland Baton Rouge Houston Pitts. NYC Boston Philadelphia Indianapolis Atlanta Nashville (1(2)) Layer 1 optical nodes at eventual ESnet Points of Presence ESnet IP switch only hubs ESnet IP switch/router hubs ESnet SDN switch hubs Layer 1 optical nodes not currently in ESnet plans Lab site ESnet IP core ESnet Science Data Network core ESnet SDN core, NLR links (existing) Lab supplied link LHC related link MAN link International IP Connections All circuits are 10Gb/s.

10 ESnet4 IP + SDN Configuration, September, 2007 Denver Seattle Sunnyvale LA San Diego Chicago Jacksonville KC El Paso Albuq. Tulsa Clev. Boise Wash DC Salt Lake City Portland Baton Rouge Houston Pitts. NYC Boston Philadelphia Indianapolis Atlanta Nashville All circuits are 10Gb/s. ESnet IP core ESnet Science Data Network core ESnet SDN core, NLR links Lab supplied link LHC related link MAN link International IP Connections Raleigh OC48 Layer 1 optical nodes at eventual ESnet Points of Presence ESnet IP switch only hubs ESnet IP switch/router hubs ESnet SDN switch hubs Layer 1 optical nodes not currently in ESnet plans Lab site

11 Layer 1 optical nodes at eventual ESnet Points of Presence ESnet IP switch only hubs ESnet IP switch/router hubs ESnet SDN switch hubs Layer 1 optical nodes not currently in ESnet plans Lab site ESnet4 IP + SDN Configuration, September, 2008 Denver Seattle Sunnyvale LA San Diego Chicago Raleigh Jacksonville KC El Paso Albuq. Tulsa Clev. Boise Wash DC Salt Lake City Portland Baton Rouge Houston Pitts. Boston Philadelphia Indianapolis Atlanta Nashville All circuits are 10Gb/s, or multiples thereof ESnet IP core (1 ) ESnet Science Data Network core ESnet SDN core, NLR links Lab supplied link LHC related link MAN link International IP Connections OC48

12 ESnet4 IP + SDN, 2011 Configuration Denver Seattle Sunnyvale LA San Diego Chicago Raleigh Jacksonville KC El Paso Albuq. Tulsa Clev. Boise Wash. DC Salt Lake City Portland Baton Rouge Houston Pitts. NYC Boston Philadelphia Indianapolis (>1 ) Atlanta Nashville Layer 1 optical nodes at eventual ESnet Points of Presence ESnet IP switch only hubs ESnet IP switch/router hubs ESnet SDN switch hubs Layer 1 optical nodes not currently in ESnet plans Lab site OC48 ESnet IP core ESnet Science Data Network core ESnet SDN core, NLR links (existing) Lab supplied link LHC related link MAN link International IP Connections (6) (12) 3 3

13 ESnet4 Built Out Core networks Gbps by (10Gb/s circuits), Gbps by (100 Gb/s circuits) Cleveland Europe (GEANT) Asia-Pacific New York Chicago Washington DC Atlanta CERN (30+ Gbps) Seattle Albuquerque Australia San Diego LA Denver South America (AMPATH) Latin America Canada (CANARIE) CERN (30+ Gbps) Canada (CANARIE) Asia- Pacific Asia Pacific GLORIAD (Russia and China) Boise Houston Jacksonville Tulsa Boston Science Data Network Core IP Core Kansas City Australia Core network fiber path is ~ 14,000 miles / 24,000 km 1625 miles / 2545 km 2700 miles / 4300 km Sunnyvale Production IP core (10Gbps) SDN core ( Gbps) MANs (20-60 Gbps) or backbone loops for site access International connections IP core hubs Primary DOE Labs SDN hubs High speed cross-connects with Ineternet2/Abilene Possible hubs USLHCNet