1. 9/23/20151 The Internet & SLAC Les Cottrell 1, SLAC Les Cottrell 1SLAC  http://www.slac.stanford.edu http://www.slac.stanford.edu  /grp/scs/net/talk/internet-connectivity-97/index.htm /grp/scs/net/talk/internet-connectivity-97/index.htm F Outline of Talk –I. SLAC’s connectivity –II. How is it Working? –III. Why is it like it is? –IV. What’s going on out there on the Internet? –V. What are we (DOE/Esnet, HEP, SLAC) doing? –VII. Summary & Future Talk presented at SLAC, July 1997

2. 9/23/20152 Some Acronyms F ARA - Appletalk Remote Access, protocol to connect up remote Macs F ATM - Autonomous Transfer Mode, a high high speed network mechanism F DSL - Digital Subscriber Loop, a proposed medium speed (100s kbps - Mbps) leased line service (phone company answer to cable modems) F ESnet - Energy Sciences network (DOE’s research network, SLAC’s main connection to Internet F ISDN - Integrated Switched Digital Network, new <= 128 kbps digital switched phone service) F POP - Point of Presence, a place where one or more networks have facilities F SLIP/PPP protocols to provide Internet access over a serial line F VPN - Virtual Private Network, a way of tunneling private data over the public Internet F WAN - Wide Area Network

3. 9/23/20153 Outline F I. SLAC’s Connectivity F II. How is it Working? F III. Why is it like it is? F IV. What’s going on out there on the Internet? F V. What are we (DOE/Esnet, HEP, SLAC) doing? F VII. Summary & Future

4. 9/23/20154 Dial in Access http://www.slac.stanford.edu/comp/net/residential.html http://www.slac.stanford.edu/comp/net/residential.html F Terminal/emulator dial in –7 ports, 14.4 kbps F ARA –16 ports, <=33.6kbps, ~340 accounts (85 active/mo) F SLIP/PPP thru campus –14.4kbps, need campus account F Netcom, $15/mo, nationwide –28.8 kbps F Wireless via Ricochet F ISDN Direct & via ISP –9 ports, <=128 kbps, in pilot mode ~25 users, production service late summer F Following VPN developments

5. 9/23/20155 SLAC’s WAN Connectivity F 43Mbps to ESnet ATM cloud (Sprint Oakland POP) F 1.5Mps to Caltech/ESnet F 1.5Mbps to LBNL/ESnet F 10Mbps to Stanford

6. 9/23/20156 Outline F I. SLAC’s Connectivity F II. How is it Working? F III. Why is it like it is? F IV. What’s going on out there on the Internet? F V. What are we (DOE/Esnet, HEP, SLAC) doing? F VII. Summary & Future

7. 9/23/20157 What is Important to User F We have to optimize the scarcest & therefore most valuable commodity - Time F How long does it take after I hit the button?

8. 9/23/20158 Value of Rapid Response Time F Studies in late 70’s early 80s by Walt Doherty of IBM & others showed the economic value of rapid response time: –0-0.4s= High productivity interactive response –0.4-2s= Fully interactive regime –2-12s=Sporadically Interactive regime –12s-600s=Break in contact regime –>600s=Batch regime F There is a threshold around 4-5s where complaints increase rapidly

9. 9/23/20159 International little change N. America E improving 210 ms -> 150ms N. America W improving 140 ms -> 80ms ESnet improving 100ms -> 50 ms Ping Response for Groups of Hosts

10. 9/23/201510 European/Japan Packet Loss to SLAC Increase UK-US bandwidth Improve Esnet - Internet connect RAL: poor to unnacceptable, most others acceptable F Packet loss much more important –loss of packet typically causes 4-5s timeout

11. 9/23/201511 Quality by Host Group 0.0-1% Good, 1-5% Acceptable, 5-12% Poor 12-25% Bad, > 25% Unusable Similar to Internet Weather Report ( 12%)

12. 9/23/201512 F I. SLAC’s Connectivity F II. How is it Working? F III. Why is it like it is? F IV. What’s going on out there on the Internet? F V. What are we (DOE/Esnet, HEP, SLAC) doing? F VII. Summary & Future Outline

13. 9/23/201513 Driving Forces - Hosts

14. 9/23/201514 Driving Forces - New Apps F WWW, multimedia, Internet voice, video conferencing –> 60% graphics –<20% HTML

15. 9/23/201515 Driving Forces - Penetration Countries with Internet access US Domains

16. 9/23/201516 Current Internet Hosts

17. 9/23/201517 Challenge - Diversity of Traffic com 14% Traffic out of FNAL

18. 9/23/201518 Challenge - No single Mgmt for Links 1 RTR-CGB4.SLAC.Stanford.EDU 2 RTR-DMZ.SLAC.Stanford.EDU 3 ESNET-A-GATEWAY.SLAC.Stanford.EDU 4 pppl-atms.es.net 5 nynap-pppl-atms.es.net 6 192.157.69.11 [Sprint NAP] 7 core3-hssi3-0.WestOrange.mci.net 8 core1.WestOrange.mci.net 9 border2-fddi-0.WestOrange.mci.net 10 border2-hssi1-0-gw.WestOrange.mci.net 11 192.204.183.3 [PREPnet] 12 DEFAULT1-GW.UPENN.EDU 13 NISC8.UPENN.EDU

19. 9/23/201519 Challenge F Commercial Internet focussed on staying alive as opposed to research or promoting advanced requirements

20. 9/23/201520 More Acronymns F CalREN2 - a California initiative to provide better educational & research networking F CHEP97 - Computing in High Energy Physics meeting in Berlin, April 1997 F ESSC - ESnet’s Steering Committee F ICFA - International Committee on Future Accelerators F Internet 2 - University initiative to provide improved networking between universities F NGI - Next Generation Internet, Presidential initiative F vBNS - very high-speed Backbone Network System, a high speed NSF funded backbone network

21. 9/23/201521 Outline F I. SLAC’s Connectivity F II. How is it Working? F III. Why is it like it is? F IV. What’s going on out there on the Internet? F V. What are we (DOE/Esnet, HEP, SLAC) doing? F VII. Summary & Future

22. 9/23/201522 New Initiatives - California F CalREN2 –joint proposal NSF, UC, Stanford, Caltech … –includes Pac Bell & Cisco –Distributed GigaPOPs in SF & LA, also SD & Sac –Hi speed (622Mbps) ring around state envisioned

23. 9/23/201523 Bay Area F Bay Area CalREN2 GigaPOP nodes: –UCSF –UCB (links to Esnet, Sprint, MCI/vBNS, UC Davis (state wide) –UCOP –Stanford (links to NASA/NSI, BBN, MCI & statewide ring)

24. 9/23/201524 New U.S. Initiatives: vBNS F NSF initiative for interconnecting supercomputer centers for “meritorious applications” F Extended to promote University interconnectivity F 622 Mbps backbone

25. 9/23/201525 New US Initiatives: Internet 2 F Started out (Oct-96) as consortium of ~ 34 major universities –Now there are over 100 u covers 80% of US university sites we monitor –~$500K / university over several years, 25% seed –Will use vBNS as backbone –GigaPOPs in major areas

26. 9/23/201526 Next Generation Internet (NGI) F Presidential Initiative –$100M/yr for 3 years –100 sites at 100 times bandwidth (1.5Mbps => 155Mbps backbone) –10 sites at 1000 times bandwidth –DARPA, DOE, NSF, NASA… F Internet2/NGI/ESnet relationship unclear –can Universities connect to Internet 2 & ESnet?

27. 9/23/201527 U.S. International Connections F Only list those of interest to HEP F Moving to colocate US end points at DC POP to improve peering F Discussing CERN Esnet KEK link F STAR-TAP = proposed Int’l GigaPOP at Chicago

28. 9/23/201528 Europe: TEN-34 F W. European and some E. European countries interconnect at 4 - 34Mbps –de, it, ch, uk, gr, nl, pt, at, lu, es, fr, be, hu, sw+dk+no+fi –Several links in production, more by Jul-97 F Intra country links generally good F Intra Europe links improving with TEN-34 F Next step TEN-155

29. 9/23/201529 Asia & FSU F Most connections thru Japan, in general good to acceptable for KEK –US/Esnet/KEK 522kbps => 1.5Mbps –China 64kbps => 128kbps (via KEK) –=> 128kbps BINP/Russia Jun-97 F 2Mbps satellite DESY MSU (Moscow)

30. 9/23/201530 Outline F I. SLAC’s Connectivity F II. How is it Working? F III. Why is it like it is? F IV. What’s going on out there on the Internet? F V. What are we (DOE/ESnet, HEP, SLAC) doing? F VII. Summary & Future

31. 9/23/201531 ESSC End-user Connectivity WG F intra-ESnet connectivity good F ESnet University connectivity often bad F ESSC set up WG to look at problem –Ranked top 20 university sites by ER funding –Monitored from SLAC & FNAL –Identified worst (bad (6) to poor (8) performance) –Recommended ESnet look at 6 BAD sites to understand costs of improving, DOE/ESnet will provide money –Typical Frame Relay 1.5Mbps connections $1- 3K/month

32. 9/23/201532 ESnet Peering F Improved peering (63=>110 NSPs), examples: –MCI & Sprint to avoid public interconnect swamps –University of California (avoid Sprint) –THEnet at UT Austin –vBNS u East Coast in place since Feb-97 (avoid W Orange MCI) u West Coast May 1997 u Chicago to come –Hubs at DC, Oakland, San Diego, Chicago –Now carry > 45K routes

33. 9/23/201533 Improved ESnet Internet connection ESnet Peers with Sprint/MCI to avoid MAE-West

34. 9/23/201534 UC-Esnet Improved Peering & UCSC 25ms 16ms

35. 9/23/201535 Improved peering between Esnet & vBNS vBNS/Esnet Peering & U. Colorado

36. 9/23/201536 ICFA Internet Working Group F Mini-workshop CHEP97 F Working groups on: monitoring, remote regions, present status, requirements analysis, and the proposal F End 1998 come up with proposal on what to do & why F Next meeting Santa Fe, Sep-97

37. 9/23/201537 Monitoring - Why F “You can’t manage what you can’t measure” F Monitor to set “user” expectations, help with problem detection, get long term trends End-to-end monitoring mainly using ping F Provides response time, packet loss, reachability, unpredictability F Short (trouble shooting) & long term (planning) F Most important metric is packet loss

38. 9/23/201538 Monitoring - Who F Many major HEP sites are monitoring end- to-end Internet performance to collaborators –several hundred remote sites monitored F Collaborative effort to provide HEP-wide and ESnet wide reports, requested by ICFA, ESnet –Partially funded by DOE FWP involving SLAC, LBL, HEPNRC –Based on SLAC early work (ping based) will complement LBL NIMI work –SLAC, HEPNRC/FNAL, LBL collaboration

39. 9/23/201539 Monitoring - How F Plan to coordinate effort, centered on SLAC/HEPNRC code –install common software –distributed architecture –SLAC, HEPNRC Analysis Sites –Umd, RAL, INFN, KEK, ARM, CMU, RMKI, IN2P3, CERN, DESY, TRIUMF, MSU signed up to be Collection Sites –247 Remote Sites as of 7/7/97 –Reduces network impact of full mesh monitoring

40. 9/23/201540 F WWW Analysis Collecting Remote HTTP Ping Data (via HTTP) Pings E.g. HEPNRC E.g. SLAC E.g. RAL Data Collection & Distribution Architecture

41. 9/23/201541 Results from ~70 Sites in 10 Countries Being Monitored from SLAC ESnet Site N. American Site International Site SLAC FNAL ORNL UMd Monitoring Site

42. 9/23/201542 Putting it all Together

43. 9/23/201543 Outline F I. SLAC’s Connectivity F II. How is it Working? F III. Why is it like it is? F IV. What’s going on out there on the Internet? F V. What are we (DOE/Esnet, HEP, SLAC) doing? F VII. Summary & Future

44. 9/23/201544 Summary F Driving forces: –Internet user growth 8.4M => 28M US users (15 mos) –Computer power doubling every 12-18 months –new applications, WWW, Internet phone, VR, Video... F Since Apr-95, no single management for planning, trouble reporting etc. F ESnet performance good to acceptable, N. America poor (~6% packet loss avg), International poor (~7% packet loss avg) F Bottlenecks at interchanges

45. 9/23/201545 Future F Many separate initiatives: –critical to make sure they interplay well –identify and avoid bottlenecks –understand and guide impact for HEP F Criticality of Internet to HEP collaborations means HEP should increase efforts in this area: –keep tuned in, understand issues –monitor end-to-end performance –work with other research and higher education users

46. 9/23/201546 F SLAC Networking F http://www.slac.stanford.edu/comp/net/net.html http://www.slac.stanford.edu/comp/net/net.html F SLAC WAN Monitoring Page, lots of pointers F http://www.slac.stanford.edu/comp/net/wan-mon.html http://www.slac.stanford.edu/comp/net/wan-mon.html F ESnet: http://www.es.net/ http://www.es.net/ F vBNS: http://www.vbns.net/ http://www.vbns.net/ F Internet2: http://www.internet2.edu/ http://www.internet2.edu/ F NGI: http://www.hpcc.gov/ngi-concept-08Apr97/ http://www.hpcc.gov/ngi-concept-08Apr97/ F TEN-34: http://www.scimitar.terena.nl/projects/ten-34/ http://www.scimitar.terena.nl/projects/ten-34/ F ICFA Workshop on HEP & the Internet: F http://www.slac.stanford.edu/xorg/icfa/chep97/wg.html http://www.slac.stanford.edu/xorg/icfa/chep97/wg.html