1 Monitoring Internet connectivity of Research and Educational Institutions Les Cottrell – SLAC/Stanford University Prepared for the workshop on “Developing Country Access to On-line Scientific Publishing", 4-5 October 2002, Trieste, Italy Partially funded by DOE/MICS Field Work Proposal on Internet End-to-end Performance Monitoring (IEPM), also supported by IUPAP

2 Outline Measurement project initially created for HEP to measure performance for various collaborations, extended to other physics & science collaborations –Value for planning, trouble shooting, setting expectations, comparisons, setting SLAs etc. Methodology Results –Round trip times –Loss Summary

3 Measurement Architecture Uses existing ubiquitous Internet ping infrastructure, no tools to install Hierarchical vs. full mesh, each monitoring site chooses remote sites Lightweight – –low network impact (100bits/s/path) –no special machines –trivial to add monitored sites Runs continuously since 1995 WWW Archive Monitoring Remote HEPNRC Reports & Data Cache Monitoring SLAC Ping HTTP Archive 1 monitor host remote host pair

4 PingER Measurement Methodology Measurement host admin choose remote hosts of interest –sends 21 pings each 30 mins to each chosen remote host –Records RTT, loss, jitter, unreachable, out of order … –Records data in local cache Archive host gathers data from measurements hosts regularly (at least daily) –Archives, analyzes and generates reports from data –Make reports and data publicly available via the web Requirements: –Remote host: need a host accessible to pings, and a contact in case host does not respond (almost no effort) –Monitoring host: a low end host to make measurements, file space for cache, admin to install toolkit, choose remote hosts, build configuration file, respond to archivers in case unable to get data & keep it running (<<10% FTE) –Archive site: probably about 20% of an FTE

5 PingER deployment Measurements from –34 monitors in 14 countries –Over 600 remote hosts –Over 72 countries –Over 3300 monitor-remote site pairs –Measurements go back to Jan-95 –Reports on RTT, loss, reachability, jitter, reorders, duplicates … Countries monitored –Contain 78% of world population –99% of online users of Internet –Mainly A&R sites Monitoring Sites Remote Sites

6 User interface 1/2 Choose: metric, monitoring site(s), remote sites(s), time granularity Shows colored values by time, allows downloading of results for further analysis

7 User interface 2/2: PingER Group History Table

8 PingWorld Java applet at

9 Performance Results Examples of the type of information that the monitoring can provide

10 History - Round Trip Time (RTT) Improving by 10-20% year More direct paths Replacing satellites with land lines –Satellite >~550ms Faster lines & network equipment Lower limit speed of light in fiber Typical lower limit today ~ distance/(0.3 * (0.6 * c)) Speed of light in fiber

11 RTT to world from US Note large number of satellite links (> 600ms dark red) Note reduction by Aug 2002 Jan 2000 Aug 2002

12 Impact of loss on applications –fairly insensitive to quality, may be delayed but keeps retrying for days and eventually gets through Web –usually has human but expectations are low, performance often more limited by server, human present so can retry Bulk file transfer –unattended, if > 10-12% loss connections can time out Interactive telnet, voice –very time & loss sensitive –E.g. telnet/ssh loss of > 3% severely impacts typing ability, interactive voice sensitive at lower losses Importance of loss/performance

13 History - Loss Loss more critical than RTT Losses cause timeouts of typically seconds 40-50% improve/yr Best networks below 0.1% Russia, SE Europe, China several years behind

14 History – Loss Quality Fewer sites have v. poor to dreadful performance More have good performance (< 1%)

15 Loss to world from US Using year 2000, fraction of world’s population/country from

16 Losses: World by region, Jan ‘02 5%=bad Russia, S America bad Balkans, M East, Africa, S Asia, Caucasus poor

17 History - Throughput quality improvements from US TCP BW < MSS/(RTT*sqrt(loss)) (1) (1) Macroscopic Behavior of the TCP Congestion Avoidance Algorithm, Matthis, Semke, Mahdavi, Ott, Computer Communication Review 27(3), July % annual improvement ~ factor 10/4yr ~Factor 100 improvement in 8 years

18 Detailed example of improvements Increase of bandwidth by factor of 460 in 6 years, more than kept pace - factor of 50 times improvement in loss Note valleys when students on vacation

19 Summary - results Internet A&R connectivity performance is improving –RTT 10-20%/yr, loss 50%/yr, throughput 80%/yr –Reduced use of satellites, mainly use for new hard to get to areas (e.g. S. Russian Republics) China, S.E. Europe, Russia rate of change keeps up but several years behind India, S. America performance is where N. America & W. Europe were 4 – 5 years ago Africa limited continuous results (UCT & Wits. no longer respond): Uganda losses in last 2 years reduced from10% to 3%, RTT fairly constant at 800ms. Improvements need constant investments to understand & improve

20 Summary - PingER Lightweight (100bps/host pair, 21 pings/30mins per pair) Very useful for inter-regional and poor links Easy to deploy (uses ubiquitous Internet ping infrastructure), however pings can be blocked Easy to deploy for monitoring of sites in developing countries –Remote sites ~ no effort (provide contact & host) –Monitoring site small effort:1 day to download software set up & configure, (shared host) choose remote hosts to monitor, make data available for upload, check working, ongoing respond to s. –SLAC would be willing to assist –Data public so anyone can do analysis/presentation of data –Provide me (business card or with contact and name of host to be

21 Help Looking for better hosts to monitor & contacts in: –Albania, Armenia, Austria, Azerbaijan –Macedonia*, Turkey*, Yugoslavia –Columbia*, Venezuela*, Cuba, Mexico* –Pakistan* –Africa (apart from Egypt, Uganda & South Africa, n.b. according to all 54 countries in Africa now have Internet access in capitals) –Note there are a few countries (about 5% of the world’s countries) that do not have full Internet connections and pay dearly by the byte. A couple of years ago these included: Afghanistan, Western Sahara, Christmas Island, S. Georgia, Marshall Islands, Myanmar, Montserrat, N. Korea, Pitcairn, St Vincente & Grenadines

22 More Information IEPM/PingER home site: –www-iepm.slac.stanford.edu/www-iepm.slac.stanford.edu/ What we do, coverage, how to download (free) software, requirements for hosts, results, data download etc. Java demonstration from your computer –