1 Network Tomography Venkat Padmanabhan Lili Qiu MSR Tab Meeting 22 Oct 2001

2 Overview Goal: discover characteristics of internal links in network using passive, end-to-end measurements Metrics: loss rate, bandwidth Why is this interesting? –finding trouble spots in the network e.g., AT&T-Sprint peering point could be congested a Web site operator can keep tabs on his/her ISP and decide whether to sign up with new ISP(s) –deciding where to place server replicas downstream of major trouble spots

3 Sprint AT&T microsoft.com UUNET C&W Qwest AOL Earthlink Darn, it’s slow! Why is it so slow?

4 Topological Metrics Topological metrics are poor predictors of packet loss rate All links are not equal  need to identify the bad links

5 Prior Work Active probing to infer link loss rate –multicast probes –striped unicast probes Pros & cons –accurate since individual loss events identified –expensive because of extra probe traffic S AB S AB

6 Our Approach Passive observation of existing traffic –measure loss rate rather than loss events Active probing to discover network topology –can be done infrequently and in the background l1l1 l8l8 l7l7 l6l6 l2l2 l4l4 l5l5 l3l3 server clients p1p1 p2p2 p3p3 p4p4 p5p5 (1-l 1 )*(1-l 2 )*(1-l 4 ) = (1-p 1 ) (1-l 1 )*(1-l 2 )*(1-l 5 ) = (1-p 2 ) … (1-l 1 )*(1-l 3 )*(1-l 8 ) = (1-p 5 ) Under-constrained system of equations

7 #1: Random Sampling Randomly sample the solution space Repeat this several times Draw conclusions based on overall statistics How to do random sampling? –determine loss rate bound for each link using best downstream client –iterate over all links: pick loss rate at random within bounds update bounds for other links Problem: little tolerance for estimation error l1l1 l8l8 l7l7 l6l6 l2l2 l4l4 l5l5 l3l3 server clients p1p1 p2p2 p3p3 p4p4 p5p5

8 #2: Linear Optimization Goals Parsimonious explanation Robust to estimation error L i = log(1/(1-l i )), P j = log(1/(1-p j )) minimize  L i +  |S j | L 1 +L 2 +L 4 + S 1 = P 1 L 1 +L 2 +L 5 + S 2 = P 2 … L 1 +L 3 +L 8 + S 5 = P 5 L i >= 0 Can be turned into a linear program l1l1 l8l8 l7l7 l6l6 l2l2 l4l4 l5l5 l3l3 server clients p1p1 p2p2 p3p3 p4p4 p5p5

9 Results Experimental setup –packet tracing machine at microsoft.com –client loss rates estimated from TCP traffic –trace analyzed: 2.12 hours, 100 million packets, clients Validation –likely candidates for lossy links: links that cross an inter-AS boundary links that have a large delay

10 –Of the 50 links identified as most lossy, cross an inter-AS boundary and/or have delay > 100 ms –Example lossy links found: –San Francisco (AT&T)  Indonesia (Indo.net) –Sprint  PacBell in California –Moscow  Tyumen, Siberia (Sovam Teleport) Random Sampling Linear Optimization

11 Simulation Experiments Advantage: no uncertainty about link loss rate! Methodology –topologies used: randomly-generated: 1000 nodes, max degree = 5-50 real topology obtained by tracing paths to microsoft.com clients –randomly-generated packet loss events at each link loss rate 0-1% for 95% of links (non-lossy links), 5-10% for 5% of links (lossy links) Goodness metric: % links classified correctly –randomly-generated topologies: 90-94% accurate lossy links alone: 85-95% found, but 30-90% false +ve –real topology: 85-90% accurate

12 Ongoing and Future Work Large scale simulations with realistic topologies and traffic patterns Better validation in the Internet setting –correlation with packet loss rate for new clients –active measurements in real time Measurement from multiple sites(e.g., replicas) Other protocols and metrics –non-TCP traffic (e.g., streaming media); link bandwidth Refinement of techniques –“pseudo-passive” probing –selective active probing