1. International Telecommunication Union Workshop on End-to-End Quality of Service.What is it? How do we get it? Geneva, 1-3 October 2003 IP Performance Metrics: Definitions and Implementation Examples Al Morton AT&T Labs

2. ITU-T 2 1-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it? Outline o Performance Management Framework Relationship to the E2E QoS goal o IP Parameters/Metrics Summary o In-progress Metric Development o Implementations 1. Service Providers 2. Customers 3. 3rd Parties 1. Performance for MPLS-enabled IP Nets

3. ITU-T 3 1-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it? Network Performance Management Framework o Fault Monitoring -- failure detection o Passive Info Collection (single point) Read MIB counters or control data Sample Traffic o Active Measurements Synthetic Traffic Dedicated to meas. o Customer Measurements Live or Synthetic traffic

4. ITU-T 4 1-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it? Relationship to E2E QoS: Provide answers to... o Network Provider Is the design meeting requirements for various traffic classes or applications? How can I demonstrate the superior performance of my service offering? o Customer Is Network Performance Agreement? o 3rd Parties What does the net look like? Hot spots? What Network Provider is best?

5. ITU-T 5 1-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it? Packet Perf. Parameters t T max t T max t > T max ingress MPegress MP Successful IP packet outcome Errored IP packet outcome Spurious IP packet outcome Lost IP packet outcome Lost IP packet outcome IPRE 2 Corrupted header or errored payload IPRE 2 (Note) IPRE 1 NOTE –Outcome occurs independent of IP packet contents Never delivered or delivered to an unpermitted egress MP IPRE 1 1 2 Valid header and error-free payload IPRE 2 (Note)

6. ITU-T 6 1-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it? Metric/Parameter Definition Summary

7. ITU-T 7 1-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it? Comparison of IETF and ITU-T Delay Variation Metrics TxRcvPlayout Time spent in: Transit Rcv Buffer 1 2 3 4 t 1 2 3 4 Inter packet arrival time, longer than send interval IETF IPDV is a measure of transfer delay variation w.r.t. previous packet. For Packet n, IPDV(n) = Delay(n) - Delay(n-1) or = R(n) - R(n-1) - T(n) - T(n-1) If the nominal transfer time is =10msec, and packet 2 is delayed in transit for an additional 5 msec, then two IPDV values will be affected. IPDV(2) = 15 - 10 = 5 msec IPDV(3) = 10 - 15 = -5 msec IPDV(4) = 10 - 10 = 0 msec ITU-T SG 13 PDV is delay w.r.t. a reference, usually minimum delay. PDV(n) = Delay(n) - Min[Delay(*)] PDV(1,3,4)=0 PDV(2)=5

8. ITU-T 8 1-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it? Transient Delay Variation caused by burst traffic ITU PDVIETF

9. ITU-T 9 1-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it? Packet Metrics for VoIP and other voiceband applications o *new* metrics in G.IPP Consecutive Packet Loss Degraded Seconds Short-term Delay Variation Overall VoIP Parameters o Alan Clarks Presentation

10. ITU-T 10 1-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it? What is Packet Reordering? Packets arrive at Dst, but not in send order. 1, 2, 3, 7, 8, 9, 10, 11,... Loss,no reordering 1, 2, 3, 7, 8, 9, 4, 5, 6, 10, 11,...reordering In the world of order all these packets are of interest. 1, 2, 3, 7, 8, 9, 4, 5, 6, 10, 11,... | Early | Late | No reordering until Late Packets Arrive # of Early Packets => Reordering Extent

11. ITU-T 11 1-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it? Affect of Reordered Packets on most applications o Receivers must perform work to restore order 1, 2, 3, 7, 8, 9,10, 4, 5, 6, 11, 12,... | Buffered ||Reordered| 1 2 3 7 8 9 10 4 5 6 11 4 5 6, 11 (& 7 to 10) 1 2 3 Dst Time axis Higher layers

12. ITU-T 12 1-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it? Definition of Reordered Packet o Packet n is designated reordered when its sequence number is less than the Next Expected threshold (set by the arrival of a previous packet). Next Expected

13. ITU-T 13 1-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it? Failure Recovery Time o When recovery was a simple outage, characterization was simple, too. o IETF Benchmarking Methodology WG has identified 5 possible recovery scenarios: Lost packets Induced delay Out-of-order packets Duplicate packets 12 34567 12 3 4 56 4 1267 12 34 Errored packets 1267

14. ITU-T 14 1-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it? Implementations: Customers, Service Providers & 3rd Parties o Select Ping Target - make Round-trip connectivity and RTT measurement o Accuracy Issues include path through router, path through net (asymmetries), response time at target, sampling rates o Compare to current perf. to normal $ ping R2 (or R3) R1 R2 R3 R4 or

15. ITU-T 15 1-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it? Beyond ping: ICMP Timestamp or Timestamp Reply Message o Time spent processing packet at target can be removed, for more accurate RTT. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type | Code | Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Identifier | Sequence Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Originate Timestamp | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Receive Timestamp | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Transmit Timestamp | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ R1 R2 R3 R4 Originate Code=13 Receive Transmit Code=14

16. ITU-T 16 1-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it? Implementations: AT&T Global IP Measurements BR (City 1) MEASUREMENT COLLECTION SERVER (MCS) BR (City 2) BR (City n) Measurement probe AT&T GLOBAL IP BACKBONE INFRASTRUCTURE WEB CLIENTS (for report viewing) MEASUREMENT AGGREGATION & REPORTING SERVER (MRS) AGGREGATED MEASUREMENT DATA Measurement Probes http://www.att.com/ipnetwork

17. ITU-T 17 1-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it? AT&Ts IP Measurement Design o Poisson Sequence (RFC2330) 15 minute duration = 0.3 pkts/sec Type UDP, IPv4 278 bytes total ~300 packets sent unbiased sample o Periodic Sequence (RFC3432) 1 minute duration Random Start Time 20 ms packet spacing Type UDP, IPv4 60 bytes total ~3000 packets sent 24 hours... 15 minutes

18. ITU-T 18 1-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it? Technical Collaborators at AT&T o Len Ciavattone o George Holubec o Madhukar Kshirsagar o Ron Kulper o Arvind Ramarajan o Gomathi Ramachandran

19. ITU-T 19 1-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it? New Measurement Challenges for MPLS-enabled IP Networks o Most (all?) IP/Packet Network challenges o Two main categories of MPLS Domains: LDP-based, connection-less Traffic Engineering, connection oriented o Label Switched Paths are Unidirectional o point to point and multi-point to point o Many options for Failure Recovery o LSP identity optionally removed (PHP) o Work in progress in SG 13 = Y.MPLSperf

20. ITU-T 20 1-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it? New Measurement Challenges for MPLS-enabled IP Networks Network section Network section Network section MPLS Edge Node, or MPLS Ingress Node, or LSR if both IP and MPLS are enabled MPLS Node Exchange link Exchange link Exchange link Exchange link Exchange link Network Section Ensemble (NSE) Label Switched Paths Scope of OA&M Measurements: single Network Section or MPLS Domain MPLS Domain MPLS Network

21. ITU-T 21 1-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it? New Measurement Challenges for MPLS-enabled IP Networks o New Protocols = New Opportunities to Blackhole Traffic o Detect this new class of failures with Y.1711 MPLS OA&M Connectivity Verific. First version approved, adding fast failure detection LSP-Ping, Like ICMP Echo Request, plus One-way Delay measurement possible LSP Traceroute possible

22. ITU-T 22 1-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it? New Measurement Challenges for MPLS-enabled IP Networks o New Availability Definition? Crossroad: Connection-Oriented Transport has used a 10 second sliding window Connection-Less Packet Transport has used a 5 minute fixed window o MPLS Networks => both transport types When Connection-oriented Services use a Connection-less transport, which precedent should the Availability Definition follow?

23. ITU-T 23 1-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it? Summary o Performance Management Framework Measurement Systems are a key step toward the goal of E2E QoS o Summary of existing Parameters/Metrics o In-progress Metric Development o Active Measurement Implementations Ping for connectivity and... Dedicated Measurement Systems o Parameter Framework for MPLS has new challenges

24. ITU-T 24 1-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it? Resources and References o L. Ciavattone, A. Morton and G. Ramachandran, "Standardized Active Measurements on a Tier 1 IP Backbone," IEEE Communications Magazine, June 2003. o Geoff Huston, Measuring IP Network Performance, The Internet Protocol Journal, vol 6, no.1, March 2003 http://www.cisco.com/ipj o X.Xiao, et al., A Practical Approach for Providing QoS in the Internet Backbone, IEEE Communications Magazine, December 2002. o D. Meyer, et al., Trends in Measurement and Monitoring of Internet Backbones, Panel at NANOG 26, slides etc. at http://www.nanog.org/mtg- 0210/measurement.html o ITU-T Rec. Y.1540, Internet Protocol Data Communication Service – IP Packet Transfer and Availability Performance Parameters, 2003. o IETF IP Performance Metrics Working Group, links to RFC 2330, other IPPM RFCs and Internet Draft on Reordering: http://www.ietf.cnri.reston.va.us/html.chart ers/ippm-charter.html o Draft New Recommendation Y.MPLSperf, Performance and Availability Parameters for MPLS Networks o Draft New Recommendation G.IPP, Performance Parameter Definitions for Quality of Speech and other Voiceband Applications Utilising IP Networks o RFC 792, Internet Control Message Protocol, J. Postel, September 1981.