1. INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13, Q14 and Q17 STUDY GROUP 12 Source:Chairman TIA TR30.3 Subcommittee Title:Liaison presentation from TIA TR30.3 concerning proposed revision of G.1050 ABSTRACT As indicated in liaison presentations to SG12 over the last 2 years, TIA TR30.3 has been working on a proposed new revision to TIA-921 (Project: PN-3-0062-RV2) and the ITU-T version of the document G.1050, Network Model for Evaluating Multimedia Transmission Performance Over the Internet Protocol. TIA TR30.3 has now completed this work and has prepared a draft of TIA-921-B for TIA ballot. TR30.3 has also prepared a harmonized draft of ITU-T G.1050 Revised. TR30.3 and submits this draft document as a liaison to the upcoming Q13/12 meetings in Geneva. We request comments and feedback during the session of the attached draft of G.1050 Revised with a goal of achieving possible consent during January session. As Chairman of TR30.3, I am submitting the proposed revision to G.1050 and electronic attachments for review and consideration by SG12 Question 13 as an update to Recommendation G.1050 at the January2011 meetings in Geneva. TIA TR-30.3 has approved submitting this document and associated materials to ITU-T SG12. This presentation provides an overview of the proposed revision of G.1050 and electronic attachments. TD-xxxxxxxxx Contacts Jack Douglass, Chairman TIA TR30.3 PacketStorm Communications Tel: +1 949-436-0663 C Email: jack@packetstorm.comjack@packetstorm.com Chip Webb Anue System, USA Tel: +1 512-600-5403 Email: cwebb@anuesystems.comcwebb@anuesystems.com Les Wu Alcatel-Lucent, USA Tel: +1 (908) 582-4964 Email: Les.J.Wu@alcatel-lucent.comLes.J.Wu@alcatel-lucent.com Attention: This is not a publication made available to the public, but an internal ITU-T Document intended only for use by the Member States of ITU, by ITU-T Sector Members and Associates, and their respective staff and collaborators in their ITU related work. It shall not be made available to, and used by, any other persons or entities without the prior written consent of ITU-T.

2. Proposed Revision to G1050 Network Model for Evaluating Multimedia Transmission Performance over the Internet Protocol Liaison to ITU-T SG12 – Q13, Q14, Q17 TIA TR30.3 Committee Jack Douglass, Chairman, TIA TR30.3 PacketStorm Communications January 18 to 27, 2011

3. Agenda o G.1050-2007 / TIA-921A o Proposed G.1050 Revision o IP Network Simulation Source Code o IP Network Simulation o Network Models: Core, Core to LAN, LAN to LAN o Impairment Combination Tables o Inputs to IP Network Simulation o IP Network Simulation Outputs o Electronic Attachment o Possible Uses for IP Network Impairment Simulation o Future TIA-921/G.1050 Work o Requests for SG12

4. ITU-T G.1050 and TIA-921 Test Profiles based on ITU-T Y1541 Different test profiles for different Service Level Agreements (SLAs) Impairment TypeUnitsProfile A Well ‑ Managed Range (min to max) Profile B Partially-Managed Range (min to max) Profile C Unmanaged Range (min to max) One Way Latencyms20 to 100 (regional) 90 to 300 (intercontinental) 20 to 100 (regional) 90 to 400 (intercontinental) 20 to 500 Jitter (peak to peak)ms0 to 500 to 1500 to 500 Sequential Packet LossmsRandom loss only (except when link failure occurs) 40 to 20040 to 10,000 Rate of Sequential Losssec-1Random loss only (except when link failure occurs) < 10-3< 10-1 Random Packet Loss%0 to 0.050 to 20 to 20 Reordered Packets%0 to 0.0010 to 0.010 to 0.1

5. ITU-T G.1050-2007 and TIA-921A Network Topologies LAN A Core IP Network DUT A Route flapping Link Failures One-way delay Jitter Packet loss Packet Reorder Local Access A DUT B LAN B Local Access B Access Rates ---------------- Occupancy level Qos Edge Router Access Rates ---------------- Occupancy level Qos Edge Router LAN Rates ----------------- Occupancy level Packet loss Core IP Network Route flapping Link Failures One-way delay Jitter Packet loss Packet Reorder DUT B LAN B Local Access B DUT A Access Rates ---------------- Occupancy level Qos Edge Router LAN Rates ----------------- Occupancy level Packet loss LAN-to-LAN IP Network Model Core-to-LAN LAN Rates ----------------- Occupancy level Packet loss

6. ITU-T G.1050-2007 and TIA-921A Impairment Severity Table LAN / Access Rate Tables IP Impairment Combinations (Delay and Loss) 1344 General Model 120 Core to LAN ≤ 3 Mbit/s 48 Core to LAN > 3 Mbit/s (IPTV)

7. Proposed G.1050 Revision o G.1050 proposed update to Recommendation  Proposed ITU-T G.1050 Revised Recommendation harmonized with draft of TIA-921 Ballot  43 Test Cases  Request comments and feedback during the session  Achieve possible consent during January session o Electronic Attachments  Normative (Simulator Inputs, Outputs and Results)  Informative (C++ Source Code and Related Files)

8. IP Network Model Simulation Source Code (Appendix) o C++  Windows XP / 7: Cygwin 1.7  Linux: OpenSUSE 11.3 (i386), Fedora 9 o Other Linux and Windows Tools  TCL Versions 8.4 and 8.5  Perl 5.10  GNUPlot 4.4 o Simulation environment running on PC  Sample data, voice, video or audio PCAP files can be run through the model and captured the output as PCAP files for analysis and comparison

9. IP Network Model Simulation o General purpose discrete Event Simulator o Event Models simulate Network Elements  Switches  Routers  Access Technologies  Firewalls  Buffers  Connections via wires o Types of Events for Network Elements  Packet Arrival  Packet Departure  Other types of events are also possible

10. Model Basic IP Network Element

11. CORE Simulation Model

12. CORE-to-LAN Simulation Model

13. LAN-to-LAN Simulation Model

14. Well Managed Network

15. Partially Managed Network

16. Unmanaged & Core Network

17. Inputs to Network Simulation Network / Interferer PCAPs o Network / Interferer Traffic  IPTV Video, VoIP, Over-the-Top Video, Peer-to-Peer, HTTP, FoIP  Standard PCAPs or user provided PCAPs o PCAP – Packet Events drive Discrete Events Simulator  Packet size  Packet Timestamps o PCAP Payload  Payloads if present in the PCAP are carried through the simulator.

18. Input PCAP File Characteristics

19. Input PCAP Files Managed Bandwidth Characteristics IPTV HDIPTV SD VoIP

20. Input PCAP Files Residual Bandwidth Characteristics Adaptive Streaming (OTT2) Progressive Download (OTT1) FoIPHTTP Peer-to-Peer

21. Aggregate Bandwidth Characteristics Well Managed Partially Managed UnmanagedCore

22. Simulation Outputs o Output and Result files .out (CSV)  Time Series of Packet Loss and Delay  Packet Delay Variation (PDV) Histogram  Cumulative Distributive Functions (CDF)  Summary of results o Output PCAPs  Analysis of impaired packets can be performed if payload is included.

23. .out File (CSV) Example: HDTV1 / DSL / Well Managed Network Description: Set Top Box #1 IPTV Downstream Content-Encoding : ASCII Delay Unit:ms Time Delay Drop 527.844550910 5.02880544727.507581210 5.03018482527.507581210 5.03156225727.507581210 5.03294163427.507581210 5.03432101227.507581210 5.03569844427.507581210 5.03707782127.507581210 5.03845525327.507581210 5.0398346327.507581210 5.04121400827.507581210 5.0425914427.507581210 5.04397081727.507581210 5.04535019527.507581210 5.04672762627.507581210 5.04810700427.507581210 5.04948638127.507581210 5.05086381327.507581210 5.05224319127.507581210 5.05362062327.507581210

24. .out File Time Series of Packet Loss and Delay Example: HDTV1 / DSL / Well Managed Network Dw1Dw2Dw3 Dw4Dw5Dw6 Dw7Dw8

25. .out File PDV Histograms Example: HDTV1 / DSL / Well Managed Network Dw1Dw2Dw3 Dw4Dw5Dw6 Dw7Dw8

26. .out File PDV Histograms and CDFs Example: HDTV1 / DSL / Well Managed Network

27. Simulation Results Summary Example: HDTV1/ DSL / Well Managed Network Test CaseFile Name Number of Packets Min Delay Max Delay Drop (Loss) Seq Loss drop % (Loss) seq % (Loss) max-min (Jitter) Dw1HDTV1_Down43400312.5013.974700.011%0.000%1.47 Dw2HDTV1_Down43399327.5129.314700.011%0.000%1.81 Dw3HDTV1_Down43397052.5267.525200.012%0.000%15.01 Dw4HDTV1_Down43394477.70106.955900.014%0.000%29.25 Dw5HDTV1_Down43352287.8994.7147420.109%0.000%6.82 Dw6HDTV1_Down433541102.90153.0444480.102%0.002%50.14 Dw7HDTV1_Down433486127.92178.0548180.111%0.002%50.14 Dw8HDTV1_Down433491153.34164.9745620.105%0.000%11.63

28. .pcap file Example: HDTV1/ Dw1

29. .out File (CSV) Example: SDTV2 / DSL / Well Managed Network Description: Set Top Box #3 IPTV Downstream Content-Encoding : ASCII Delay Unit:ms Time Delay Drop 553.276041330 5.08415512553.152064580 5.21390858753.099753330 5.30023822753.099753330 5.42984210553.978936070 5.51625484853.099753330 5.6459889253.099753330 5.73251800653.985145130 5.81886703653.099753330 5.94851523553.099753330 6.03491135753.099753330 6.1213324154.017074030 6.25109418353.620698710 6.38063157953.099753330 6.46702770153.099753330 6.59687811653.099753330 6.68328254853.099753330 6.76958448853.145778620 6.89915789553.22292260 6.98557063753.099753330

30. .out File Time Series of Packet Loss and Delay Example: SDTV2 / DSL / Partially Managed Network Dp1Dp2Dp3 Dp4Dp5Dp6

31. .out File PDV Example: SDTV2 / DSL / Partially Managed Network Dp1Dp2Dp3 Dp4Dp5Dp6

32. .out File PDV Histograms and CDFs Example: SDTV2 / DSL / Partially Managed Network

33. Simulation Results Summary Example: SDTV2/ DSL / Partially Managed Network Test CaseFile Name Number of Packets Min Delay Max Delay Drop (Loss) Seq Loss drop % (Loss) seq % (Loss) max-min (Jitter) Dp1SDTV2_Down11032712.6215.451200.011%0.000%2.83 Dp2SDTV2_Down11020727.7845.7612900.117%0.000%17.98 Dp3SDTV2_Down11019653.10119.9513400.121%0.000%66.85 Dp4SDTV2_Down10901579.05213.131309691.186%0.063%134.09 Dp5SDTV2_Down108813105.89375.2415051681.364%0.152%269.35 Dp6SDTV2_Down109009204.09338.181288491.168%0.044%134.09

34. .pcap file Example: SDTV2/ DP1

35. .out File Time Series of Packet Loss and Delay Examples: Managed Bandwidth HDTV1 SDTV2 HDTV2 SDTV1 VoIP

36. .out File PDV Histograms Examples: Managed Bandwidth HDTV1 SDTV2 HDTV2 SDTV1 VoIP

37. .out File Time Series of Packet Loss and Delay Examples: Residual Bandwidth OTT1 Peer-to-Peer OTT2HTTP FoIP

38. .out File PDV Histograms Examples: Residual Bandwidth OTT1 Peer-to-Peer OTT2HTTP FoIP

39. Electronic Attachment o Compressed File Archive (approx 1.2G compressed)  G.1050 Document (PDF)  Input PCAP files (without payloads)  TCL files for 43 Test Case scenarios .out files  Output PCAPs  PNGs  Results Summary (.csv)  Impairment Combination Table  Simulation Code – C++ Code (Informative)

40. Possible Uses for IP Network Impairment Simulation o Simulate Real-World IP network impairments  PDV Histogram and Packet Loss characteristics o Test any type of IP stream(s) (PCAP) under simulated network conditions  Standard Test Cases or user define simulated network conditions  Test with or without payload  Capture output PCAP for analysis and comparison o Test any type of IP stream using Hardware Emulation of simulated network model  Standard Test Cases or user define simulated network conditions

41. Example CORE-to-LAN Simulation Test Stream(s) Interfering Stream(s) HD VBR IPTV 2 QoS Interferers Bandwidth Interferers PDV & Loss MOS Calculated from captured PCAP OTT1 OTT2 SD VBR IPTV HTTP HD CBR IPTV QoS 1 2 7 2 7 7 VoIP Down

42. Example MOS Calculation Derived from Simulation Output PCAP Simulation Output PCAP Video Stream Analysis: ============================================= Stream source: 192.168.69.110:49713 Stream destination: 234.5.6.101:2345 Stream duration: 00:09:59.936 Stream CODEC: H.264 Stream transport: RTP (SSRC=0x11223344) Stream transport: MP2TS (PID=481, Ctrl=0x0000) Packet counts: received 2469009 lost 649 (0.00026) discarded 819 (0.00032) out-of-seq 6 (0.00000) duplicate0 (0.00000) corrected 0 (0.00000) MOS-V (min/avg/max/inst): 2.74/3.93/4.56/3.19 MOS-AV (min/avg/max/inst): 2.89/3.82/4.32/3.29 Captured output PCAPs can be use to analyze and compare competing algorithms Simulation.out Packet Delay and Loss Characteristic can be used to cause impairments on test streams

43. Hardware Emulation from IP Network Simulation Select a standard test case that includes a technology similar to the test stream Configure the emulator with the standard.out filefor the selected test case Run the test stream through the emulator, and evaluate the effect of the network impairments on the stream

44. Advanced Use of the Network Model User-Defined Test Cases o Replace any of the standard interferer pcap files with user interfere pcap files o Change the traffic mix o Change network conditions  number of switches  bit error ratio  access technology parameters,  etc.

45. Future TIA-921/G.1050 Work o Active Network Emulation using Interferer PCAP files o Add Cable Access Characteristics o Add Wireless Access Characteristics o Update as IP Network evolves

46. Requests to SG12 o Provide comments and feedback on G.1050 Revised during session  Document  Simulation and Results  Electronic Attachments o Consider possible Consent of G.1050 Revised during the January session