1. INTERNATIONAL TELECOMMUNICATION TELECOMMUNICATION STANDARDIZATION SECTOR STUDY PERIOD 2009-2012 January 18-27, 20110 English only Original: English Question(s):Q13, Q14 and Q17 STUDY GROUP 12 Source:Chairman TIA TR30.3 Subcommittee Title:Updated Liaison presentation from TIA TR30.3 concerning proposed revision of G.1050 ABSTRACT As indicated in liaison presentations to SG12 over the last 2 years, TIA TR30.3 has been working on a proposed new revision to TIA-921 (Project: PN-3-0062-RV2) and the ITU-T version of the document G.1050, Network Model for Evaluating Multimedia Transmission Performance Over the Internet Protocol. TIA TR30.3 has now completed this work and has prepared a draft of TIA-921-B for TIA ballot. TR30.3 has also prepared a harmonized draft of ITU-T G.1050 Revised. TR30.3 and submits this draft document as a liaison to the upcoming Q13/12 meetings in Geneva. We request comments and feedback during the session of the attached draft of G.1050 Revised with a goal of achieving possible consent during January session. As Chairman of TR30.3, I am submitting the proposed revision to G.1050 and electronic attachments for review and consideration by SG12 Question 13 as an update to Recommendation G.1050 at the January2011 meetings in Geneva. TIA TR-30.3 has approved submitting this document and associated materials to ITU-T SG12. This presentation provides an overview of the proposed revision of G.1050 and electronic attachments. TD 435rev1 (GEN/12) Contacts Jack Douglass, Chairman TIA TR30.3 PacketStorm Communications Tel: +1 949-436-0663 C Email: jack@packetstorm.comjack@packetstorm.com Chip Webb Anue System, USA Tel: +1 512-600-5403 Email: cwebb@anuesystems.comcwebb@anuesystems.com Les Wu Alcatel-Lucent, USA Tel: +1 (908) 582-4964 Email: Les.J.Wu@alcatel-lucent.comLes.J.Wu@alcatel-lucent.com Attention: This is not a publication made available to the public, but an internal ITU-T Document intended only for use by the Member States of ITU, by ITU-T Sector Members and Associates, and their respective staff and collaborators in their ITU related work. It shall not be made available to, and used by, any other persons or entities without the prior written consent of ITU-T.

2. Proposed Revision to G1050 Network Model for Evaluating Multimedia Transmission Performance over the Internet Protocol Liaison to ITU-T SG12 – Q13, Q14, Q17 TIA TR30.3 Committee Jack Douglass, Chairman, TIA TR30.3 PacketStorm Communications January 18 to 27, 2011

3. Agenda o G.1050-2007 / TIA-921A o Summary of Changes from G.1050 (2007) o Proposed G.1050 Revision o IP Network Simulation Source Code o IP Network Simulation o Network Models: Core, Core to LAN, LAN to LAN o Impairment Combination Tables o Inputs to IP Network Simulation o IP Network Simulation Outputs o Electronic Attachment o Possible Uses for IP Network Impairment Simulation o Future TIA-921/G.1050 Work o Requests for SG12

4. Summary of Changes from G.1050 (2007) o Proposed revision to G.1050 replaces G.1050 (2007) o G.1050 (2007)  Gilbert-Elliott mathematical model that fit certain observed network behavior but was not easily extended to other scenarios o Revised G.1050 – Provide more realism than the earlier version  Discrete Event Simulation  Models the mechanisms using that contribute to packet delay, jitter, and loss: interfering streams, queue delays in network elements, and the characteristics of specific access technologies.  Greatly reduce the number of standard test cases (43 test cases)  True bidirectional model  Removed LOO “likelihood of occurrence” concept  Impairment levels are updated to keep current with evolving IP networks  Users can customize test cases to fit their specific needs

5. ITU-T G.1050 and TIA-921 Test Profiles based on ITU-T Y1541 Different test profiles for different Service Level Agreements (SLAs) Impairment TypeUnitsProfile A Well ‑ Managed Range (min to max) Profile B Partially-Managed Range (min to max) Profile C Unmanaged Range (min to max) One Way Latencyms20 to 100 (regional) 90 to 300 (intercontinental) 20 to 100 (regional) 90 to 400 (intercontinental) 20 to 500 Jitter (peak to peak)ms0 to 500 to 1500 to 500 Sequential Packet LossmsRandom loss only (except when link failure occurs) 40 to 20040 to 10,000 Rate of Sequential Losssec-1Random loss only (except when link failure occurs) < 10-3< 10-1 Random Packet Loss%0 to 0.050 to 20 to 20 Reordered Packets%0 to 0.0010 to 0.010 to 0.1

6. ITU-T G.1050-2007 and TIA-921A Network Topologies LAN A Core IP Network DUT A Route flapping Link Failures One-way delay Jitter Packet loss Packet Reorder Local Access A DUT B LAN B Local Access B Access Rates ---------------- Occupancy level Qos Edge Router Access Rates ---------------- Occupancy level Qos Edge Router LAN Rates ----------------- Occupancy level Packet loss Core IP Network Route flapping Link Failures One-way delay Jitter Packet loss Packet Reorder DUT B LAN B Local Access B DUT A Access Rates ---------------- Occupancy level Qos Edge Router LAN Rates ----------------- Occupancy level Packet loss LAN-to-LAN IP Network Model Core-to-LAN LAN Rates ----------------- Occupancy level Packet loss

7. ITU-T G.1050-2007 and TIA-921A Impairment Severity Table LAN / Access Rate Tables IP Impairment Combinations (Delay and Loss) 1344 General Model 120 Core to LAN ≤ 3 Mbit/s 48 Core to LAN > 3 Mbit/s (IPTV)

8. Proposed G.1050 Revision o G.1050 proposed update to Recommendation  Proposed ITU-T G.1050 Revised Recommendation harmonized with draft of TIA-921 Ballot  Request comments and feedback during the session  Achieve possible consent during January session o Electronic Attachments  C++ Source Code and Related Files  Simulator Inputs, Outputs and Results

9. IP Network Model Simulation Source Code o C++  Windows XP / 7: Cygwin 1.7  Linux: OpenSUSE 11.3 (i386), Fedora 9 o Other Linux and Windows Tools  TCL Versions 8.4 and 8.5  Perl 5.10  GNUPlot 4.4 o Simulation environment running on PC  Sample data, voice, video or audio PCAP files can be run through the model and captured the output as PCAP files for analysis and comparison o The Software Copyright Holder (Anue Systems) will grant a free license based on Option 1.3 of ITU Software Copyright Statement and Licensing Declaration – Form has been filed

10. IP Network Model Simulation o General purpose discrete Event Simulator o Event Models simulate Network Elements  Switches  Routers  Access Technologies  Firewalls  Buffers  Connections via wires o Types of Events for Network Elements  Packet Arrival  Packet Departure  Other types of events are also possible

11. Model Basic IP Network Element

12. CORE Simulation Model

13. CORE-to-LAN Simulation Model

14. LAN-to-LAN Simulation Model

15. Well Managed Network

16. Partially Managed Network

17. Unmanaged & Core Network

18. Inputs to Network Simulation Network / Interferer PCAPs o Network / Interferer Traffic  IPTV Video, VoIP, Over-the-Top Video, Peer-to-Peer, HTTP, FoIP  Standard PCAPs or user provided PCAPs o PCAP – Packet Events drive Discrete Events Simulator  Packet size  Packet Timestamps o PCAP Payload  Payloads if present in the PCAP are carried through the simulator.

19. Input PCAP File Characteristics

20. Input PCAP Files Managed Bandwidth Characteristics IPTV HDIPTV SD VoIP

21. Input PCAP Files Residual Bandwidth Characteristics Adaptive Streaming (OTT2) Progressive Download (OTT1) FoIPHTTP Peer-to-Peer

22. Aggregate Bandwidth Characteristics Well Managed Partially Managed UnmanagedCore

23. Simulation Outputs o Output and Result files .out (CSV)  Time Series of Packet Loss and Delay  Packet Delay Variation (PDV) Histogram  Cumulative Distributive Functions (CDF)  Summary of results o Output PCAPs  Analysis of impaired packets can be performed if payload is included.

24. .out File (CSV) Example: HDTV1 / DSL / Well Managed Network Description: Set Top Box #1 IPTV Downstream Content-Encoding : ASCII Delay Unit:ms Time Delay Drop 527.844550910 5.02880544727.507581210 5.03018482527.507581210 5.03156225727.507581210 5.03294163427.507581210 5.03432101227.507581210 5.03569844427.507581210 5.03707782127.507581210 5.03845525327.507581211 5.0398346327.507581211 5.04121400827.507581210 5.0425914427.507581210 5.04397081727.507581210 5.04535019527.507581210 5.04672762627.507581210 5.04810700427.507581210 5.04948638127.507581210 5.05086381327.507581210 5.05224319127.507581211 5.05362062327.507581210

25. .out File Time Series of Packet Loss and Delay Example: HDTV1 / DSL / Well Managed Network Dw1Dw2Dw3 Dw4Dw5Dw6 Dw7Dw8

26. .out File PDV Histograms Example: HDTV1 / DSL / Well Managed Network Dw1Dw2Dw3 Dw4Dw5Dw6 Dw7Dw8

27. .out File PDV Histograms and CDFs Example: HDTV1 / DSL / Well Managed Network

28. Simulation Results Summary Example: HDTV1/ DSL / Well Managed Network Test CaseFile Name Number of Packets Min Delay Max Delay Drop (Loss) Seq Loss drop % (Loss) seq % (Loss) max-min (Jitter) Dw1HDTV1_Down43400312.5013.974700.011%0.000%1.47 Dw2HDTV1_Down43399327.5129.314700.011%0.000%1.81 Dw3HDTV1_Down43397052.5267.525200.012%0.000%15.01 Dw4HDTV1_Down43394477.70106.955900.014%0.000%29.25 Dw5HDTV1_Down43352287.8994.7147420.109%0.000%6.82 Dw6HDTV1_Down433541102.90153.0444480.102%0.002%50.14 Dw7HDTV1_Down433486127.92178.0548180.111%0.002%50.14 Dw8HDTV1_Down433491153.34164.9745620.105%0.000%11.63

29. .pcap file Example: HDTV1 (Payloads Truncated)

30. .out File (CSV) Example: SDTV2 / DSL / Well Managed Network Description: Set Top Box #3 IPTV Downstream Content-Encoding : ASCII Delay Unit:ms Time Delay Drop 553.276041330 5.08415512553.152064580 5.21390858753.099753330 5.30023822753.099753330 5.42984210553.978936071 5.51625484853.099753330 5.6459889253.099753330 5.73251800653.985145130 5.81886703653.099753330 5.94851523553.099753330 6.03491135753.099753330 6.1213324154.017074030 6.25109418353.620698710 6.38063157953.099753331 6.46702770153.099753330 6.59687811653.099753330 6.68328254853.099753330 6.76958448853.145778621 6.89915789553.22292260 6.98557063753.099753330

31. .out File Time Series of Packet Loss and Delay Example: SDTV2 / DSL / Partially Managed Network Dp1Dp2Dp3 Dp4Dp5Dp6

32. .out File PDV Example: SDTV2 / DSL / Partially Managed Network Dp1Dp2Dp3 Dp4Dp5Dp6

33. .out File PDV Histograms and CDFs Example: SDTV2 / DSL / Partially Managed Network

34. Simulation Results Summary Example: SDTV2/ DSL / Partially Managed Network Test CaseFile Name Number of Packets Min Delay Max Delay Drop (Loss) Seq Loss drop % (Loss) seq % (Loss) max-min (Jitter) Dp1SDTV2_Down11032712.6215.451200.011%0.000%2.83 Dp2SDTV2_Down11020727.7845.7612900.117%0.000%17.98 Dp3SDTV2_Down11019653.10119.9513400.121%0.000%66.85 Dp4SDTV2_Down10901579.05213.131309691.186%0.063%134.09 Dp5SDTV2_Down108813105.89375.2415051681.364%0.152%269.35 Dp6SDTV2_Down109009204.09338.181288491.168%0.044%134.09

35. .pcap file Example: SDTV1 (With Payload)

36. .out File Time Series of Packet Loss and Delay Examples: Managed Bandwidth HDTV1 SDTV2 HDTV2 SDTV1 VoIP

37. .out File PDV Histograms Examples: Managed Bandwidth HDTV1 SDTV2 HDTV2 SDTV1 VoIP

38. .out File Time Series of Packet Loss and Delay Examples: Residual Bandwidth OTT1 Peer-to-Peer OTT2HTTP FoIP

39. .out File PDV Histograms Examples: Residual Bandwidth OTT1 Peer-to-Peer OTT2HTTP FoIP

40. Electronic Attachment  ISO 9660 format file that is downloadable from the ITU web site  Part of the Recommendation  Simulation Code – C++ Code  G.1050 Document (PDF)  Input PCAP files (without payloads)  TCL files for 43 Test Case scenarios .out files  Output PCAPs  PNGs  Results Summary (.csv)  Impairment Combination Table

41. Possible Uses for IP Network Impairment Simulation o Simulate Real-World IP network impairments  PDV Histogram and Packet Loss characteristics o Test any type of IP stream(s) (PCAP) under simulated network conditions  Standard Test Cases or user define simulated network conditions  Test with or without payload  Capture output PCAP for analysis and comparison o Test any type of IP stream using Hardware Emulation of simulated network model  Standard Test Cases or user define simulated network conditions

42. Example CORE-to-LAN Simulation Test Stream(s) Interfering Stream(s) HD VBR IPTV 2 QoS Interferers Bandwidth Interferers PDV & Loss MOS Calculated from captured PCAP OTT1 OTT2 SD VBR IPTV HTTP HD CBR IPTV QoS 1 2 7 2 7 7 VoIP Down

43. Example MOS Calculation Derived from Simulation Output PCAP Simulation Output PCAP Video Stream Analysis: ============================================= Stream source: 192.168.69.110:49713 Stream destination: 234.5.6.101:2345 Stream duration: 00:09:59.936 Stream CODEC: H.264 Stream transport: RTP (SSRC=0x11223344) Stream transport: MP2TS (PID=481, Ctrl=0x0000) Packet counts: received 2469009 lost 649 (0.00026) discarded 819 (0.00032) out-of-seq 6 (0.00000) duplicate0 (0.00000) corrected 0 (0.00000) MOS-V (min/avg/max/inst): 2.74/3.93/4.56/3.19 MOS-AV (min/avg/max/inst): 2.89/3.82/4.32/3.29 Captured output PCAPs can be use to analyze and compare competing algorithms Simulation.out Packet Delay and Loss Characteristic can be used to cause impairments on test streams

44. Example MOS V Plots Derived from Simulation Output PCAP MOS V (1s Intervals) for HDTV (CBR) Well Managed DLS 1 to 8 MOS V (1s Intervals) for SDTV (VBR) Partially Managed DLS 1 to 6

45. Hardware Emulation from IP Network Simulation Select a standard test case that includes a technology similar to the test stream Configure the emulator with the standard.out file for the selected test case Run the test stream through the emulator, and evaluate the effect of the network impairments on the stream

46. Advanced Use of the Network Model User-Defined Test Cases o Replace any of the standard interferer pcap files with user interfere pcap files o Change the traffic mix o Change network conditions  number of switches  bit error ratio  access technology parameters,  etc.

47. Future TIA-921/G.1050 Work o Active Network Emulation using Interferer PCAP files o Add Cable Access Characteristics o Add Wireless Access Characteristics o Update as IP Network evolves

48. Requests to SG12 o Provide comments and feedback on G.1050 Revised during session  Document  Simulation and Results  Electronic Attachments o Consider possible Consent of G.1050 Revised during the January session