1. Telecommunications Industry AssociationTR-30.3/10-04-011 By Teleconference April 26, 2010

2. Jack Douglass VP of Marketing and Business Development Chairman of TIA TR30.3 Committee Standards Based Network Model for Evaluating Multimedia Transmission Performance Over IP

3. Agenda Standards Based IP Network Model –ITU-T G.1050 / ANSI TIA-921 G.1050-2007 / TIA-921A –Top Down Approach G.1050–201X / TIA-921B (proposed revision) –Bottom-Up Approach Example Results Applications for G.1050/TIA-921 IP Network Model

4. Standards Based IP Network Model ITU-T G.1050 / ANSI TIA-921 Committees –TIA TR30.3 Committee –ITU-T SG12 Question 14 Range of test scenarios –Video, VoIP and Data –“Real-World” Network Characteristics Service Level Agreements (SLAs) based impairments (ITU-T Y.1541) –Well-Managed –Partially-Managed –Unmanaged conditions

5. Top Down Approach G.1050-2007 / TIA-921A (Current Model) Surveyed many networks to evaluate the jitter and loss characteristics (Bursty, Not Random) Level of the impairment characteristics were adjusted to match the service levels in Y.1541 Created impairment combinations based on Impairment Severity Levels and LAN/Access Rates Multiple Two State Time Series Modified Gilbert-Elliott / Markov Models were used to emulate loss and delay characteristics –Emulated bursty packet loss and jitter that are similar to the surveyed network characteristics –Time based model with pre-calculated delay and loss

6. Network Impairments Network Architecture Types of Access Links QoS controlled Edge Routing MTU Size Packet Loss (Frame Loss) Out of order packets One Way Delay (Latency) Variable Delays (Jitter) Occupancy (Background Traffic, Congestion, Network Load) Route Flapping Network faults Link Failures

7. Network Topologies ITU-T G.1050-2007 / TIA-921A LAN A Core IP Network DUT A Route flapping Link Failures One-way delay Jitter Packet loss Packet Reorder Local Access A DUT B LAN B Local Access B Access Rates ---------------- Occupancy level Qos Edge Router Access Rates ---------------- Occupancy level Qos Edge Router LAN Rates ----------------- Occupancy level Packet loss Core IP Network Route flapping Link Failures One-way delay Jitter Packet loss Packet Reorder DUT B LAN B Local Access B DUT A Access Rates ---------------- Occupancy level Qos Edge Router LAN Rates ----------------- Occupancy level Packet loss LAN-to-LAN IP Network Model Core-to-LAN LAN Rates ----------------- Occupancy level Packet loss

8. Test Profiles based on ITU-T Y1541 ITU-T G.1050-2007 / TIA-921A Different test profiles for different Service Level Agreements (SLAs) Impairment TypeUnitsProfile A Well ‑ Managed Range (min to max) Profile B Partially-Managed Range (min to max) Profile C Unmanaged Range (min to max) One Way Latencyms20 to 100 (regional) 90 to 300 (intercontinental) 20 to 100 (regional) 90 to 400 (intercontinental) 20 to 500 Jitter (peak to peak)ms0 to 500 to 1500 to 500 Sequential Packet LossmsRandom loss only (except when link failure occurs) 40 to 20040 to 10,000 Rate of Sequential Losssec-1Random loss only (except when link failure occurs) < 10-3< 10-1 Random Packet Loss%0 to 0.050 to 20 to 20 Reordered Packets%0 to 0.0010 to 0.010 to 0.1

9. 8 Impairment Severity Combinations ITU-T G.1050-2007 and TIA-921A Impairment Severity Combinations Severity=>ABCDEFGH* ImpairmentUnits Profie A LOO%50301550000 Profile B LOO%525302510500 Profile C LOO%5510152025155 Source Location (A) Parameters LAN A Occupancy%12358121620 Access A Occupancy%01248153050 MTU Abytes512 1508 Core Network Impairments Route flap intervalseconds03600180090048024012060 Route flap delayms0248163264128 Delayms48163264128256512 Jitter (peak to peak)ms510244070100150500 Link fail intervalseconds03600180090048024012060 Link fail durationms06412825640080016003000 Packet loss%00.010.020.040.10.20.51 Reordered packets%00.000250.00050.0010.0050.010.050.1 Destination Location (B) Parameters Access B Occupancy%01248153050 MTU Bbytes512 1508 LAN B Occupancy%12358121620

10. LAN-Access Rate Combinations ITU-T G.1050-2007 and TIA-921A 6 IPTV Core-to-LAN (> 3 Mbit/s) Rate Combinations 48 Impairment Combinations (6 rates * 8 Severities) 168 LAN-to-LAN Scenario Rate Combinations 1344 Impairment Combinations (168 rates * 8 Severities) 15 Core-to-LAN (Excluding IPTV≤ 3 Mbit/s) Rate Combinations 120 Impairment Combinations (15 rates * 8 Severities)

11. Testing with Fixed Levels of IP Impairments Real IP Network Impairments Vary Over Time significantly over time and are bursty in nature IP Impairment network emulators exist today that provide fixed or semi-fixed levels of IP impairments. Fixed Jitter & Packet Loss Fixed Packet Jitter

12. Real World Impairment Characteristics Good Bad Severe 1344 General Model 120 Core to LAN ≤ 3 Mbit/s 48 Core to LAN > 3 Mbit/s Link Failure Route Flap

13. IP Impairments IPTV Testing G.1050-2007/TIA-921A 1XA 1XB 1XC 1XD 1XE 1XF 1XG 1XH 48 Core to LAN > 3 Mbit/s Impairment Combinations

14. Reasons Revising G.1050-2007 / TIA-921A Increased Realism Keep in step with evolving IP network Reduced number of test cases –Small set of Standard Long Duration Test Cases –Focused Test Cases for transient impairments such as link failures and route flaps –Short burst tests (15 seconds) for voice/video subjective testing and fax True bi-directional model Ability to test with mixed traffic Statistically repeatable tests Ability for users to customize test cases and use their own.pcap files to model interferers. Methods for verifying that test equipment implementations match the model

15. Bottom-Up Approach G.1050-201X / TIA-921B (Revised Model) Model interfering traffic –Capture different types of real network traffic –Create Packet Delay Variation (PDV) Histograms from captured traffic –Make interferers scalable Model the Mechanisms that cause Jitter, Delay and Loss –Packet loss and delay are caused by interferers and self-interferers Model of each Network Element (Core, Access, LAN) –Capture characteristics of each Network Elements –Capture characteristics of Access and LAN technologies DSL, Cable, Fiber Optic, WiMAX, LTE; LAN – Wi-Fi, Ethernet, HPNA –Bandwidth Throttling, Smoothing and Shaping Effects of network elements –Bi-directional characteristics Model Quality of Service (QoS) Adjust and scale the amounts and types of interfering traffic –Match Surveyed Network Characteristics –Match Service Levels in Y.1541 A similar model is being considered in ITU-T SG15 Question 13 – G.8261

16. Test Profiles based on ITU-T Y1541 ITU-T G.1050-201X / TIA-921B (Revised Model) Different test profiles for different Service Level Agreements (SLAs) Impairment TypeUnitsProfile A Well ‑ Managed Range (min to max) Profile B Partially-Managed Range (min to max) Profile C Unmanaged Range (min to max) One Way Latencyms20 to 100 (regional) 90 to 300 (intercontinental) 20 to 100 (regional) 90 to 400 (intercontinental) 20 to 500 Jitter (peak to peak)ms0 to 500 to 1500 to 500 Sequential Packet LossmsRandom loss only (except when link failure occurs) 40 to 20040 to 10,000 Rate of Sequential Losssec-1Random loss only (except when link failure occurs) < 10-3< 10-1 Random Packet Loss%0 to 0.050 to 20 to 20 Reordered Packets%0 to 0.0010 to 0.010 to 0.1

17. Impairment Combination Table Residential Impairment Combinations in the direction from the core Network Element Well Managed Partially Managed Unmanaged ImpairmentSev1Sev2Sev3 Sev4Sev5Sev6 Sev7Sev8Sev9 QoS IPTV222 222 333 VoIP (primary)111 123 333 Data333 333 333 Test streamYYY ??N NNN LAN - wired Effective Rate (Mbit/s)100 Peer-to-peer001 POP3001 HTTP011 Web video000 VoIP (secondary)010 FoIP (secondary)001 Gaming000 SAP000 LAN - wireless Effective Rate (Mbit/s)20 4 Peer-to-peer000 POP3111 HTTP112 Web video001 VoIP (secondary)001 FoIP (secondary)000 Gaming011 SAP001 Firewall Buffer size64 KB Rate (Mbit/s)100 10 Proc delay (µs)100 "Modem" Buffer size64 KB Proc delay000 VoIP (primary) FoIP (primary) IPTV service Access (pick one technology) DSLAccess Rate (Mbit/s)332214 7 CableAccess Rate (Mbit/s)100 50 ONTAccess Rate (Mbit/s)100

18. Network Topologies ITU-T G.1050-201X / TIA-921B (Revised Model) Core-to-LAN LAN-to-LAN IP Network Model

19. LAN and Access Block Diagram ITU-T G.1050-201X / TIA-921B (Revised Model) Test Streams IPTV Web Video VoIP / FoIP DATA Network Interferers TCP Peer-to-Peer POP3 email protocol HTTP web browsing Web Video (You Tube, Hulu) VoIP FoIP Gaming Network Interferers TCP Peer-to-Peer POP3 email protocol HTTP web browsing Web Video (You Tube, Hulu) VoIP FoIP Gaming LAN DSLAM / CMTS / OLT DSL Modem / Cable Modem / ONT FirewallEdge Router Wired Wireless Residential Gateway

20. Capturing “Real-World” Characteristics Capture Packet Delay Variation (PDV) characteristics –Access and LAN Technologies –Different types of Interferers (network traffic ) using.pcap (from Wireshark) Result is composite Packet Delay Variation (PDV) for combined Access/LAN and Interferers

21. Disturbance load generator Basic Model Element ITU-T G.1050-201X / TIA-921B 21 Disturbance load generator + Input packets Output packets Link Latency Simulated Packet Queue with Loss Store/ Fwd

22. Network Model Structure (Simplified) Interferers From PCAP IPTV HTTP POP3 Stream Under Test Edge Router DSLAM/ CMTS OLT DSL Modem/ Cable Modem/ ONT Firewall/ Router LAN 100M 20Mb/s or 30Mb/s 100M 1G 64kB buffer 64kB buffer 64kB buffer 64kB buffer 1ms 100ns 0ns Stream Under Test Composite Interference STB

23. HTTP Interference Characteristics Delay (milliseconds) Packet Delay Variation (PDV) HistogramDelay vs. Time

24. IPTV Interference (VBR) Characteristics Delay (milliseconds) Packet Delay Variation (PDV) HistogramDelay vs. Time

25. POP3 Interference Characteristics Delay (milliseconds) Packet Delay Variation (PDV) HistogramDelay vs. Time

26. Example #1 Test Streams STB#1 IPTV – SD 2Mb/s – QoS 1 Access DSL 20Mb/s Network Interferers PC#1, HTTP web browsing – QoS 2 PC#2, POP3 email protocol – QoS 2 STB#2, IPTV – SD (2 Mb/s) – QoS 1 STB#3, IPTV – SD (2 Mb/s) – QoS 1 Network Interferers PC#1, HTTP web browsing – QoS 2 PC#2, POP3 email protocol – QoS 2 STB#2, IPTV – SD (2 Mb/s) – QoS 1 STB#3, IPTV – SD (2 Mb/s) – QoS 1 LAN DSLAM / CMTS / OLT DSL Modem / Cable Modem / ONT FirewallEdge Router Wired Wireless Residential Gateway

27. Example #1 Network Interferers PC#1, HTTP web browsing – QoS 2 PC#2, POP3 email protocol – QoS 2 STB#2, IPTV – SD (2 Mb/s) – QoS 1 STB#3, IPTV – SD (2 Mb/s) – QoS 1 Test Streams STB#1, IPTV – SD 2Mb/s – QoS 1 Access DSL 20Mb/s

28. Example #1 Test Stream Packet Delay Variation (PDV) HistogramDelay vs. Time Test Streams STB#1, IPTV – SD 2Mb/s – QoS 1 Access DSL 20Mb/s

29. Example #2 Test Streams STB#1, IPTV – HD 8Mb/s – QoS 1 Access DSL 20Mb/s LAN DSLAM / CMTS / OLT DSL Modem / Cable Modem / ONT FirewallEdge Router Wired Wireless Residential Gateway Network Interferers PC#1, HTTP web browsing – QoS 2 PC#2, POP3 email protocol – QoS 2 STB#2, IPTV – SD (2 Mb/s) – QoS 1 STB#3, IPTV – SD (2 Mb/s) – QoS 1 Network Interferers PC#1, HTTP web browsing – QoS 2 PC#2, POP3 email protocol – QoS 2 STB#2, IPTV – SD (2 Mb/s) – QoS 1 STB#3, IPTV – SD (2 Mb/s) – QoS 1

30. Example #2 Network Interferers PC#1, HTTP web browsing – QoS 2 PC#2, POP3 email protocol – QoS 2 STB#2, IPTV – SD (2 Mb/s) – QoS 1 STB#3, IPTV – SD (2 Mb/s) – QoS 1 Test Streams STB#1, IPTV – HD 8Mb/s – QoS 1 Access DSL 20Mb/s

31. Example #2 Test Stream Packet Delay Variation (PDV) HistogramDelay vs. Time Test Streams STB#1, IPTV – HD 8Mb/s – QoS 1 Access DSL 20Mb/s

32. Example #3 Test Streams STB#1, IPTV – HD 8Mb/s – QoS 1 Access DSL 30Mb/s Network Interferers PC#1, HTTP web browsing – QoS 2 PC#2, POP3 email protocol – QoS 2 STB#2, IPTV – SD (2 Mb/s) – QoS 1 STB#3, IPTV – SD (2 Mb/s) – QoS 1 Network Interferers PC#1, HTTP web browsing – QoS 2 PC#2, POP3 email protocol – QoS 2 STB#2, IPTV – SD (2 Mb/s) – QoS 1 STB#3, IPTV – SD (2 Mb/s) – QoS 1 LAN DSLAM / CMTS / OLT DSL Modem / Cable Modem / ONT FirewallEdge Router Wired Wireless Residential Gateway

33. Example #3 Network Interferers PC#1, HTTP web browsing – QoS 2 PC#2, POP3 email protocol – QoS 2 STB#2, IPTV – SD (2 Mb/s) – QoS 1 STB#3, IPTV – SD (2 Mb/s) – QoS 1 Test Streams STB#1, IPTV – HD 8Mb/s – QoS 1 Access DSL 30Mb/s

34. Example #3 Test Stream Packet Delay Variation (PDV) HistogramDelay vs. Time Test Streams STB#1, IPTV – HD 8Mb/s – QoS 1 Access DSL 30Mb/s

35. Applications G.1050/TIA-921– Compare the Encoders and STBs Video Encoders A B C A B C STBs G.1050-2007 / TIA-921A Impairments o Compatibility of Encoder and Decoder o Effectiveness Decoders to conceal network errors o Stability of STB under network conditions o Regression Testing o Standard, Definitive, Repeatable set of tests representing Real-World IP Network Conditions Imp Gen

36. Applications G.1050/TIA-921– Emulate Field Problems, Error Correcting, Server Loading A Server G.1050-2007 / TIA-921A Impairments D Server RUDP o Emulate field problems o Emulate effectiveness of error correcting scheme o Test Loading of A&D Server in presence of Network Errors o Test ability of system to meet required specifications Standard, Definitive, Repeatable set of tests representing Service Levels Imp Gen

37. Applications G.1050/TIA-921 – Effectiveness of FEC Video Server G.1050-2007 / TIA-921A Impairments o Emulate effectiveness of Forward Error Correcting (FEC) Techniques and Equipment o Test ability of FEC to meet required specifications Standard, Definitive, Repeatable set of tests representing Real-World IP Network Conditions FEC Equipment FEC Equipment Imp Gen

38. Applications G.1050/TIA-921 – General Network Emulation DUT A DUT B Imp Gen Test Any Type of Network Equipment under Real-World Network Characteristics DUT B

39. TR30.3 and ITU-T SG12 Welcome Your Input to Make G.1050 More Accurate