Telecommunications Industry AssociationTR-30.3/ By Teleconference April 26, 2010

Jack Douglass VP of Marketing and Business Development Chairman of TIA TR30.3 Committee Standards Based Network Model for Evaluating Multimedia Transmission Performance Over IP

Agenda Standards Based IP Network Model –ITU-T G.1050 / ANSI TIA-921 G / TIA-921A –Top Down Approach G.1050–201X / TIA-921B (proposed revision) –Bottom-Up Approach Example Results Applications for G.1050/TIA-921 IP Network Model

Standards Based IP Network Model ITU-T G.1050 / ANSI TIA-921 Committees –TIA TR30.3 Committee –ITU-T SG12 Question 14 Range of test scenarios –Video, VoIP and Data –“Real-World” Network Characteristics Service Level Agreements (SLAs) based impairments (ITU-T Y.1541) –Well-Managed –Partially-Managed –Unmanaged conditions

Top Down Approach G / TIA-921A (Current Model) Surveyed many networks to evaluate the jitter and loss characteristics (Bursty, Not Random) Level of the impairment characteristics were adjusted to match the service levels in Y.1541 Created impairment combinations based on Impairment Severity Levels and LAN/Access Rates Multiple Two State Time Series Modified Gilbert-Elliott / Markov Models were used to emulate loss and delay characteristics –Emulated bursty packet loss and jitter that are similar to the surveyed network characteristics –Time based model with pre-calculated delay and loss

Network Impairments Network Architecture Types of Access Links QoS controlled Edge Routing MTU Size Packet Loss (Frame Loss) Out of order packets One Way Delay (Latency) Variable Delays (Jitter) Occupancy (Background Traffic, Congestion, Network Load) Route Flapping Network faults Link Failures

Network Topologies ITU-T G / TIA-921A LAN A Core IP Network DUT A Route flapping Link Failures One-way delay Jitter Packet loss Packet Reorder Local Access A DUT B LAN B Local Access B Access Rates Occupancy level Qos Edge Router Access Rates Occupancy level Qos Edge Router LAN Rates Occupancy level Packet loss Core IP Network Route flapping Link Failures One-way delay Jitter Packet loss Packet Reorder DUT B LAN B Local Access B DUT A Access Rates Occupancy level Qos Edge Router LAN Rates Occupancy level Packet loss LAN-to-LAN IP Network Model Core-to-LAN LAN Rates Occupancy level Packet loss

Test Profiles based on ITU-T Y1541 ITU-T G / TIA-921A Different test profiles for different Service Level Agreements (SLAs) Impairment TypeUnitsProfile A Well ‑ Managed Range (min to max) Profile B Partially-Managed Range (min to max) Profile C Unmanaged Range (min to max) One Way Latencyms20 to 100 (regional) 90 to 300 (intercontinental) 20 to 100 (regional) 90 to 400 (intercontinental) 20 to 500 Jitter (peak to peak)ms0 to 500 to 1500 to 500 Sequential Packet LossmsRandom loss only (except when link failure occurs) 40 to to 10,000 Rate of Sequential Losssec-1Random loss only (except when link failure occurs) < 10-3< 10-1 Random Packet Loss%0 to to 20 to 20 Reordered Packets%0 to to to 0.1

8 Impairment Severity Combinations ITU-T G and TIA-921A Impairment Severity Combinations Severity=>ABCDEFGH* ImpairmentUnits Profie A LOO% Profile B LOO% Profile C LOO% Source Location (A) Parameters LAN A Occupancy% Access A Occupancy% MTU Abytes Core Network Impairments Route flap intervalseconds Route flap delayms Delayms Jitter (peak to peak)ms Link fail intervalseconds Link fail durationms Packet loss% Reordered packets% Destination Location (B) Parameters Access B Occupancy% MTU Bbytes LAN B Occupancy%

LAN-Access Rate Combinations ITU-T G and TIA-921A 6 IPTV Core-to-LAN (> 3 Mbit/s) Rate Combinations 48 Impairment Combinations (6 rates * 8 Severities) 168 LAN-to-LAN Scenario Rate Combinations 1344 Impairment Combinations (168 rates * 8 Severities) 15 Core-to-LAN (Excluding IPTV≤ 3 Mbit/s) Rate Combinations 120 Impairment Combinations (15 rates * 8 Severities)

Testing with Fixed Levels of IP Impairments Real IP Network Impairments Vary Over Time significantly over time and are bursty in nature IP Impairment network emulators exist today that provide fixed or semi-fixed levels of IP impairments. Fixed Jitter & Packet Loss Fixed Packet Jitter

Real World Impairment Characteristics Good Bad Severe 1344 General Model 120 Core to LAN ≤ 3 Mbit/s 48 Core to LAN > 3 Mbit/s Link Failure Route Flap

IP Impairments IPTV Testing G /TIA-921A 1XA 1XB 1XC 1XD 1XE 1XF 1XG 1XH 48 Core to LAN > 3 Mbit/s Impairment Combinations

Reasons Revising G / TIA-921A Increased Realism Keep in step with evolving IP network Reduced number of test cases –Small set of Standard Long Duration Test Cases –Focused Test Cases for transient impairments such as link failures and route flaps –Short burst tests (15 seconds) for voice/video subjective testing and fax True bi-directional model Ability to test with mixed traffic Statistically repeatable tests Ability for users to customize test cases and use their own.pcap files to model interferers. Methods for verifying that test equipment implementations match the model

Bottom-Up Approach G X / TIA-921B (Revised Model) Model interfering traffic –Capture different types of real network traffic –Create Packet Delay Variation (PDV) Histograms from captured traffic –Make interferers scalable Model the Mechanisms that cause Jitter, Delay and Loss –Packet loss and delay are caused by interferers and self-interferers Model of each Network Element (Core, Access, LAN) –Capture characteristics of each Network Elements –Capture characteristics of Access and LAN technologies DSL, Cable, Fiber Optic, WiMAX, LTE; LAN – Wi-Fi, Ethernet, HPNA –Bandwidth Throttling, Smoothing and Shaping Effects of network elements –Bi-directional characteristics Model Quality of Service (QoS) Adjust and scale the amounts and types of interfering traffic –Match Surveyed Network Characteristics –Match Service Levels in Y.1541 A similar model is being considered in ITU-T SG15 Question 13 – G.8261

Test Profiles based on ITU-T Y1541 ITU-T G X / TIA-921B (Revised Model) Different test profiles for different Service Level Agreements (SLAs) Impairment TypeUnitsProfile A Well ‑ Managed Range (min to max) Profile B Partially-Managed Range (min to max) Profile C Unmanaged Range (min to max) One Way Latencyms20 to 100 (regional) 90 to 300 (intercontinental) 20 to 100 (regional) 90 to 400 (intercontinental) 20 to 500 Jitter (peak to peak)ms0 to 500 to 1500 to 500 Sequential Packet LossmsRandom loss only (except when link failure occurs) 40 to to 10,000 Rate of Sequential Losssec-1Random loss only (except when link failure occurs) < 10-3< 10-1 Random Packet Loss%0 to to 20 to 20 Reordered Packets%0 to to to 0.1

Impairment Combination Table Residential Impairment Combinations in the direction from the core Network Element Well Managed Partially Managed Unmanaged ImpairmentSev1Sev2Sev3 Sev4Sev5Sev6 Sev7Sev8Sev9 QoS IPTV VoIP (primary) Data Test streamYYY ??N NNN LAN - wired Effective Rate (Mbit/s)100 Peer-to-peer001 POP3001 HTTP011 Web video000 VoIP (secondary)010 FoIP (secondary)001 Gaming000 SAP000 LAN - wireless Effective Rate (Mbit/s)20 4 Peer-to-peer000 POP3111 HTTP112 Web video001 VoIP (secondary)001 FoIP (secondary)000 Gaming011 SAP001 Firewall Buffer size64 KB Rate (Mbit/s) Proc delay (µs)100 "Modem" Buffer size64 KB Proc delay000 VoIP (primary) FoIP (primary) IPTV service Access (pick one technology) DSLAccess Rate (Mbit/s) CableAccess Rate (Mbit/s) ONTAccess Rate (Mbit/s)100

Network Topologies ITU-T G X / TIA-921B (Revised Model) Core-to-LAN LAN-to-LAN IP Network Model

LAN and Access Block Diagram ITU-T G X / TIA-921B (Revised Model) Test Streams IPTV Web Video VoIP / FoIP DATA Network Interferers TCP Peer-to-Peer POP3 protocol HTTP web browsing Web Video (You Tube, Hulu) VoIP FoIP Gaming Network Interferers TCP Peer-to-Peer POP3 protocol HTTP web browsing Web Video (You Tube, Hulu) VoIP FoIP Gaming LAN DSLAM / CMTS / OLT DSL Modem / Cable Modem / ONT FirewallEdge Router Wired Wireless Residential Gateway

Capturing “Real-World” Characteristics Capture Packet Delay Variation (PDV) characteristics –Access and LAN Technologies –Different types of Interferers (network traffic ) using.pcap (from Wireshark) Result is composite Packet Delay Variation (PDV) for combined Access/LAN and Interferers

Disturbance load generator Basic Model Element ITU-T G X / TIA-921B 21 Disturbance load generator + Input packets Output packets Link Latency Simulated Packet Queue with Loss Store/ Fwd

Network Model Structure (Simplified) Interferers From PCAP IPTV HTTP POP3 Stream Under Test Edge Router DSLAM/ CMTS OLT DSL Modem/ Cable Modem/ ONT Firewall/ Router LAN 100M 20Mb/s or 30Mb/s 100M 1G 64kB buffer 64kB buffer 64kB buffer 64kB buffer 1ms 100ns 0ns Stream Under Test Composite Interference STB

HTTP Interference Characteristics Delay (milliseconds) Packet Delay Variation (PDV) HistogramDelay vs. Time

IPTV Interference (VBR) Characteristics Delay (milliseconds) Packet Delay Variation (PDV) HistogramDelay vs. Time

POP3 Interference Characteristics Delay (milliseconds) Packet Delay Variation (PDV) HistogramDelay vs. Time

Example #1 Test Streams STB#1 IPTV – SD 2Mb/s – QoS 1 Access DSL 20Mb/s Network Interferers PC#1, HTTP web browsing – QoS 2 PC#2, POP3 protocol – QoS 2 STB#2, IPTV – SD (2 Mb/s) – QoS 1 STB#3, IPTV – SD (2 Mb/s) – QoS 1 Network Interferers PC#1, HTTP web browsing – QoS 2 PC#2, POP3 protocol – QoS 2 STB#2, IPTV – SD (2 Mb/s) – QoS 1 STB#3, IPTV – SD (2 Mb/s) – QoS 1 LAN DSLAM / CMTS / OLT DSL Modem / Cable Modem / ONT FirewallEdge Router Wired Wireless Residential Gateway

Example #1 Network Interferers PC#1, HTTP web browsing – QoS 2 PC#2, POP3 protocol – QoS 2 STB#2, IPTV – SD (2 Mb/s) – QoS 1 STB#3, IPTV – SD (2 Mb/s) – QoS 1 Test Streams STB#1, IPTV – SD 2Mb/s – QoS 1 Access DSL 20Mb/s

Example #1 Test Stream Packet Delay Variation (PDV) HistogramDelay vs. Time Test Streams STB#1, IPTV – SD 2Mb/s – QoS 1 Access DSL 20Mb/s

Example #2 Test Streams STB#1, IPTV – HD 8Mb/s – QoS 1 Access DSL 20Mb/s LAN DSLAM / CMTS / OLT DSL Modem / Cable Modem / ONT FirewallEdge Router Wired Wireless Residential Gateway Network Interferers PC#1, HTTP web browsing – QoS 2 PC#2, POP3 protocol – QoS 2 STB#2, IPTV – SD (2 Mb/s) – QoS 1 STB#3, IPTV – SD (2 Mb/s) – QoS 1 Network Interferers PC#1, HTTP web browsing – QoS 2 PC#2, POP3 protocol – QoS 2 STB#2, IPTV – SD (2 Mb/s) – QoS 1 STB#3, IPTV – SD (2 Mb/s) – QoS 1

Example #2 Network Interferers PC#1, HTTP web browsing – QoS 2 PC#2, POP3 protocol – QoS 2 STB#2, IPTV – SD (2 Mb/s) – QoS 1 STB#3, IPTV – SD (2 Mb/s) – QoS 1 Test Streams STB#1, IPTV – HD 8Mb/s – QoS 1 Access DSL 20Mb/s

Example #2 Test Stream Packet Delay Variation (PDV) HistogramDelay vs. Time Test Streams STB#1, IPTV – HD 8Mb/s – QoS 1 Access DSL 20Mb/s

Example #3 Test Streams STB#1, IPTV – HD 8Mb/s – QoS 1 Access DSL 30Mb/s Network Interferers PC#1, HTTP web browsing – QoS 2 PC#2, POP3 protocol – QoS 2 STB#2, IPTV – SD (2 Mb/s) – QoS 1 STB#3, IPTV – SD (2 Mb/s) – QoS 1 Network Interferers PC#1, HTTP web browsing – QoS 2 PC#2, POP3 protocol – QoS 2 STB#2, IPTV – SD (2 Mb/s) – QoS 1 STB#3, IPTV – SD (2 Mb/s) – QoS 1 LAN DSLAM / CMTS / OLT DSL Modem / Cable Modem / ONT FirewallEdge Router Wired Wireless Residential Gateway

Example #3 Network Interferers PC#1, HTTP web browsing – QoS 2 PC#2, POP3 protocol – QoS 2 STB#2, IPTV – SD (2 Mb/s) – QoS 1 STB#3, IPTV – SD (2 Mb/s) – QoS 1 Test Streams STB#1, IPTV – HD 8Mb/s – QoS 1 Access DSL 30Mb/s

Example #3 Test Stream Packet Delay Variation (PDV) HistogramDelay vs. Time Test Streams STB#1, IPTV – HD 8Mb/s – QoS 1 Access DSL 30Mb/s

Applications G.1050/TIA-921– Compare the Encoders and STBs Video Encoders A B C A B C STBs G / TIA-921A Impairments o Compatibility of Encoder and Decoder o Effectiveness Decoders to conceal network errors o Stability of STB under network conditions o Regression Testing o Standard, Definitive, Repeatable set of tests representing Real-World IP Network Conditions Imp Gen

Applications G.1050/TIA-921– Emulate Field Problems, Error Correcting, Server Loading A Server G / TIA-921A Impairments D Server RUDP o Emulate field problems o Emulate effectiveness of error correcting scheme o Test Loading of A&D Server in presence of Network Errors o Test ability of system to meet required specifications Standard, Definitive, Repeatable set of tests representing Service Levels Imp Gen

Applications G.1050/TIA-921 – Effectiveness of FEC Video Server G / TIA-921A Impairments o Emulate effectiveness of Forward Error Correcting (FEC) Techniques and Equipment o Test ability of FEC to meet required specifications Standard, Definitive, Repeatable set of tests representing Real-World IP Network Conditions FEC Equipment FEC Equipment Imp Gen

Applications G.1050/TIA-921 – General Network Emulation DUT A DUT B Imp Gen Test Any Type of Network Equipment under Real-World Network Characteristics DUT B

TR30.3 and ITU-T SG12 Welcome Your Input to Make G.1050 More Accurate