Understanding the Impact of Network Dynamics on Mobile Video User Engagement M. Zubair Shafiq (Michigan State University) Jeffrey Erman (AT&T Labs - Research) Lusheng Ji (AT&T Labs - Research) Alex X. Liu (Michigan State University) Jeffrey Pang (AT&T Labs - Research) Jia Wang (AT&T Labs - Research)

2/20 Introduction  Online video is very popular on mobile networks Video makes up > 50% of global mobile data traffic Video traffic volume increasing (16x )  Video Quality of Experience (QoE) Mean Opinion Score (MOS) is not measureable at scale QoE goes beyond traditional QoS metrics

3/20 Background  Client-side Instrumentation [IMC’12][SIGCOMM’11] [SIGCOMM’13]  Information from video players and content servers  Buffering, startup delay, bitrate  Can these be extended for ISPs?  Doesn’t have end-to-end view of video streaming  Can only rely on network-side measurements ServerNetworkUser

Data

5/20 Architecture Overview  Cellular network architecture  RNCs control transmission scheduling and handovers  GGSNs anchor IP Tunnel to devices using GPRS tunneling protocol (GTP)

6/20 Data Collection  Real-world data: 27 terabytes, > 0.5 million users  Radio Access Network ─ RAB state, handover, bitrate, signal strength, RRC signaling  Core Network ─ TCP/IP headers  Ground truth: using HTTP information  Live vs. video-on-demand, mobile vs. desktop ─ Focus on mobile traffic of a popular video service provider ─ HTTP progressive download with byte-range requests  All traffic records are anonymized and aggregated, no personally identifiable information

What is Quality of Experience (QoE)?

8/20 Defining Video QoE  How to quantify QoE?  In terms of user engagement  Discreet  Abandoned/Completed  Skipped/Non-skipped  Continuous  Fraction of video streamed

9/20 Defining Video QoE  Completed, non-skipped (17.6%)  Abandoned, non-skipped (48.5%)  Completed, skipped (3.6%)  Abandoned, skipped (30.3%)

Measurement & Analysis of Network Factors

11/20 What’s the impact of network load? Network load increases abandonment rate

12/20 Is signal/interference a factor? More transmission power doesn’t help

13/20 Is signal/interference a factor? Interference plays a major role

14/20 Is more throughput helpful? Higher throughput does not always mean lower abandonment

Modeling User Engagement

16/20 Why Model? 1.Real time trending and alarming applications  Self-Organizing Network (SON) for dynamic resource allocation 2.Prioritize infrastructure update  Target the most important network factors first 3.Direct estimation from TCP/IP data alleviates cost and privacy concerns  No need for DPI

17/20 Predictive Model  Predict video abandonment within the initial portion ( Ƭ =10, 60 seconds) of a video session  Decision trees  SVM  Jointly use more than 150 features  Take into account non-linearity and inter-dependence

18/20 Classification Results  Discreet classification  Completed vs. Abandoned  Completed, Non-skipped vs. Rest

19/20 Limitations and Implications  Limitations  Traces from a single video provider  How to distinguish between abandonment due to lack of user interest and network issues  Actionable implications  Identify and prioritize vulnerable sessions  Prioritize infrastructure upgrades to target network features

20/20 Conclusion  First characterization of mobile video streaming from the perspective of network operators  Identify network factors that impact video QoE  Predictive model of video QoE  87% accuracy by observing the initial 10 seconds  Using only standard radio network and TCP/IP header information  Model allows large scale monitoring of video QoE

Understanding the Impact of Network Dynamics on Mobile Video User Engagement M. Zubair Shafiq (Michigan State University) Jeffrey Erman (AT&T Labs - Research) Lusheng Ji (AT&T Labs - Research) Alex X. Liu (Michigan State University) Jeffrey Pang (AT&T Labs - Research) Jia Wang (AT&T Labs - Research)