Testing ‘Video over TCP/IP’ on example of YouTube Streaming 09.05.12 Testing ‘Video over TCP/IP’ on example of YouTube Streaming Dr. Jens Berger, SwissQual AG, Zuchwil, Switzerland May 2012 © SwissQual 2012

YouTube as a service Some numbers (2010) YouTube and Video Streaming 09.05.12 Some numbers (2010) Per minute users upload > 48h video material to YouTube 143 Mio users (means: accounts for upload) in July 2010 > 2000 Mio viewing sessions per day (End 2010) Average viewing time per visitor (in one month) > 4:30h >10% of the entire data traffic worldwide! YouTube and Video Streaming YouTube receives (HiQ) video content and trans-codes it Usually H.264 (AVC) is applied along with a proprietary container format (FLV) for FlashPlayer HTML 5 becomes used if possible, but the principle is the same Original video is scaled to the next matching image size by YouTube 240p, 360p, 480p, 720p, 1080p: stand for lines, aspect ratio remains unchanged, usually 4:3 or 16:9 3D-Video, 2K / 4K HD under study If original resolution is sufficient in size, a set of ‘smaller’ videos become created in parallel YouTube is a moving target! Changed settings, controls, ... rapidely  YouTube create a set of compressed video streams out of one uploaded video and deploys them across several cache servers (mirrors) where they can be accessed from  YouTube controls the bitrate of the individual video streams by the (reduced) image size and compression ratio. 2

Testing YouTube as a service 09.05.12 Setup Context Video Access 3

YouTube as a service YouTube and FlashPlayer YouTube becomes mobile 09.05.12 YouTube and FlashPlayer YouTube enables FlashPlayer via a HTTP script FlashPlayer has to be pre-installed on the host Users just download a script for the player control, not the player itself Request for player can control its appearance (e.g. ‘chromeless’, ‘autoplay’, desired image size) YouTube becomes mobile Special tailored YouTube site for mobile users (http://m.youtube.com) Information about Operating System is given in the HTTP session (HTTP GET) ‚pre-Android‘ mobile devices don‘t support FlashPlayer, YouTube provides dedicated RTSP links for RealPlayer (e.g. Nokia) Information about Video Player is given over RTSP (RTSP DESCRIBE) Other mobile phones (Android) get 3GP via HTTP/TCP Special sites / links for iPhone Common mobile players: Real Player, Packet Video, QuickTime (iPhone), FlashPlayer (Android) 4

YouTube in the cloud 09.05.12 5

YouTube as a service 09.05.12 6

Testing YouTube as a service 09.05.12 Testing an application emulates at most the users experience Video Streaming services in general can be subdivided into two phases Setup Context (Request Service  Ready to play video) Request website DNS Resolution Download HTML context (e.g. YouTube site by just requesting ‚the link‘ in a browser, link contains video ID) Download player configuration (in case of FlashPlayer, HTML5) Ready to play Recent Browsers can parallelize HTTP requests (e.g. Buffering starts while HTML context is still downloaded) Can include navigation Download player information is not required for RealPlayer / QuickTime Entire Setup Context phase is irrelevant for direct streaming a 3GP link A YouTube App instead of YouTube in a browser will not load a HTML context rather only the player configuration Video Service (access, buffer and play video) (‚Press Play‘ Video displayed completely / displaying stopped) Request video DNS Resolution to the target video clip Pre-buffer video Playback video 7

Testing YouTube as a service 09.05.12 Discussion – Setup Context Resolve time DNS Is not related to YouTube Influences the perceived time to get response from the service Time to download HTML context HTML context changes from call to call (favorites, history, news) Users don‘t recognize this time, rather the time until the player is configured Time to download player configuration script FlashPlayer configuration script downloaded via HTTP/TCP This step does not exist for 3GP players as RealPlayer / QuickTime ‚Meta KPI‘: Setup Context  Ready to Buffering/Playing The overall setup time of the context is the most meaningful trigger point in this HTTP part of the YouTube test or any ‚Video over TCP/IP‘ test Individual sub-steps are subject of random network behaviour and not constant by definition User stays interested due to upcoming information and rendering the web site  Setup Context can be skipped by embedding players in an empty HTML page 8

Testing YouTube as a service 09.05.12 Discussion – YouTube video access, buffering and start displaying Video Access Time Time between ‚Ready to Play‘ and Start Buffering Depends on resolving DNS and re-routing Pre-Buffering time Time between Start Buffering and Displaying of first image Influenced by throughput and player / browser settings Time to First Picture Total time from first request to first image displayed Can be derived from Player screen capture Player log trace value Time to Smooth Total time until play-out is stabile at target frame-rate What does Buffering Time tell us? YouTube w/ FlashPlayer enables ‚progressive‘ download After a short buffering time, the player starts to play while in the background, the buffer becomes filled more and more Depending on the player and its settings more or less of the video becomes pre-stored Experiences that an entire 5min clip was completely downloaded in 15s; most often, download is 30 to 90s ahead displaying Request Video Continous Download Display Ends (last 90s is just playout from buffer) Initial Download peak Download Complete (~90s ahead to display) 9

Testing YouTube as a service 09.05.12 Discussion – YouTube Video Displaying What is determining the ‚video quality‘ ‘Original’ compression artifacts (driven by bit-rate and video complexity) YouTube compresses to a certain ‚target rate‘ per resolution, quality depends on video complexity Additional compression by the network before transmission to the airlink Transmission problems (delayed and skipped packets, in unsafe protocols: erroneous video information) In TCP no packets get lost or become destroyed, but they can come too late Freezing (pausing without skipping) happens in case of buffer under-run Freezing (pausing with skipping) can happen in case of device overload What does Freezing Time tell us? Progressive download enables an ahead buffering of 20s...90s usually Build-up time of buffer depends on (current) network throughput Once buffering is completed only heavy gaps in transmission causes a freeze followed by long re-buffering Freezing should be set in relation to display duration! 10

Testing YouTube as a service 09.05.12 Main Concerns with today‘s approach Today we are focussed on short term visual quality (MOS) and basic IP analysis There is more than short term MOS and IP throughput Actual artefacts as longer freezings and slicing with error propagation can‘t be evaluated in the 10s clip evlaluation concept Longer evaluation times needed, for example 120s continous video playback Concept: Short term evaluation abbout actual compression quality (driven by Blockiness, Blurring, ...) Aggregation (non-linear averaged similar to ETSI Quality per Call in telephony) Combination with freezing (ratio, number of freezing events) Consideration of buffering time Finally, there is a service quality indictor, individual ‚dimensions‘ and even a time profile for compression and transmission quality available 11