1. Testing ‘Video over TCP/IP’ on example of YouTube Streaming
Testing ‘Video over TCP/IP’ on example of YouTube Streaming
Dr. Jens Berger, SwissQual AG, Zuchwil, Switzerland
May 2012
© SwissQual 2012

2. YouTube as a service Some numbers (2010) YouTube and Video Streaming
Some numbers (2010)
Per minute users upload > 48h video material to YouTube
143 Mio users (means: accounts for upload) in July 2010
> 2000 Mio viewing sessions per day (End 2010)
Average viewing time per visitor (in one month) > 4:30h
>10% of the entire data traffic worldwide!
YouTube and Video Streaming
YouTube receives (HiQ) video content and trans-codes it
Usually H.264 (AVC) is applied along with a proprietary container format (FLV) for FlashPlayer
HTML 5 becomes used if possible, but the principle is the same
Original video is scaled to the next matching image size by YouTube
240p, 360p, 480p, 720p, 1080p: stand for lines, aspect ratio remains unchanged, usually 4:3 or 16:9
3D-Video, 2K / 4K HD under study
If original resolution is sufficient in size, a set of ‘smaller’ videos become created in parallel
YouTube is a moving target! Changed settings, controls, ... rapidely
 YouTube create a set of compressed video streams out of one uploaded video and deploys them across several cache servers (mirrors) where they can be accessed from
 YouTube controls the bitrate of the individual video streams by the (reduced) image size and compression ratio. 2

3. Testing YouTube as a service
Setup Context
Video Access 3

4. YouTube as a service YouTube and FlashPlayer YouTube becomes mobile
YouTube and FlashPlayer
YouTube enables FlashPlayer via a HTTP script
FlashPlayer has to be pre-installed on the host
Users just download a script for the player control, not the player itself
Request for player can control its appearance (e.g. ‘chromeless’, ‘autoplay’, desired image size)
YouTube becomes mobile
Special tailored YouTube site for mobile users (
Information about Operating System is given in the HTTP session (HTTP GET)
‚pre-Android‘ mobile devices don‘t support FlashPlayer, YouTube provides dedicated RTSP links for RealPlayer (e.g. Nokia)
Information about Video Player is given over RTSP (RTSP DESCRIBE)
Other mobile phones (Android) get 3GP via HTTP/TCP
Special sites / links for iPhone
Common mobile players: Real Player, Packet Video, QuickTime (iPhone), FlashPlayer (Android) 4

5. YouTube in the cloud 5

6. YouTube as a service 6

7. Testing YouTube as a service
Testing an application emulates at most the users experience
Video Streaming services in general can be subdivided into two phases
Setup Context (Request Service  Ready to play video)
Request website
DNS Resolution
Download HTML context (e.g. YouTube site by just requesting ‚the link‘ in a browser, link contains video ID)
Download player configuration (in case of FlashPlayer, HTML5)
Ready to play
Recent Browsers can parallelize HTTP requests (e.g. Buffering starts while HTML context is still downloaded)
Can include navigation
Download player information is not required for RealPlayer / QuickTime
Entire Setup Context phase is irrelevant for direct streaming a 3GP link
A YouTube App instead of YouTube in a browser will not load a HTML context rather only the player configuration
Video Service (access, buffer and play video) (‚Press Play‘ Video displayed completely / displaying stopped)
Request video
DNS Resolution to the target video clip
Pre-buffer video
Playback video 7

8. Testing YouTube as a service
Discussion – Setup Context
Resolve time DNS
Is not related to YouTube
Influences the perceived time to get response from the service
Time to download HTML context
HTML context changes from call to call (favorites, history, news)
Users don‘t recognize this time, rather the time until the player is configured
Time to download player configuration script
FlashPlayer configuration script downloaded via HTTP/TCP
This step does not exist for 3GP players as RealPlayer / QuickTime
‚Meta KPI‘: Setup Context  Ready to Buffering/Playing
The overall setup time of the context is the most meaningful trigger point in this HTTP part of the YouTube test or any ‚Video over TCP/IP‘ test
Individual sub-steps are subject of random network behaviour and not constant by definition
User stays interested due to upcoming information and rendering the web site
 Setup Context can be skipped by embedding players in an empty HTML page 8

9. Testing YouTube as a service
Discussion – YouTube video access, buffering and start displaying
Video Access Time
Time between ‚Ready to Play‘ and Start Buffering
Depends on resolving DNS and re-routing
Pre-Buffering time
Time between Start Buffering and Displaying of first image
Influenced by throughput and player / browser settings
Time to First Picture
Total time from first request to first image displayed
Can be derived from Player screen capture
Player log trace value
Time to Smooth
Total time until play-out is stabile at target frame-rate
What does Buffering Time tell us?
YouTube w/ FlashPlayer enables ‚progressive‘ download
After a short buffering time, the player starts to play while in the background, the buffer becomes filled more and more
Depending on the player and its settings more or less of the video becomes pre-stored
Experiences that an entire 5min clip was completely downloaded in 15s; most often, download is 30 to 90s ahead displaying
Request Video
Continous Download
Display Ends (last 90s is just playout from buffer)
Initial Download peak
Download Complete (~90s ahead to display) 9

10. Testing YouTube as a service
Discussion – YouTube Video Displaying
What is determining the ‚video quality‘
‘Original’ compression artifacts (driven by bit-rate and video complexity)
YouTube compresses to a certain ‚target rate‘ per resolution, quality depends on video complexity
Additional compression by the network before transmission to the airlink
Transmission problems (delayed and skipped packets, in unsafe protocols: erroneous video information)
In TCP no packets get lost or become destroyed, but they can come too late
Freezing (pausing without skipping) happens in case of buffer under-run
Freezing (pausing with skipping) can happen in case of device overload
What does Freezing Time tell us?
Progressive download enables an ahead buffering of 20s...90s usually
Build-up time of buffer depends on (current) network throughput
Once buffering is completed only heavy gaps in transmission causes a freeze followed by long re-buffering
Freezing should be set in relation to display duration! 10

11. Testing YouTube as a service
Main Concerns with today‘s approach
Today we are focussed on short term visual quality (MOS) and basic IP analysis
There is more than short term MOS and IP throughput
Actual artefacts as longer freezings and slicing with error propagation can‘t be evaluated in the 10s clip evlaluation concept
Longer evaluation times needed, for example 120s continous video playback
Concept:
Short term evaluation abbout actual compression quality (driven by Blockiness, Blurring, ...)
Aggregation (non-linear averaged similar to ETSI Quality per Call in telephony)
Combination with freezing (ratio, number of freezing events)
Consideration of buffering time
Finally, there is a service quality indictor, individual ‚dimensions‘ and even a time profile for compression and transmission quality available 11