1. CPS 214 Computer Networks and Distributed Systems
“Live” Video and Audio Streaming
End System Multicast
Analysis of Akamai Workload

2. Presentations Monday, April 21 Wednesday, April 23 Abhinav, Risi
Bi, Jie
Jason, Michael
Martin, Matt
Amre, Kareem
Wednesday, April 23
Ben, Kyle
Jayan, Michael
Kshipra, Peng
Xuhan, Yang

3. Aditya Ganjam, Bruce Maggs*, and Hui Zhang
The Feasibility of Supporting Large-Scale Live Streaming Applications with Dynamic Application End-Points
Kay Sripanidkulchai,
Aditya Ganjam, Bruce Maggs*, and Hui Zhang
Carnegie Mellon University
* and Akamai Technologies
Color

4. Motivation Ubiquitous Internet broadcast Anyone can broadcast
Anyone can tune in

5. Overlay multicast architectures
App end-point vs. Infrastructure vs. waypoints
Transition: infrastructure -> remove them
Router
Source
Application end-point

6. Infrastructure-based architecture [Akamai]
+ Well-provisioned
App end-point vs. Infrastructure vs. waypoints
Transition: infrastructure -> remove them
Router
Source
Application end-point
Infrastructure server

7. Application end-point architecture [End System Multicast (ESM)]
+ Instantly deployable
+ Enables ubiquitous
broadcast
App end-point vs. Infrastructure vs. waypoints
Transition: infrastructure -> remove them
Router
Source
Application end-point

8. Waypoint architecture [ESM]
+ Waypoints as insurance
Order : comes after purely app end-point
App end-point vs. Infrastructure vs. waypoints
Transition: infrastructure -> remove them
Router
Source
Application end-point
Waypoint W

9. Sample ESM Broadcasts http://esm.cs.cmu.edu
Event
Duration (hours)
Unique Hosts
Peak Size
SIGCOMM ’02 25
338 83
SIGCOMM ’03 72
705
101
SOSP’03 24
401 56
DISC’03 16 30 20
Distinguished Lectures 11
400 80
AID Meeting 14 43
Buggy Race 85 44
Slashdot
1609
160
Grand Challenge 6
2005
280
Change the number based on the tech report. Perhaps we can merge the performace num here with new columns.
. Entity duration and incarnation duration may be interesting, we don’t have all of them here so we don’t put them in the table
examples for sigcomm’03: 29 min/71min.

10. Feasibility of supporting large-scale groups with an application end-point architecture?
Is the overlay stable enough despite dynamic participation?
Is there enough upstream bandwidth?
Are overlay structures efficient?
Break into 2 slides

11. Large-scale groups
Challenging to address these fundamental feasibility questions
Little knowledge of what large-scale live streaming is like
Break into 2 slides

12. Chicken and egg problem
Publishers with compelling content need proof that the system works.
System has not attracted large-scale groups due to lack of compelling content.

13. The focus of this paper
Generate new insight on the feasibility of application end-point architectures for large scale broadcast
Our methodology to break the cycle
Analysis and simulation
Leverage an extensive set of real-world workloads from Akamai (infrastructure-based architecture)
Merge with previous, mention with the other Akamai slides
Drop these points
well-provisioned live streaming infrastructure
compelling content -> large-scale groups

14. Talk outline Akamai live streaming workload
With an application end-point architecture
Is the overlay stable enough despite dynamic participation?
Is there enough upstream bandwidth?
Are overlay structures efficient?
Summary

15. Measurements used in this study
Akamai live streaming traces
Trace format for a request
[IP, Stream URL, Session start time, Session duration]
Additional measurements collected
Hosts’ upstream bandwidth

16. An Analysis of Live Streaming Workloads on the Internet
Kunwadee Sripanidkulchai, Bruce Maggs*, Hui Zhang
Carnegie Mellon University and
*Akamai Technologies

17. Akamai live streaming infrastructure
Source A A A A
Unicast connections to each client
Reflectors
Edge servers

18. Extensive traces 1,000,000 daily requests
200,000 daily client IP addresses from over 200 countries
1,000 daily streams
1,000 edge servers
Everyday, over a 3-month period

19. Largest stream 75,000 x 250 kbps = 18 Gbps!
250 kbps => total bandwidth is huge

20. Highlight of findings Popularity of events [Bimodal Zipf]
Session arrivals [Exponential for short time-scales, time-of-day and time-zone-correlated behavior, LOTS of flash crowds]
Session durations [Heavy-tailed]
Transport protocol usage [TCP rivals UDP]
Client lifetime
Client diversity

21. Request volume (daily)
Weekdays
Number of requests
Weekends
Missing logs

22. Audio vs. video Video 7% Most streams are audio. Audio 71% Unknown 22%
The logs do not provide content type information. To determine whether a stream is audio or video, we look at the streaming bit rate which is defined as the median bit rate received by the receivers for that given stream.
Unknown 22%

23. Stream types Non-stop (76%) vs. short duration (24%)
All video streams have short duration
Smooth arrivals (50%) vs. flash crowds (50%)
Flash crowds are common
Flash crowds are common! The figures show group membership vs. time for 3 different streams. The first is non-stop + smooth arrival. The second is non-stop + flash crowd. The third is short duration + flash crowd.

24. Client lifetime Motivating questions Want to know
Should servers maintain “persistent” state about clients (for content customization)?
Should clients maintain server history (for server selection problems)?
Want to know
Are new clients tuning in to an event?
What is the lifetime of a client?
Windows media had a lot of requests.

25. Analysis methodology Windows media format
Player ID field to identify distinct users
Birth rate = Number of new distinct users
Total number of distinct users
New distinct users are defined as new with respect to all previous days.

26. Daily new client birth rate
New client birth rate is % across all events.
For these 2 events, birth rate is 10-30%.
Weekends
Weekdays
Xmas
Two events here. For the red curve (US radio station), peaks in the birth rate correspond to weekends. People are tuning in from home? For the blue curve, there was a peak during Xmas holidays. Perhaps because people are using their home computers or their relatives’ computers to tune in.

27. One-timers: tune in for only 1 day
In almost all events, 50% of clients
are one-timers!
Possible explanations for one-timers: (1) checked out the content and didn’t like it, (2) infrequent events (once a month compared to everyday) have more one-timers

28. Client lifetime (excluding one-timers)
y = 3x
y = x
For most events, average client lifetime
is at least 1/3 of the event duration.

29. Client lifetime Motivating questions
Should servers maintain “persistent” state about clients (for content customization)? Any state should time-out quickly because most clients are one-timers.
Should clients maintain server history (for server selection problems)? Yes, recurring clients tend to hang around for a while.
Windows media had a lot of requests.

30. Where are clients from? Number of IP Addresses Countries
Clients are from over 200 countries.
Most clients are from the US and Europe.

31. Analysis methodology
Map client IP to location using Akamai’s EdgeScape tool
Definitions
Diversity index = Number of distinct ‘locations’ that a stream reaches
Large streams are streams that have a peak group size of more than 1,000 clients

32. Time zone diversity Many small streams reach more than half the world!
Almost all large streams reach more than half the world.

33. Client diversity Motivating questions
Where should streaming servers be placed in the network? Clients are tuning in from many different locations.
How should clients be mapped to servers? For small streams which happen to have a diverse set of clients, it may be too wasteful for a CDN to map every client to the nearest server.

34. Summary
Publishers are using the Internet to reach a wider audience than traditional radio and TV
Interesting observations
Lots of audio traffic
Lots of flash crowds (content-driven behavior)
Lots of one-timers
Lots of diversity amongst clients, even for small streams

35. Nice pictures…see paper for details
Percentage of requests
Quicktime
Real
Windows media

36. Abandon all hope, ye who enter here.

37. Talk outline Akamai live streaming workload
With an application end-point architecture
Is the overlay stable enough despite dynamic participation?
Is there enough upstream bandwidth?
Are overlay structures efficient?
Summary

38. When is a tree stable? X X X Not stable More stable
Departing hosts have no descendants
Stable nodes at the top of the tree
Stable nodes X X
Less stable
nodes X
Interruptions
Time
Ancestor leaves

39. Extreme group dynamics
15% stay longer than 30 minutes
(heavy-tailed)
45% stay less than 2 minutes!
Summarize into points

40. Stability evaluation: simulation
Hosts construct an overlay amongst themselves using a single-tree protocol
Skeleton protocol of the one presented in the ESM Usenix ’04 paper
Findings are applicable to many protocols
Goal: construct a stable tree
Parent selection is key
Group dynamics from Akamai traces (join/leave)
Honor upstream bandwidth constraints
Assign degree based on bandwidth estimation
Too much text;
2 slides; degree == num children; invert the text (2 key bullets)

41. Join
Join
IP1
IP2
...

42. Probe and select parent
IP2
IP1
IP2
...
IP1

43. Probe and select parent
Parent selection algorithms
Oracle: pick a parent who will leave after me
Random
Minimum depth (select one out of 100 random)
Longest-first (select one out of 100 random)

44. Parent leave X
Host leaves

45. Parent leave ?
Host leaves
All descendants are disconnected

46. Find new parent Host leaves All descendants are disconnected
All descendants probe to find new parents

47. Stability metrics Mean interval between ancestor change
Number of descendants of a departing host
Interruptions X
Get rid
Time
Ancestor leaves

48. Stability of largest stream
Longest-first
Random
Min depth
Explain 5 mins
Oracle: there is stability!

49. Is longest-first giving poor predictions?
Oracle, ~100% no descendants
Longest-first, 91%
Min depth, 82%
Random, 72%

50. Stability of 50 large-scale streams
Longest-first
between ancestor change < 5 minutes
Percentage of sessions with interval
Random
Interval less than 5 minutes; inherent stability: what does this mean
GAP between oracle vs min depth
Min depth
Oracle
There is stability! Of the practical algorithms,
min depth performs the best.

51. There is inherent stability
Given future knowledge, stable trees can be constructed
In many scenarios, practical algorithms can construct stable trees
Minimum depth is robust
Predicting stability (longest-first) is not always robust; when wrong, the penalty is severe
Mechanisms to cope with interrupts are useful
Multiple trees (see paper for details)
Too much text; remove last bullet. Leave multiple trees.
Delete this slide;

52. Talk outline Akamai live streaming workload
With an application end-point architecture
Is the overlay stable enough despite dynamic participation?
Is there enough upstream bandwidth?
Are overlay structures efficient?
Summary

53. Is there enough upstream bandwidth to support all hosts?
What if application
end-points are all DSL?
Video 300 kbps
DSL
DSL
DSL
Upstream bandwidth
only 128 kbps
Saturated tree
Emphasize: concern: infeasible; hightlight; slow down and explain

54. Metric: Resource index
Ratio of the supply to the demand of upstream bandwidth Resource index == 1 means the system is saturated
Resource index == 2 means the system can support two times the current members in the system
Resource Index:
(3+5)/3 = 2.7

55. Large-scale video streams
A few
streams
are in
trouble
or close.
1/3 of the streams are in trouble.
Most streams have sufficient upstream bandwidth.

56. Talk outline Akamai live streaming workload
With an application end-point architecture
Is the overlay stable enough despite dynamic participation?
Is there enough upstream bandwidth?
Are overlay structures efficient?
Summary

57. Relative Delay Penalty (RDP)
How well does the overlay structure match the underlying network topology?
RDP = Overlay distance
Direct unicast distance US US
20ms
50ms US
50ms US
50ms
Europe
Europe
Results are more promising than previous
studies using synthetic workloads and topologies.

58. Summary
Indications of the feasibility of application end-point architectures
The overlay can be stable despite dynamic participation
There often is enough upstream bandwidth
Overlay structures can be efficient
Our findings can be generalized to other protocols
Future work: Validate through real deployment
On-demand use of waypoints in End System Multicast
Attract large groups
Inherent resources! Upstream access bandwidth
Future: ESM phasing out waypoints=> already app end-point architecture ; attract large groups
Thank you!