1. Proxy Caching for Streaming Media

2. You Are Here Network Encoder Sender Middlebox Receiver Decoder
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

3. Types of Caches Browser cache Proxy cache Gateway cache For one user
Shared cache between clients and server
Gateway cache
Content Delivery Networks (CDN)
“Scale” server
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

4. Gateway Caches Deployed (or hired) by web site owners
Makes sites more scalable and reliable
Content is pushed out to caching nodes around the world
Use DNS redirection to find closest cache
Commercial CDNs:
Akamai, Amazon CloudFront, …
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

5. Cache Proxies for Web A NUS.SOC.CS5248-2012
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

6. Hierarchical Caching B A NUS.SOC.CS5248-2012
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

7. Cooperative Caching A B NUS.SOC.CS5248-2012
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

8. Distributed Caching A B NUS.SOC.CS5248-2012
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

9. Streaming Media vs. Webpage
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

10. Video Access Pattern by S. Acharya and B. Smith in 1999
Study at Lulea University, Sweden
55% complete, 45% stop very early
High temporal locality
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

11. Prefix Access Distribution
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

12. Video Popularity Assume Zipf “Law”
Probability of access to i-th most popular video is
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

13. Benefits of Caching

14. Reduce Access Latency :) :( NUS.SOC.CS5248-2012
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

15. Reduce Server Load NUS.SOC.CS5248-2012
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

16. Reduce Start-up Latency
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

17. Hide Network Congestion
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

18. Other Issues What to cache? Who to fetch from?
When cache is full, what to kick out?
How to measure popularity?
Can cache adapt to popularity?
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

19. What to Cache?

20. Segmentation Cache “all or none” is bad
Divide media file into segments S and consider each segment individually
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

21. Effects of Segment Size S
Large S : Low utilization
Small S : Lots of gaps (fragmentation)
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

22. Prefix Caching Policy 1 Chunk = k segments NUS.SOC.CS5248-2012
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

23. Caching Policy Basic unit of caching: segment Cache prefix in chunk
Replace suffix in chunk
Never replace segments in currently accessed chunk
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

24. Where To Fetch From?

25. Cooperative Caching A B NUS.SOC.CS5248-2012
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

26. Fetch from Server Server B A Client 1 Client 2
server unicast to client 2 A
Client 1
Client 2
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

27. Fetch from Fellow Proxy
Server B
server unicast to client 2 A
Client 1
Client 2
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

28. Issues How to advertise? How to choose “helper”? NUS.SOC.CS5248-2012
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

29. How to Advertise? Balance between network load
freshness of information
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

30. Scalable Advertisement
Expanding Ring Advertisement
TTL PERIOD
16 1
32 2
64 4
128 8
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

31. How to Choose Helper? Consideration for Static Cache
network distance (1,2,3,4)
number of streams being served
avoid frequent switches
Build a cost function, integrating the metrics
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

32. Cost Function Cost for retrieving a segment from node X to node Y=
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

33. Algorithm Consider the next gap in local caches
Find the next helper with minimum cost, which can fill in at least k segments in gaps
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

34. Distributed Caching
Y. Chae et al.
JSAC 2002

35. Cooperative vs. Distributed
Cooperative caching caches independently
Distributed caching caches as a team
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

36. Cold Start Server B A new clip! NUS.SOC.CS5248-2012
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

37. Segment Map Local segment map Global segment map
Which segment should I cache?
Global segment map
Who is supposed to cache what?
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

38. Cache Hit Server B A NUS.SOC.CS5248-2012
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

39. Cache Miss Server B A NUS.SOC.CS5248-2012
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

40. Distributed Caching Server B A NUS.SOC.CS5248-2012
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

41. Problems Who should cache what? Which segment to kick out?
How to adapt segment distribution?
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

42. Who Should Cache What? RCache scheme
Segment video into equal size segments
A proxy will cache each segment with some probability
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

43. RCache Np proxies video of length Lv divide into Ns equal segments
Each proxy caches each segment with a/Np probability
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

44. Analysis Probability that whole video is cached is
Ns:: num of segments
Np: num of proxies
a/Np: prob of caching 1 segment
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

45. RCache’s Segmentation
Video is divided into segments of equal length
Can we do better?
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

46. Bimodal Distribution NUS.SOC.CS5248-2012
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

47. Silo segment size probability of storage NUS.SOC.CS5248-2012
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

48. Further Improvement segment size probability of storage
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

49. Problems Who should cache what? Which segment to kick out?
How to redistribute data?
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

50. Segment “Popularity” For each video i For each segment j
F(i,j) = Prob(i is accessed)*Prob(j is accessed)
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

51. Rainbow Algorithm less popular Segment Number NUS.SOC.CS5248-2012
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

52. Rainbow Algorithm Problem: Solution:
Many large video – too many segments
computationally expensive to sort
Solution:
Just approximate by quantizing popularity
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

53. Rainbow Algorithm less popular NUS.SOC.CS5248-2012
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

54. Problems Who should cache what? Which segment to kick out?
How to redistribute data?
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

55. Data Redistribution When popularity changes, need to redistribute.
Redistribute “on-demand” (lazy)
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

56. Cache Token Each segment have two bits (T,C)
T: I am suppose to have the segment
C: I have the segment
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

57. Data Redistribution (T=1,C=1) (T=0,C=0) (T=1,C=0) (T=0,C=1)
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

58. Example I J A B C D T=0,C=1 T=0,C=1 old new T=1,C=0 T=1,C=0 T=1,C=0
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

59. Example I J A B C D T=1,C=1 T=0,C=1 old new T=0,C=0 T=1,C=0 T=1,C=0
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

60. Example I J A B C D T=1,C=1 T=1,C=1 old new T=0,C=0 T=1,C=0 T=1,C=0
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

61. Example I J A B C D T=1,C=1 T=1,C=1 old new T=0,C=0 T=1,C=1 T=1,C=0
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

62. Quality Adaptive Caching
Reza Rajaie et al.
INFOCOM 2000

63. Objective Quality adaptive streaming
Use Scalable Video Coding (SVC)
How to integrate with proxy caching?
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

64. Scenario (10am) Server A Client 2 NUS.SOC.CS5248-2012
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

65. Scenario (1am) Server A Client 2 NUS.SOC.CS5248-2012
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

66. Problem: Solution: Caches interfere with congestion control algorithm
Make cache aware of quality adaptation
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

67. Solutions Making cache “quality-aware” Prefetch Replacement Algorithm
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

68. Cache Miss Server A Client 2 NUS.SOC.CS5248-2012
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

69. Cache Hit Server repair + prefetch A Client 2 NUS.SOC.CS5248-2012
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

70. Peeking Inside the Cache
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

71. Cache Hit: Repair NUS.SOC.CS5248-2012
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

72. Cache Hit: Prefetch NUS.SOC.CS5248-2012
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

73. Prefetch Algorithm playback point prefetch window NUS.SOC.CS5248-2012
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

74. Proxy Request to Server
Multiple requests (for different clients) are batched
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

75. Server Response Missing segments are sent in decreasing priority 3 4 21
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

76. Server Response
Send as many segments as possible until next prefetch request
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

77. Trade-offs How far in the future should we prefetch?
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

78. Solutions Making cache “quality-aware” Prefetch Replacement Algorithm
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

79. Goal of Replacement Goal: converge to efficient state
If a stream is popular
average quality is high
variation in quality is low
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

80. The Algorithm NUS.SOC.CS5248-2012
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

81. Thrashing and Locking NUS.SOC.CS5248-2012
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

82. Choosing Victim whit (weighted hit) = Tplay/Ttotal
Calculate whit for each layer in a stream over a popularity window
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

83. Example WHIT Lock? Name Layer 5.9 1 Nemo 4.3 4.0 Matrix 2 3.9 1.1
4.3
4.0
Matrix 2
3.9
1.1
Gigli
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

84. Static Cache Cache segments in proxy do not change over time
Can we exploit further properties of streaming media?
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

85. Rules of Thumb
Simple rules that should be considered in the design of caching architectures
Cache must be large enough to hold “working set”, otherwise thrashing will happen
Unified caches perform better than partitioned caches
NUS.SOC.CS
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)