1. Smart Content Delivery in Large Networks: En-Route Caching
Hong Shen
School of Computer Science
University of Adelaide, Australia
Dept. of Computer Sci. & Tech.
University of Sci. & Tech. of China

2. Outline of the Talk Problem formulation Unconstrained solution
Constrained solutions
Solution for m servers

3. Content Distribution Network
Sits between content providers and content consumers.
Contains hundreds of servers throughout Internet.
Replicates and maintains customers’ content in CDN servers.

4. CDN Example: Google platform
Maintains over 450,000 CDN servers, arranged in racks located in clusters in cities around the world
Allows users to access its content most rapidly by sending them lightly loaded and geographically proximate servers.

5. Bottleneck of CDNs Multiple transmission flows for the same object.
Solution: caching the object in selected nodes.
WHEN and HOW?
Challenges

6. En-Route Object Caching
Object caching: Store most commonly accessed objects close to clients
En-route object caching: Objects are cached at selective nodes on the access path from client to server
Server
Hold no copy
Hold a copy
request
object

7. En-Route Object Caching (cont.)
Why en-route?
Important observation:
Users normally have regular access patterns;
Storing object at en-route nodes during delivery does not consume extra bandwidth.

8. Caching Performance
The performance of en-route object caching depends mainly on two factors:
The locations of the caches (Cache Location)
The management of the cache contents (Content Replacement)
Coordinated Caching: Consider both factors when making cache decision.

9. Our Work Web object en-route caching in tree networks
ACM Transactions on Internet Technology, Vol. 5, No. 3, 2005, p  
Multimedia object en-route caching in tree networks
ACM Transactions on Multimedia Computing, Communications and Applications, Vol. 1, No. 3, 2005, p
Multimedia object placement for transparent data replication in linear array
IEEE Transactions on Parallel & Distributed Systems, Vol. 18 , No. 2, 2007, p
Multiserver en-route web caching
IEEE Transactions on Computers (under review), 2007.

10. Definitions and Notations
G=(V,E) is a graph, where V is the set of nodes and E is the set of links.
Cost saving s(v): the cost saving of storing a new object in node (cache) v.
Cost loss l(v): the cost loss of removing other objects from node v in order to accommodate the new object.
Cost gain g(v): g(v)=s(v) –l(v).

11. Problem Formulation G(P)=
Find a node set P to store the object s.t. the total cost gain is maximized:
G(P)=

12. Problem Formulation for Tree NetworksG Aw w
D(w)
Server v’
Hold no copy
Hold a copy v
f(v)
f’(v)

13. Constraints The different cases of C include:
C is null (unconstrained).
The cost gain for each node is greater than zero, i.e., g(v)>0 for all v in P.
The number of copies is exactly k, i.e., |Aw|=k.
The number of copies is no more than k, i.e. |Aw| k.

14. Solution for Unconstrained Case
Main idea:
Decompose the tree level by level recursively to a set of lines or singletons (nodes) whose solutions are known. Solution (Aw) to tree Tw is obtained by combining (union of) the solutions (Aw,x) to Tw’s subtrees.

15. Tree Decomposition (1)
C(w): set of all children of node w.

16.  Decomposition of A * w Aw,w2 Aw,w1 * w Aw w * w1 w2 w2 w1
Theorem 1 shows that the optimal solution for the whole tree can be divided into the optimal solutions for some subtrees according to the children of the root. For this example, since the root has two children, the optimal solution for the whole tree can be obtained by the optimal solutions for the two subtrees on the left.

17. Tree Decomposition (2)

18.  Decomposition of A  G( ) G( ) A A A T * w, x Theo. 2 1 2 Theo.1 ,, w, A x *
,,1 w, A ,, T ,
G( )
,,2 
G( )
{x}  1 x x x 1 2
2. otherwise 2 1.

19. Algorithm 1

20. Algorithm 1: Continued

21. Time Complexity The algorithm runs in time:
tw= O( vC(w) ( C(v)+tv) )
= O(vVD(v))
= O(n2),
where n is the total number of nodes in the network.

22. Solution for Constrained Case I
Non-negative cost gain per node
(1)

23. Transformation The optimal solution for Problem (1) is equivalent to
(2)

24. Algorithm 2

25. Algorithm 2 (Continued)
Time Complexity:
O(n2)

26. Solution for Constrained Case II
Placing exactly k copies
(3)

27. Algorithm 3
Time Complexity:
O(n2log(fn)), where f=max{f(v)}.

28. Solution for Constrained Case III
Placing at most k copies
(4)

29. Algorithm 4
Time Complexity:
O(kn2log(fn)), where f=max{f(v)}.

30. Extension to ASes
System Model:

31. Solution
Dividing the whole system into two parts and one part is a tree.
Continuing to divide the other part in the same way until there is only one tree left.
Applying the methods for tree network.

32. More General Setting: m-Sever En-route Caching
A set of servers S={sj, 1≤ j ≤m} located at leaves of a tree.
Cost saving for node w, s(w, dj), under the condition that the distances from w to the nearest high level node towards server sj that holds a copy is dj.
Find a node set P to store the object, s.t. the total gain is maximized (vP serves nodes g(v,S)) 32

33. The Challenge
We can’t get optimal solution to multi-server problem by simply combining solutions to 1-server problem. +
A Simple 2-Server Problem
Solve 1-server problem
Optimal
Solution ≠
Hold a copy No copy 33

34. A More General Definition
Condition Dw, Dw=[d1,…dj,…dm], dj is the distance from node w to the nearest node towards sj, for example u, that hold a copy of object O.
G(w, Dw), is the objective value of (6) in Tw under condition Dw,
A(w, Dw) is the solution corresponding to G(w, Dw). 34

35. Lemma 1
For tree Tr containing m servers at leave nodes, the distances from wi to the nearest node towards sj that holds a copy are denoted by e(wi,dj) and k(wi,dj) for the cases that node wi holds a copy and no copy respectively, then we have r s1 s2 s3
wi ∈ path[r, sj] means server sj is in the sub-tree twi, because servers are located at leaves.
An example of multi-server network 35

36. Theorem 3
For tree Tr containing m servers at leave nodes, the optimal solution of (6) is A(r, Dr) and corresponding objective value is G(r, Dr), where Dr is the vector of distances from root node to servers and 36

37. Theorem 3 (cont.)

38. Time complexity of the algorithm is O(nhm).
Main idea:
Problem is split top-down and solution A(r, Dr) is generated bottom-up according to Theorem 3, with corresponding objective value G(r, Dr).
Time complexity:
Algorithm computes all G(w, Dw), where w∈ V, Dw = [d1,…dj,…dm], 0 ≤ dj ≤ hw, hw is the distance from w to sj, hw ≤ 2h.
Time complexity of the algorithm is O(nhm). 38

39. Conclusion New tree decomposition techniques for en-route web caching.
Polynomial-time algorithms for the first time for 1-server en-route web-caching in tree networks.
p-server en-route web caching in tree networks: O(nhm ) time.

40. Questions?

41. Calculating cost loss l(v)
Cost loss l(v): The additional cost caused by removing some objects from v to make room for the new object:
Holding no copy
Server
Missing penalty m(v): The additional cost of accessing the object if it is not cached at v E.g. m(3)=c(3,0), m(7)=c(7,4).
Holding a copy 2 1
c(3,0) 5
f’(3)=0
f’(4)=f(6)
f’(5)=f(8)
+f(9) 3 4
c(9,5) 6 7 8 9