Optimal Content Delivery with Network Coding Derek Leong, Tracey Ho California Institute of Technology Rebecca Cathey BAE Systems CISS 2009 March 19, 2009

Motivation

S

S t t t t t t

S t t t t t t

S t t t t t t

S t t t t t t

S t t t t t t How to pick storage nodes? How to route dissemination and fetch flows?

Review: Subgraph Selection

Resultant shared flow must support individual flows Flows must be realizable within capacity Flows through each node must be conserved

Review: Subgraph Selection for CDNs

Storage flow in node memory, occurring over time Network coding allowed Network coding NOT allowed

Review: Subgraph Selection for CDNs

Total dissemination cost Total storage costTotal fetch cost

Review: Subgraph Selection for CDNs Resultant shared flow must support individual flows Flows must be realizable within capacity Flows through each node must be conserved

Review: Subgraph Selection for CDNs

Modified Formulations k-hop Fetch Constraint: Restrict fetch flows to within the k-hop neighborhood of each receiver

Robust Storage Distributed storage can be used to improve robustness of data availability in unreliable networks Intuitively, in a network where nodes or arcs fail probabilistically, the probability of a receiver being able to successfully fetch data increases with the amount of redundant storage and the proximity of storage nodes

Robust Storage Goal: Ensure each receiver can still successfully access content in the event that some nodes or arcs fail during the fetch stage

Robust Storage Exact but prohibitively complex approach: – Consider all possible failure events = { (occurrence prob, set of nodes & arcs that fail)} – Replace each receiver with a set of virtual receivers, one for each failure event affecting it – Allow fetch flows only through the corresponding unaffected nodes & arcs for each virtual receiver – Modify objective function to include fetch cost incurred by each virtual receiver, weighted by their respective probabilities, so that it continues to express the expected total cost

Robust Storage k-hop fetch constraint reduces the number of virtual receivers required – Example: To protect against the failure of up to any m arcs in the network, the number of virtual receivers required per receiver is Without fetch constraint : With 1-hop fetch constraint : – However, the number of virtual receivers still grows exponentially with the number of hops k

Robust Storage To allow flexibility in the number of hops k in the fetch constraint while remaining tractable, we can specify the virtual receivers first, and then compute the resulting minimum fetch success probability This approach enables us to lower-bound the success probability in terms of – number of hops k – probability of node or arc failure p – number of disjoint paths in the k-hop neighborhood of a receiver d for example

Robust Storage Using the following (d+1) virtual receivers for t – v 0 corresponding to the zero-failure event – v i corresponding to the event where only the arcs on paths P j, j ≠ i, are allowed to carry fetch flows for v i We can achieve success probability ≥ Prob[at most one of the d paths fails] =

Other Modified Formulations Potential Storage Nodes: Restrict potential storage nodes to some subset by changing the capacity

Other Modified Formulations Storage Budget Constraint: Impose an aggregate storage budget over all nodes

Other Modified Formulations Fetch Load Constraint: Bound the expected load on nodes and arcs during the fetch stage

Other Modified Formulations Dissemination Stage Receivers: Add node in the dissemination stage to the set of receivers

Other Modified Formulations Delivery of Multiple Content: Introduce separate flows for individual content, allowing network coding during only the dissemination stage and among flows for the same content

Performance Evaluation Compare Network-Coded Formulation (NCF) vs Minimum k-median Formulation (KMF), for the Delivery of Multiple Content (“multiobject placement”) subject to a given storage budget constraint Major difference between NCF and KMF: KMF minimizes only the total fetch cost, but NCF minimizes total fetch + dissemination cost Performance metric: NCF expected total dissemination + fetch cost KMF expected total dissemination + fetch cost =,

Performance Evaluation Compare Network-Coded Formulation (NCF) vs Minimum k-median Formulation (KMF), for the Delivery of Multiple Content (“multiobject placement”) subject to a given storage budget constraint Major difference between NCF and KMF: KMF minimizes only the total fetch cost, but NCF minimizes total fetch + dissemination cost Performance metric: NCF expected total dissemination + fetch cost KMF expected total dissemination + fetch cost =, Unit {dissemination, fetch} cost Expected total number of requests by a receiver over all objects

Performance Evaluation = 0 (“free dissemination”) NCF KMF

Performance Evaluation = 0 (“free dissemination”) Number of objects Storage budget NCF KMF

Performance Evaluation = 0 (“free dissemination”) NCF KMF

Performance Evaluation Number of objects |W| = 1 NCF KMF

Performance Evaluation Number of objects |W| = 1 Storage budget NCF KMF

Performance Evaluation Number of objects |W| = 1 NCF KMF

Performance Evaluation Number of objects |W| = 3 NCF KMF

Performance Evaluation Number of objects |W| = 5 NCF KMF

Performance Evaluation Number of objects |W| = 7 NCF KMF

Implementation Considerations Using a domain name system (DNS) to direct origin servers and end users to the nearest available CDN node Using source routing for the dissemination stage, and a DNS to resolve requests during the fetch stage Opportunistically caching coded data flows during the fetch stage

Augmenting with a P2P Network A hybrid CDN–P2P network achieves better scalability since end users help contribute bandwidth, storage, and computation resources CDN provides a reliable backbone for content delivery, and prevents severe service degradation in the face of high churn rates Specifying dissemination stage receivers when adopting a distributed hash table (DHT) based storage and lookup mechanism

Conclusion & Future Work Presented a unified LP formulation for optimal content delivery in CDNs Simulation results suggest NCF performs significantly better, even under modest circumstances (small network, few objects, low storage budget, low dissemination costs) Look forward to addressing content delivery in dynamic environments (e.g. mobile ad hoc networks)

Optimal Content Delivery with Network Coding Derek Leong, Tracey Ho California Institute of Technology Rebecca Cathey BAE Systems CISS 2009 March 19, 2009