1. Delay-Based Network Utility Maximization Michael J. Neely University of Southern California http://www-rcf.usc.edu/~mjneely IEEE INFOCOM, San Diego, March 2010 *Sponsored in part by the DARPA IT-MANET Program, NSF Career CCF-0747525 a 1 (t) a 2 (t) a K (t) Utility Thruput x k S 1 (t) S 2 (t) S K (t)

2. Network Model: 1-Hop Network with K Queues --- (Q 1 (t), …, Q K (t)) Slotted time, t in {0, 1, 2, … } S(t) = (S 1 (t), …, S K (t)) = “Channel State Vector” (i.i.d. over slots) x(t) = (x 1 (t), …, x K (t)) = “Transmission Decision Vector” a(t) = (a 1 (t), …, a K (t)) = “Packet Arrival Vector” (i.i.d. over slots) Fixed Length Packets: a(t), x(t) are 0/1 vectors. Arrival Rates: E{a(t)} = (λ 1, …, λ K ). Reliability is a Function of Transmission Decision: Observe S(t) every slot. Choose 0/1 transmission vector x(t). Pr[Success on link k | x(t), S(t)] = Ψ k (x(t), S(t)) a k (t)Ψ k (x(t), S(t))

3. Packet Dropping: We can decide to drop packets at any time. Packets that fail in transmission can either be retransmitted or dropped. Thruput on channel k = y k = λ k – drop rate on channel k a k (t)Ψ k (x(t), S(t)) Delay-Based Control: Stamp Head-of-Line (HOL) Packets with their Delays H k (t). Utility Maximization Objective: Maximize: g 1 ( y 1 ) + g 2 ( y 2 ) + … + g K ( y K ) where g k (y) are concave, non-decreasing utility functions Delay = 4 Delay = 4 Delay = 2 Delay = 2 Delay = 1 Delay = 1 Drops k (t) Queue k: H k (t) = 4

4. Prior work on Stochastic Network Optimization: Stability: (“Max-Weight” = “Min-Lyapunov-Drift”) Tassiulas, Ephremides [1992, 1993] (Minimize drift Δ(t)) Kahale, Wright [1997] Andrews et. al. [2001] Neely, Modiano, Rohrs [2003, 2005] Kobayashi, Caire, Gesbert [2005] Joint Stability and Utility Optimization: Neely, Modiano [2003, 2005] (Minimize Δ(t) + V*Penalty(t)) Georgiadis, Neely, Tassiulas [2006] Stolyar [2005] (Primal-Dual “Fluid Model” analysis) Special case of “Infinitely Backlogged Sources”: Agrawal, Subramanian [2002], Kushner, Whiting [2002] Eryilmaz, Srikant [2005], Lin, Shroff [2004] All of these references use queue backlog as weights!

5. Alternative “Delay-Based” Rules that use HOL Delays as weights are known only for Network Stability: Mekkittikul, McKeown [1996] Shakkottai, Stolyar [2002] Andrews, Kumaran, Ramanan, Stolyar, Vijaykumar, Whiting [2004] Our work fills the gap by developing a Delay-Based Rule for Maximizing Network Utility Subject to Stability. a k (t) Ψ k (x(t), S(t)) Delay = 4 Delay = 4 Delay = 2 Delay = 2 Delay = 1 Delay = 1 Drops k (t) Utility Thruput x k

6. Challenges: Prior “Drift-Plus-Penalty” Algorithm Admit/Drops Packets Immediately when New Packets Arrive: Δ(t) + V*Penalty(t). This does not directly Affect the HOL Values! Tricky Correlation Issues between HOL sizes and Decisions! Key Ideas for this paper: Queue all packets. Admit/Drop only at HOL. Use “Drift-Plus-Penalty” with a different queue structure Use a “concavely extended utility function” Advantages of Delay-Based Approach: Provides “Delay-Fairness.” Queue-Based Rules can leave loner packets stranded. Provides Worst Case Delay Guarantees. Disadvantage: Must know (λ 1, …, λ K ) to implement.

7. Concavely Extending a Utility Function: 0 1 ykyk g k (y k ) 0 1 ykyk f k (y k ) slope = η k slope = η k Function g k (y k ): Defined over 0 ≤ y k ≤ 1 Function f k (y k ): Defined over -1 ≤ y k ≤ 1

8. Transformed Stochastic Net Optimization Problem: Maximize: f 1 ( θ 1 ) + f 2 ( θ 2 ) + … + f K ( θ K ) Subject to: (1) E{Q k } < infinity, for all k in {1, …, K} (2) y k ≥ θ k, for all k in {1, …, K} (3) -1 ≤ θ k (t) ≤ 1, for all t (4) y k, θ k are time averages achievable on the network Auxiliary Variables and Thruput Variables: y k (t) = λ k – Drops k (t) Time Avg: y k = λ k – D k θ k (t), choose s.t. -1 ≤ θ k (t) ≤ 1 Time Avg: θ k

9. Transformed Stochastic Net Optimization Problem: Maximize: f 1 ( θ 1 ) + f 2 ( θ 2 ) + … + f K ( θ K ) Subject to: (1) E{Q k } < infinity, for all k in {1, …, K} (2) y k ≥ θ k, for all k in {1, …, K} (3) -1 ≤ θ k (t) ≤ 1, for all t (4) y k, θ k are time averages achievable on the network Auxiliary Variables and Thruput Variables: y k (t) = λ k – Drops k (t) Time Avg: y k = λ k – D k θ k (t), choose s.t. -1 ≤ θ k (t) ≤ 1 Time Avg: θ k

10. Transformed Stochastic Net Optimization Problem: Maximize: f 1 ( θ 1 ) + f 2 ( θ 2 ) + … + f K ( θ K ) Subject to: (1) E{Q k } < infinity, for all k in {1, …, K} (2) y k ≥ θ k, for all k in {1, …, K} (3) -1 ≤ θ k (t) ≤ 1, for all t (4) y k, θ k are time averages achievable on the network Virtual Queue Z k (t) for enforcing constraint (2): Z k (t) y k (t) = λ k – Drops k (t)θ k (t)

11. Head-of-Line Delay Update Equation (Queue k): H k (t+1) = 1 {k full} (t) max[H k (t) + 1 – (μ k (t)+D k (t))T k (t), 0] + 1 {k empty} (t) A k (t) Inter-Arrival Time T k (t) DropsService Lyapunov Function: L(t) = (1/2)[H 1 (t) 2 + … + H K (t) 2 ] + (1/2)[Z 1 (t) 2 + … + Z K (t) 2 ] Drift-Plus-Penalty Approach: Observe {H 1 (t), …, H K (t)}, {Z 1 (t),…,Z K (t)}, {S 1 (t), …, S K (t)} Take action to minimize: Δ(t) – V [f 1 (θ 1 (t)) +... + f K (θ K (t))]

12. Resulting Algorithm: 1)(Aux Variables) Each k observes Z k (t). Choose θ k (t) to: Maximize : V f(θ k (t)) – Z k (t)θ k (t) Subject to: -1 ≤ θ k (t) ≤ 1 2) (Transmission) Observe S(t), H(t), Z(t). Choose x(t) to: Maximize : x k (t) min[H k (t), Z k (t)] Ψ k (x(t),S(t)) Subject to: (x 1 (t),…, x K (t)) a 0/1 vector 3) (Packet Dropping) For each queue k with HOL packet that was not successfully transmitted, drop iff Z k (t)≤H k (t). 4) (Update Virtual and Actual Queues) Δ(t) – V [f 1 (θ 1 (t)) +... + f L (θ L (t))] ΣkΣk

13. Focus on Auxiliary Variable Update: Choose θ k (t) to: Maximize : V f(θ k (t)) – Z k (t)θ k (t) Subject to: -1 ≤ θ k (t) ≤ 1 Key Lemma: If Z k (t) > Vη k, then… (a)…the above chooses θ k (t) = -1 (b)…hence, Z k (t) cannot increase on that slot: Z k (t) y k (t) = λ k – Drops k (t)θ k (t) = -1 ≥ -1 [This is why we concavely extended the utility function over -1 ≤ y ≤ 1 ] 1 Z k (t) < Vη k

14. Focus on Auxiliary Variable Update: Choose θ k (t) to: Maximize : V f(θ k (t)) – Z k (t)θ k (t) Subject to: -1 ≤ θ k (t) ≤ 1 Key Lemma: If Z k (t) > Vη k, then… (a)…the above chooses θ k (t) = -1 (b)…hence, Z k (t) cannot increase on that slot: Z k (t) y k (t) = λ k – Drops k (t)θ k (t) = -1 ≥ -1 [This is why we concavely extended the utility function over -1 ≤ y ≤ 1 ] 1 Z k (t) > Vη k

15. Concluding Theorem: Implementing the above algorithm for any parameter V>0, we have… (a)Delay: Worst Case Delay in Queue k ≤ Vη k + 2 slots. (b)Total Utility Satisfies: g 1 ( y 1 ) + … + g K ( y K ) ≥ g 1 ( y 1 *) + … + g K ( y K *) – B/V Achieved UtilityOptimal Utility VV Utility Delay

16. Concluding Theorem: Implementing the above algorithm for any parameter V>0, we have… (a)Delay: Worst Case Delay in Queue k ≤ Vη k + 2 slots. (b)Total Utility Satisfies: g 1 ( y 1 ) + … + g K ( y K ) ≥ g 1 ( y 1 *) + … + g K ( y K *) – B/V Achieved UtilityOptimal Utility VV Utility Delay

17. Concluding Theorem: Implementing the above algorithm for any parameter V>0, we have… (a)Delay: Worst Case Delay in Queue k ≤ Vη k + 2 slots. (b)Total Utility Satisfies: g 1 ( y 1 ) + … + g K ( y K ) ≥ g 1 ( y 1 *) + … + g K ( y K *) – B/V Achieved UtilityOptimal Utility VV Utility Delay

18. Concluding Theorem: Implementing the above algorithm for any parameter V>0, we have… (a)Delay: Worst Case Delay in Queue k ≤ Vη k + 2 slots. (b)Total Utility Satisfies: g 1 ( y 1 ) + … + g K ( y K ) ≥ g 1 ( y 1 *) + … + g K ( y K *) – B/V Achieved UtilityOptimal Utility VV Utility Delay

19. Concluding Theorem: Implementing the above algorithm for any parameter V>0, we have… (a)Delay: Worst Case Delay in Queue k ≤ Vη k + 2 slots. (b)Total Utility Satisfies: g 1 ( y 1 ) + … + g K ( y K ) ≥ g 1 ( y 1 *) + … + g K ( y K *) – B/V Achieved UtilityOptimal Utility VV Utility Delay

20. Concluding Theorem: Implementing the above algorithm for any parameter V>0, we have… (a)Delay: Worst Case Delay in Queue k ≤ Vη k + 2 slots. (b)Total Utility Satisfies: g 1 ( y 1 ) + … + g K ( y K ) ≥ g 1 ( y 1 *) + … + g K ( y K *) – B/V Achieved UtilityOptimal Utility VV Utility Delay

21. Concluding Theorem: Implementing the above algorithm for any parameter V>0, we have… (a)Delay: Worst Case Delay in Queue k ≤ Vη k + 2 slots. (b)Total Utility Satisfies: g 1 ( y 1 ) + … + g K ( y K ) ≥ g 1 ( y 1 *) + … + g K ( y K *) – B/V Achieved UtilityOptimal Utility VV Utility Delay

22. Concluding Theorem: Implementing the above algorithm for any parameter V>0, we have… (a)Delay: Worst Case Delay in Queue k ≤ Vη k + 2 slots. (b)Total Utility Satisfies: g 1 ( y 1 ) + … + g K ( y K ) ≥ g 1 ( y 1 *) + … + g K ( y K *) – B/V Achieved UtilityOptimal Utility VV Utility Delay

23. Example Utility Function g k ( y ) = log( 1 + η k y ) This approximates “proportionally fair” log-utility when η k is large. The log-utility log( y ) has a singularity at y=0, and so is not always a good choice of utility function. VV Utility Delay