1. Myopic and Optimal Distributed Routing Wei Chen and Sean P. Meyn ECE & CSL University of Illinois

2. MaxWeight: Issues Raised before and after 2007 Routing requires information. In the MaxWeight policy, this information is obtained through local queue length values. This can lead to irrational behavior when information is scarce Better performance through geometric insight and better information sharing Issues addressed by 12/07 Why does MW work? Answer led to h-MW policies introduced in CTCN – Performance evaluation and approximate optimality Improved robustness Specialization to routing Example: (and Leith, 2007 Subramanian, submitted). MaxWeight or Backpressure routing will send packets upstream! Issues addressed in 2008 Understanding dynamic multi- commodity flow problem, providing workload relaxation Distributed implementation in wireless setting based on message passing Simulation and analysis

3. Context of the reported work Upper bounding techniquesAssumptions Novel techniquesRealistic New application of old techniquesUnrealistic but robust Unconsummated union – applicable “layers” Application layer Overhead, feedback Transport layerIgnored Routing layerConsidered Scheduling/queuing layer Topologies Physical layerCanonical problems Dynamics Infinite / scaling law Arbitrary movementThrough separation of time scales Finite but arbitrary Random waypoint Other considerations No mobilityLow latency as well as optimal throughput Capacity-achieving techniques and architectures Proposed network coding application: Relaxations are required to reduce complexity Yes, new techniquesAdaptation No, old techniquesMobility a potential extension Work shows how important global information can be used if available. Generally, amount of global information required for approximate optimization is low

4. Performance impact of consensus algorithms, and other learning schemes used in this project Simulation study in realistic wireless setting Full performance analysis of multiple bottlenecks Integration with Network Coding projects What is the state of the art and what are its limitations?  MW routing inflexible with respect to performance improvement; requires global information (wireless); very little flexibility  New h-MW technique requires information, but is highly flexible, and approximately optimal in certain cases Optimizing MaxWeight for Routing KEY NEW INSIGHTS: Goal: develop insights of 07 Distributed implementation via message-passing New learning technique based on Oja’s algorithm Learning cuts/workload through stochastic approximation techniques Construction of h for models with multiple bottlenecks based on DP insights for w. relaxation Un-consummated union challenge: Integrate coding and resource allocation Generally, solutions to complex decision problems should offer insight Algorithms for dynamic routing: Visualization and Optimization END-OF-PHASE GOAL COMMUNITY CHALLENGE ACHIEVEMENT DESCRIPTION STATUS QUO NEW INSIGHTS MAIN RESULT:  Decentralized implementation, use of consensus algorithms  Simulation study for wireless model  Multiple cuts/bottlenecks addressed HOW IT WORKS: Step 1: Estimation of workload – generalization of network cuts Step 2: Estimation of congestion Step 3: Choice of h 0 - piecewise quadratic Special case: Single dominant destination gives h 0 quadratic function of workload, cost, and effective cost w.r.t. workload relaxation

5. Simulations for Multiple Traffic Streams Network approximately 100 nodes. Single destination, multiple sources MaxWeight compared to policy based on logarithmic perturbation of Simulation for high load: 50% improvement over greedy, 25% over MW Performance improves for functions h that more closely approximate DP solution GreedyApproximation of DP solution Source of performance loss in MW : Cycling back and forth across bottleneck network cut leads to higher workload values

6. PERFORMANCE Without attention to bottlenecks, a decentralized routing algorithm will create inefficiency through cycling. Stability has been established for logarithmic perturbation - results from simulation studies give optimism. Analysis is required. LEARNING Distributed learning of workload in a dynamic setting Distributed learning of control parameters Learning of channel characteristics Summaries and challenges Research bottlenecks: Learning dynamic bottleneck location and workload CHALLENGES: Implementation requires learning on several levels – from global architecture, to local policy parameters. CONCLUSIONS: In wireless model with multiple bottlenecks we again see 25% delay improvement in simulation. Logarithmic perturbation gives universally stabilizing policies, with no evident performance loss in experiments. Learning insights: 1. Message passing can lead to performance loss - analysis underway. 2. Stochastic approximation Oja’s algorithm completed. Application to network decomposition. 3. Local capacity learning investigated. SCIENTIFIC FOUNDATIONS Stochastic Lyapunov theory + workload relaxation to approximate DP solution

7. References S. P. Meyn. Stability and asymptotic optimality of generalized MaxWeight policies. To appear, SIAM J. Control Opt. (Preliminary version to appear at the 46th IEEE Conference on Decision and Control, December 2007). W. Chen and S. P. Meyn Optimizing MaxWeight For Routing. In preparation. S. P. Meyn. Control Techniques for Complex Networks. Cambridge University Press, 2007. Chinese edition to appear. S. P. Meyn and R. L. Tweedie, Markov Chains and Stochastic Stability. CUP Mathematical Library edition, to appear 2008. C. Lin, V. V. Veeravalli, and S. P. Meyn. Distributed beamforming with feedback: Convergence analysis. Submitted to the IEEE Transactions on Information Theory. http://arxiv.org/abs/0806.3023, 2008 (adaptation at nodal level) V. Borkar and S. Meyn. Oja’s algorithm for graph clustering and Markov spectral decomposition. In Proceedings of VALUETOOLS, Athens, Greece, 2008 (adaptation at network level) References in on-going research: CTCN and … Iterative Scheduling Algorithms, M. Bayati, B. Prabhakar, D. Shah and M. Sharma, Proceedings of IEEE Infocom 2007 Distributed Subgradient Methods for Multi-agent Optimization Angelia Nedic and Asuman Ozdaglar. To appear in IEEE Transactions on Automatic Control. Polynomial Complexity Algorithms for Full Utilization of Multi-hop Wireless Networks Atilla Eryilmaz, Asuman Ozdaglar and Eytan Modiano. INFOCOM 2007.

8. Decentralized implementation, use of consensus algorithms Wireless models: Apply D. Shah’s insights on maxproduct convergence Full analysis of multiple bottlenecks Integration with Network Coding projects: Can we code around network hot- spots? What is the state of the art and what are its limitations? MW routing inflexible with respect to performance improvement MW corresponds to h-myopic, with h quadratic. Key geometric property of quadratic identified by Meyn prior to July meeting. Optimizing MaxWeight: From 12/07 Meeting KEY NEW INSIGHTS: New perturbation technique: Application to routing & refinements for decentralization Heavy traffic optimality Taylor series approximation gives interpretation as adaptive MaxWeight - Diagonal matrix adapts to varying congestion Un-consummated union challenge: Integrate coding and resource allocation Generally, solutions to complex decision problems should offer insight Algorithms for dynamic routing: Visualization and Optimization END-OF-PHASE GOAL COMMUNITY CHALLENGE ACHIEVEMENT DESCRIPTION STATUS QUO NEW INSIGHTS MAIN RESULT: h-myopic policy is universally stabilizing Application to policy synthesis for approximately optimal performance (delay or backlog) in heavy traffic, with log regret HOW IT WORKS: Step 1: Estimation of network cuts Step 2: Estimation of congestion on either side Step 3: Choice of h 0 - piecewise quadratic Special case: Single dominant destination gives h 0 quadratic function of workload, cost, and effective cost w.r.t. workload relaxation Decentralized implementation appears feasible. Numerical study underway Investigate performance and feasibility: 100 nodes, multiple arrivals. Only wireline models investigated to-date. Excellent performace as predicted by theory