1. 101 111 Using Feedback in MANETs: a Control Perspective Todd P. Coleman colemant@illinois.edu University of Illinois DARPA ITMANET TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A AAAA A A A

2. Current Uses of Feedback Theory Feedback modeled noiseless Point-to-point: capacity unchanged Significantly improved error exponents Reduction in complexity MANETs: Enlargement of capacity region

3. Current Uses of Feedback Practice Feedback is noisy, used primarily for Robustness to channel uncertainty Estimation of channel parameters ARQ-style communication w/ erasures

4. Current Uses of Feedback Practice Feedback is noisy, used primarily for Robustness to channel uncertainty Estimation of channel parameters ARQ-style communication w/ erasures But: Burnashev-style “forward error correction+ARQ” schemes are extremely fragile w/ noisy feedback (Kim, Lapidoth, Weissman 07)

5. Instantiate network feedback control algorithms for MANETs Develop iterative practical schemes for noisy feedback? Coding w/ feedback over statistically unknown channels? Develop fundamental limits of error exponents with feedback w/ fixed block length Applicability of Feedback in MANETs 101 111

6. 00.250.500.751.00 00011011 01 Communication w/ Noiseless Feedback

7. 00.250.500.751.00 00011011 01 Communication w/ Noiseless Feedback Given an encoder’s Tx strategy, decoding is almost trivial (Baye’s rule)

8. 00.250.500.751.00 00011011 01 Communication w/ Noiseless Feedback Given an encoder’s Tx strategy, decoding is almost trivial (Baye’s rule) How do we select a (recursive) encoder strategy for an arbitrary memoryless channel?

9. A Control Interpretation of the Dynamics of the Posterior Coleman ’09: “A Stochastic Control Approach to ‘Posterior Matching’-style Feedback Communication Schemes”

10. A Control Interpretation of the Dynamics of the Posterior Coleman ’09: “A Stochastic Control Approach to ‘Posterior Matching’-style Feedback Communication Schemes”

11. F k-1 Controller Z -1 P(F k |F k-1, u k ) ukuk FkFk reference signal Fw*Fw* A Control Interpretation of the Dynamics of the Posterior Coleman ’09: “A Stochastic Control Viewpoint on ‘Posterior Matching’-style Feedback Communication Schemes”

12. XkXk FkFk F k+1 Fw*Fw* D(F w * ||F k+1 ) D(F w * ||F k ) Reward at any stage k is the reduction in “distance” to target Stochastic Control: Reward Coleman ’09

13. Maximum Long-Term Average Reward Coleman ’09

14. Maximum Long-Term Average Reward (1),(2) hold w/ equality if: a) Y’s all independent b) Each X i drawn according to P*(x) Coleman ’09

15. Maximum Long-Term Average Reward (1),(2) hold w/ equality if: a) Y’s all independent b) Each X i drawn according to P*(x) Horstein ’63 (BSC) Schalwijk-Kailath ’66 (AWGN) Shayevitz-Feder ‘07, ‘08 (DMC) Coleman ’09

16. The Posterior Matching Scheme: an Optimal Solution Next input indep of everything decoder has seen so far, with capacity-achieving marginal distribution No forward error correction. Adapt on the fly. Coleman ’09 Posterior matching scheme

17. The Posterior Matching Scheme: an Optimal Solution Coleman ’09 Next input indep of everything decoder has seen so far, with capacity-achieving marginal distribution No forward error correction. Adapt on the fly. Posterior matching scheme

18. 01 01 01 Implications for Demonstrating Achievable Rates 01 1 Coleman ’09

19. Lyapunov Function 01 01 Posterior matching scheme: Coleman ’09

20. Lyapunov Function (cont’d) 01 01 01 1 Coleman ’09

21. Control Theory Information Theory Symbiotic Relationship Converse Thms Give Upper Bounds on Average Long-Term Rewards for Stochastic Control Problem Coleman ’09: “A Stochastic Control Viewpoint on ‘Posterior Matching’-style Feedback Communication Schemes”

22. Control Theory Information Theory Symbiotic Relationship Converse Thms Give Upper Bounds on Average Long-Term Rewards for Stochastic Control Problem KL Divergence Lyapunov functions guarantee all rates achievable Coleman ’09: “A Stochastic Control Viewpoint on ‘Posterior Matching’-style Feedback Communication Schemes”

23. Research Results with This Methodology Interpret feedback communication encoder design as stochastic control of posterior towards certainty Converse theorems specify fundamental performance bounds on a stochastic control problem related to controlling posterior. An optimal policy implies the existence of a Lyapunov function, which is in essence a KL divergence Lyapunov function directly implies achievability for all R < C Coleman ’09

24. Research Results with This Methodology Gorantla and Coleman ‘09: Encoders that achieve El Gamal 78: “Physically degraded broadcast channels w/ feedback“ capacity region in an iterative fashion w/ low complexity Interpret feedback communication encoder design as stochastic control of posterior towards certainty Converse theorems specify fundamental performance bounds on a stochastic control problem related to controlling posterior. An optimal policy implies the existence of a Lyapunov function, which is in essence a KL divergence Lyapunov function directly implies achievability for all R < C Coleman ’09

25. New Important Directions this Approach Enables Control Theory Information Theory Develop iterative low-complexity encoders/decoders for noisy feedback? Partially Observed Markov Decision Process 101 111

26. New Important Directions this Approach Enables Control Theory Information Theory Develop iterative low-complexity encoders/decoders for noisy feedback? Partially Observed Markov Decision Process Optimal coding w/ feedback over statistically unknown channels? Reinforcement learning from control literature 101 111

27. New Important Directions this Approach Enables Control Theory Information Theory Develop iterative low-complexity encoders/decoders for noisy feedback? Partially Observed Markov Decision Process Optimal coding w/ feedback over statistically unknown channels? Reinforcement learning from control literature Develop fundamental limits of error exponents with feedback w/ fixed block length Lyapunov function enables a fundamental Martingale condition 101 111

28. New Important Directions this Approach Enables Control Theory Information Theory Develop iterative low-complexity encoders/decoders for noisy feedback? Partially Observed Markov Decision Process Optimal coding w/ feedback over statistically unknown channels? Reinforcement learning from control literature Develop fundamental limits of error exponents with feedback w/ fixed block length Lyapunov function enables a fundamental Martingale condition Also: stochastic control approach provides a rubric to check tightness of converses via structure of optimal solution 101 111