1. CMSC 34702 ML for End-to-End Adaptation Congestion Control (2)
Junchen Jiang
October 22, 2019

2. Remy review
RemyCC:
Remy-generated
Congestion control
Remy

3. The effect of prior knowledge
-1 RemyCC 10x
RemyCC exact
log(normalized throughput) - log(delay) -2 -3
Why not learning the CC strategies in the wild?
Cubic- over- sfqCoDel -4 -5 -6
4.74 15
link speed (megabits/sec)
47.4

4. Takeaways from last lecture (Remy)
TCP & CC is an important problem
CC is used in all sorts of environments  hard to precisely define goals
Traditionally relying on hand-craft heuristics/assumptions
You can replace “CC” with anything
Cache policy, routing, database mngt, deep learning arch search, catching bugs, …

5. Another subtle yet important note
not always a good idea to go “ML” directly
you can’t find the words ML/machine-learning/training/etc in the paper!
Instead, focus on what’s missing in the literature
CC has traditionally been optimized without clear specification
But Remy did pave the road for blackbox/ML solutions

6. PCC Vivace: Online-Learning Congestion Control A Deep Reinforcement Learning Perspective on Internet Congestion Control

7. PCC Vivace: Online-Learning Congestion Control
Slides from Tong Meng’s NSDI Talk

8. An alternative perspective on CC
100Mbps
100Mbps
100Mbps Li
10Mbps
100Mbps C
100Mbps
100Mbps

9. An alternative perspective on CC
Online learning
(PCC)
Utility
f(tpt, loss, latency, …)
Blackbox Internet
Next send rate
Acks
Send rate r

10. PCC online learning problem

11. Key design choices of PCC
PCC = Utility function + Rate control algorithm
PCC Allegro [NSDI’15]
Loss-based utility function
Fixed rate change step size + or
No latency-aware
Can cause bufferbloat
Difficult to converge
Slow reaction to network changes

12. Key design choices of PCC
PCC = Utility function + Rate control algorithm
PCC Allegro [NSDI’15]
Loss-based utility function
Fixed rate change step size +
PCC Vivace [NSDI’18]
Strictly concave utility function
Gradient-ascent rate control +

13. PCC rapid reaction to network changes
10-100Mbps, ms RTT, 0-1% random loss

14. PCC rapid reaction to network changes
10-100Mbps, ms RTT, 0-1% random loss

15. A Deep Reinforcement Learning Perspective on Internet Congestion Control
Presented by Matt Baughman

17. Final remarks: Remy vs. PCC

18. Two fundamentally different approaches
Remy =
Global optimization +
Whitebox network model
PCC =
Local optimization +
Blackbox network model

19. The argument (w.r.t blackbox vs whitebox) goes like this…
(PCC)
Whitebox
(Remy)
You cannot accurately model the Internet
Instead, seek empirical good mappings from past experience
There’s no free lunch!
Every scheme embodies preconceptions. PCC’s constants and behaviors embody assumptions too (e.g., loss < 5%)
Hypothesis: Blackbox works in computer vision, it will work in CC too!
In computer vision, giving up on basing decisions on accurate explanatory models has led to better solutions.
Hypothesis: No, it won’t!