1. SCREAM: Sketch Resource Allocation for Software-defined Measurement
(CoNEXT’15)
I will show you how to stuff many measurement tasks with high accuracy on switches with limited resources.
Masoud is on job market
Masoud Moshref, Minlan Yu,
Ramesh Govindan, Amin Vahdat

2. Measurement is Crucial for Network Management
Network Management on multiple tenants:
Traffic Engineering
Anomaly Detection
Accounting
Traffic Engineering
DDoS detection
Anomaly Detection
Need fine-grained visibility of how networks work
Multiple tenants, many mgm goals, these mgm goals are achieved with these measurement tasks
Anomaly detection: to find performance bugs
Traffic engineering algorithms need to know big flows
For DDoS detection we need to know if a set of IPs send too much traffic
An anomaly is to find hosts that communicate with too many destinations and can be detected using a super source detection task.
Measurement:
Heavy Hitter detection
Heavy hitter detection (HH)
Heavy Hitter detection
Hierarchical heavy hitter detection (HHH)
Change detection
Super source detection (SSD)

3. Software Defined Measurement
Controller
DREAM [SIGCOMM’14] / SCREAM [CoNEXT’15]
Task 1
Task 2
Configure
Emphasize on the workflow (Tasks configure switches, switches measure and send counters to the controller, tasks make report to the user)
Collect
Switch A
Switch B
Task 1 counters
Task 1 counters
Task 2 counters
Task 2 counters

4. Our Focus: Sketch-based Measurement
Summaries of streaming data to approximately answer specific queries
Ex: Bitmap for counting unique items
OpenFlow
Counters
Sketches
DREAM
[SIGCOMM’14]
SCREAM [CoNEXT’15]
Memory
Expensive,
Cheaper SRAM
power-hungry TCAM
Don’t say it in a way that sketches are our contribution
Sketches: Memory efficient, streaming, approximate, specific query
All traffic all-the-time  sketches are more accurate for variable traffic and support more tasks like flow size distribution
Counters
Volume counters
Volume and
Connection counters
Flows
Selected prefixes
All traffic all-the-time
Sketches use a cheaper memory and are more expressive

5. Sketch Example: Count-Min Sketch
At packet arrival: w
h1(IP)
2+1=3
h2(IP)
(IP, 1 Kbytes)
4+1=5 d
h3(IP)
1+1=2
At query:
What is the traffic size of IP? = row with min collision = Min(3,5,2) = 2
Explain what Count-Min means
At query we pick minimum to pick the row that had minimum HASH COLLISIONS
Highlight the traffic dependence early: Given a sketch of a specific size, its error depends on traffic properties such as total traffic size. (emphasize here)
Resource accuracy trade-off:
Provable error bound given traffic properties

6. Challenges: Limited Counters for Many Tasks
Limited shared resources:
SRAM capacity (e.g., 128 MB)
Shared with other functions (e.g., routing)
Too many resources to guarantee accuracy:
1 MB-32 MB per task
Less than tasks in SRAM
Many task instances:
3 types (Heavy hitter, Hierarchical heavy hitter, Super source)
Different flow aggregates (Rack, App, Src/Dst/Port)
1000s of tenants

7. Goal: Many Accurate Sketch-based Measurements
Users dynamically instantiate a variety of measurement tasks
SCREAM supports the largest number of measurement tasks while maintaining measurement accuracy
At high level, our contribution is to enable flexible measurements in networks where users can dynamically instantiate a number of complex measurements into the network state.
Our system, SCREAM, accommodates the largest number of measurements while maintaining accuracy, by leveraging tradeoffs between aggregate switch resource consumption and measurement accuracy.

8. Approach: Dynamic Resource Allocation
Resource accuracy trade-off depends on traffic
Count Min: Provable error bound given traffic properties
Ex: Skew of traffic from each IP
Worst-case uses >10x counters than average
Required memory
Skew
Dynamic allocation for current traffic

9. Opportunity: Temporal Multiplexing
Memory requirement varies over time
Task 1
Task 2
Required Memory
This gives us the opportunity of temporal multiplexing to support more tasks.
Time
Multiplex memory among tasks over time

10. Opportunity: Spatial Multiplexing
Memory requirement varies across switches
Task 1
Task 2
Required Memory
It also gives us the opportunity of spatial multiplexing to support more tasks.
Switch A
Switch B
Multiplex memory among tasks across switches

11. Key Insight Leverage spatial and temporal multiplexing
and dynamically allocate switch memory per task
to achieve sufficient accuracy for many tasks
DREAM has the same insight
SCREAM applies it for sketches

12. SCREAM Contributions
1- Allocate memory among sketch-based task instances
across switches while maintaining sufficient accuracy
SCREAM
Dynamic resource allocator
Allocation
Heavy hitter
(HH) tasks
Hierarchical heavy hitter
(HHH) tasks
Super Source
(SSD) tasks
2- Supports 3 sketch-based task types
Anomaly detection
Traffic engineering
DDoS detection

13. SCREAM Iterative Workflow
Collect & report
Counters & output
Estimate accuracy
Accuracy
Allocate resources
Memory size

14. SCREAM Iterative Workflow
Collect & report
Merge counters from switches
Accuracy
Estimate accuracy
Task1 accuracy <80%
Allocate resources
Give more memory to task1

15. SCREAM Iterative Workflow
Collect & report
Merge counters from switches
Accuracy
Estimate accuracy
Skew of traffic for task2 changes
Task2 accuracy <80%
Allocate resources
Give more memory to task2

16. SCREAM Challenges Collect & report
Network-wide task implementation using sketches
Estimate accuracy
Accuracy estimation without the ground-truth
Allocate resources
Fast & Stable allocation in DREAM [SIGCOMM’14]

17. Challenge: Merge Sketches of Different Sizes
Network-wide Task
Heavy hitter (HH)
Source IPs sending > 10Mbps
1- Tasks can have traffic from multiple switches
2- The sketches on these switches may have different sizes
3- However, previous work can only merge sketches of the same size
Let me give an example, Consider heavy hitter detection task that finds source Ips that send > 10Mbps.
It has traffic from two switches A and B and it wants to find the size of a flow that sends 10 Mbps on one and 15 on another
It uses count-min sketch. When we change the memory to its sketch on a switch, we change the number of counters per row (w),.
Previous work just add two arrays but it is impossible if the arrays have different sizes. 25 10 15
Switch A
Switch B d d w1 w2

18. SCREAM Solution to Merge Sketches for HH Detection
Previous work: Min of sums
SCREAM: Sum of mins 10 40 30 50 70 20 50 80 90 + 10 40 30 50 70 20 ≥ 10 20
Min
Min + 50 30
Previous work can only merge sketches of the same size.
A natural extension for sketches of different sizes is to find the corresponding counter for each prefix at each row and sum the counters at similar rows across sketches.
We call this approach min of sums.
Here, I show the counters in each row with different colors. Taking the minimum of sums results in 50.
Another approach is to get the approximation from each sketch and add them together.
We call this sum of mins and results in 30 in this example
Sum of mins is always smaller than min of sums.
Because count-min always over-approximates, smaller is more accurate thus SCREAM uses sum of mins. 25 10 15
Switch A
Switch B 10 30 70 40 50 20
Both over-approximate  smaller is more accurate

19. SCREAM Solutions Collect & report
Network-wide task implementation using sketches
Merge sketches of different sizes for HH, HHH, SSD
SSD algorithm with higher and more stable accuracy
Estimate accuracy
Accuracy estimation without the ground-truth
Allocate resources
Fast & Stable allocation in DREAM [SIGCOMM’14]

20. Precision Estimation for Heavy Hitter Detection
True detected HH
Detected HHs
Precision =
= Sum(P[Detected HH is true])
Insight: Relate probability to Error on counters of detected HHs
Threshold
True HH
False HH
Estimate-Threshold
Error
Estimate-Threshold
Thus to estimate the probability that a detected HH is a true HH we need to find the probability that the error is larger than the difference between estimated value and threshold
Next I describe how to find this probabilty
Estimated
Real
P[Detected HH is true]
= 1 - P[Error ≥ Estimate-Threshold]

21. Precision Estimation Step 1: Find a Bound on The Error
Insight: Relate probability to Error on counters of detected HHs
Idea: Use average Error in Markov’s inequality to bound it
This is the strawman solution.
We know the average error on each counter of count-min. Thus using Markov’s inequality, we find a bound on the probability that error goes above estimation minus threshold
Step 1
P[Detected HH is true]
= 1 - P[Error ≥ Estimate-Threshold]

22. Precision Estimation Step 2: Improve The Bound
A row in Count-Min:
Step 2
We know counter indices for heavy items so we can find their collisions. Thus we don’t need to rely on Markov’s inequality to find their errors.
Step 1
Insight:
Average Error = heavy items collision + small items collision
Counter indices of detected HHs show heavy collisions
Idea: Markov’s inequality only for small items

23. SCREAM Solutions Collect & report
Network-wide task implementation using sketches
Merge sketches of different sizes for HH, HHH, SSD
SSD algorithm with higher and more stable accuracy
Estimate accuracy
Accuracy estimation without the ground-truth
Precision estimators for HH, HHH and SSD tasks
Allocate resources
Fast & Stable allocation in DREAM [SIGCOMM’14]

24. Evaluation
Metrics:
Satisfaction of a task: Fraction of task’s lifetime with sufficient accuracy
% of rejected tasks
OpenSketch allocates for bounded relative error based on the worst-case traffic. We test it for different error bounds.
Alternatives:
OpenSketch: Allocate for bounded error for worst-case traffic at task instantiation (test with different bounds)
Oracle: Knows required resource for a task in each switch in advance

25. Evaluation Setting Simulation for 8 switches:
256 task instances (HH, HHH, SSD, combination)
Accuracy bound = 80%
5 min tasks arriving in 20 minutes
2 hours CAIDA trace
We tested for each type of tasks and combination of them

26. SCREAM Provides High Accuracy for More Tasks
SCREAM: High satisfaction and low reject
We tested open sketch with 10%, 50% and 90% relative error bounds
OpenSketch:
Loose bound  Under provision  low satisfaction
Tight bound  Over provision  high reject

27. SCREAM’s Performance Is Close to An Oracle
SCREAM satisfaction is lower because:
Iterative allocation takes time
Accuracy estimation has error

28. Other Evaluations Changing traffic skew
SCREAM supports more accurate tasks than OpenSketch
Accuracy estimation error
SCREAM accuracy estimation has 5% error in average
Other accuracy metrics
Tasks in SCREAM have high recall (low false negative)

29. Conclusion Measurement is crucial for SDN management
in a resource-constrained environment
Practical sketch-based SDM by dynamic memory allocation
Implementing network-wide tasks using sketches
Estimating accuracy for 3 types of tasks
SCREAM is available at github.com/USC-NSL/SCREAM