1. Network-Aware Operator Placement for Stream-Processing Systems CS253 project presentation Min Chen, Danhua Guo {michen, dguo}@cs.ucr.edu 12/4/2006

2. 2 Roadmap Motivation Motivation Background Knowledge Background Knowledge Project Architecture Project Architecture Evaluation Evaluation Conclusion Conclusion

3. {michen,dguo}@cs.ucr.edu3 Motivation Stream-based overlay network (SBON) Stream-based overlay network (SBON) –Operator placement (RELAXATION) Designed for large-scale DSPS Designed for large-scale DSPS Network efficiency Network efficiency Low delay Low delay –Cost space

4. {michen,dguo}@cs.ucr.edu4 Motivation Cost Space Cost Space –Idea: Problem -> Virtual space -> physical node –Latency (& Load) dimensions

5. {michen,dguo}@cs.ucr.edu5 Background Knowledge Network Usage: A Blended Metric Network Usage: A Blended Metric –Bandwidth-delay product –Individual applications perspective Small delay Small delay –Network perspective Scaling bandwidth by latency Scaling bandwidth by latency

6. {michen,dguo}@cs.ucr.edu6 Architecture Overview Overview

7. {michen,dguo}@cs.ucr.edu7 Operator Placement Algorithm RELAXATION RELAXATION –Adapt to changing conditions –Decentralized –Considers impacts of shared placement decisions

8. {michen,dguo}@cs.ucr.edu8 Operator Placement Algorithm Algorithm Algorithm –Virtual Operator Placement

9. {michen,dguo}@cs.ucr.edu9 Operator Placement Algorithm Algorithm Algorithm –Physical Operator Mapping Find k nodes whose coordinates are near the target cost space coordinate. Sort the list by distance to the target coordinate. Walk the sorted list, returning the first node already running the operator. Failing that, return the nearest node that meets the application’s resource criteria

10. {michen,dguo}@cs.ucr.edu10 Operator Placement Result Δ=0.1 and Ft = 1 Δ=0.1 and Ft = 1

11. {michen,dguo}@cs.ucr.edu11 Evaluation Based on PlanetLab trace files Based on PlanetLab trace files –Efficiency compared with other algorithm Optimal: exhaustive search Optimal: exhaustive search Producer: place the operator at producer Producer: place the operator at producer Consumer: place the operator at consumer Consumer: place the operator at consumer Random: place the operator at a random node Random: place the operator at a random node

12. {michen,dguo}@cs.ucr.edu12 Evaluation Set up Set up –Bandwidth: 2KB/s –# of Nodes: 226 nodes from PlanetLab –# of Queries: 1000 –Query set up: 4 producers, 1 comsumer, 1 unpinned operator per query

13. {michen,dguo}@cs.ucr.edu13 Evaluation Result Result

14. {michen,dguo}@cs.ucr.edu14 Evaluation Delay penalty after placing 1000 queries Delay penalty after placing 1000 queries Algorithm Network Usage PenaltyDelay Penalty Optimal0.00%1.01% Relaxation1.34%7.71% Producer24.91%50.99% Consumer27.36%0.00% Random54.14%57.26%

15. {michen,dguo}@cs.ucr.edu15 Conclusion Our experiment shows that SBON is efficient in terms of Network Usage (Bandwidth, Latency)

16. {michen,dguo}@cs.ucr.edu16 Reference Peter Pietzuch et. al. “Network-Aware Operator Placement for Stream-Processing Systems” Peter Pietzuch et. al. “Network-Aware Operator Placement for Stream-Processing Systems” Peter Pietzuch, Jonathan Ledlie, and Margo Seltzer, “, In Proceedings of WORLDS 2005, San Francisco, CA, December 2005 Peter Pietzuch, Jonathan Ledlie, and Margo Seltzer, “Supporting Network Coordinates on PlanetLab”, In Proceedings of WORLDS 2005, San Francisco, CA, December 2005