1. 1 Cloud Computing Research at TU Delft – A. Iosup Alexandru Iosup Parallel and Distributed Systems Group Delft University of Technology The Netherlands Our team: Undergrad Gargi Prasad, Arnoud Bakker, Nassos Antoniou, Thomas de Ruiter, … Grad Siqi Shen, Nezih Yigitbasi, Ozan Sonmez Staff Henk Sips, Dick Epema, Alexandru Iosup Collaborators Ion Stoica and the Mesos team (UC Berkeley), Thomas Fahringer, Radu Prodan (U. Innsbruck), Nicolae Tapus, Mihaela Balint, Vlad Posea (UPB), Derrick Kondo, Emmanuel Jeannot (INRIA),... Cloud Computing Research at TU Delft (and TU/e) 3TU. =++ EIT Meeting, Berlin, 24 February 2011

2. Cloud Computing Research at TU Delft – A. Iosup 2 PDS Group: Past Research in Grids The Koala Grid Scheduler Co-allocation Deployed since 2005 DAS-2,-3,-4 A Grid Research Toolbox The Grid Workloads Archive GrenchMark DGSim Performance models Availability models DGSim http://gwa.ewi.tudelft.nl

3. Cloud Computing Research at TU Delft – A. Iosup 3 Cloud Futures Workshop 2010 – Cloud Computing Support for Massively Social Gaming 3 A View on Cloud Computing “The path to abundance” On-demand capacity Pay what you use Great for web apps (EIP, web crawl, DB ops, I/O) “The killer cyclone” Not so great performance for compute- or data- intensive e-Science apps 1 Performance variability 2 http://www.flickr.com/photos/dimitrisotiropoulos/4204766418/ Tropical Cyclone Nargis (NASA, ISSS, 04/29/08) 1- Iosup et al., Performance Analysis of Cloud Computing Services for Many Tasks Scientific Computing, IEEE TPDS, 2011 (in print) http://www.st.ewi.tudelft.nl/~iosup/cloud-perf10tpds_in-print.pdf http://www.st.ewi.tudelft.nl/~iosup/cloud-perf10tpds_in-print.pdf 2- Iosup et al., On the Performance Variability of Production Cloud Services, CCGrid 2011, pds.twi.tudelft.nl/reports/2010/PDS-2010-002.pdfpds.twi.tudelft.nl/reports/2010/PDS-2010-002.pdf VS

4. Cloud Computing Research at TU Delft – A. Iosup 4 Cloud Performance Studies Many-Tasks Scientific Computing Quantitative definition: J jobs and B bags-of-tasks Extracted proto-MT users from grid and parallel production environments Performance Evaluation of Four Commercial Clouds Amazon EC2, GoGrid, Elastic Hosts, Mosso Resource acquisition, Single- and Multi-Instance benchmarking Low compute and networking performance Clouds vs Other Environments Order of magnitude better performance needed for clouds Clouds already good for short-term, deadline-driven scientific computing 1- Iosup et al., Performance Analysis of Cloud Computing Services for Many Tasks Scientific Computing, IEEE TPDS, 2011 (in print) http://www.st.ewi.tudelft.nl/~iosup/cloud-perf10tpds_in-print.pdf http://www.st.ewi.tudelft.nl/~iosup/cloud-perf10tpds_in-print.pdf 2- Iosup et al., On the Performance Variability of Production Cloud Services, CCGrid 2011, pds.twi.tudelft.nl/reports/2010/PDS-2010-002.pdfpds.twi.tudelft.nl/reports/2010/PDS-2010-002.pdf

5. Cloud Computing Research at TU Delft – A. Iosup 5 The Cloud Workloads Archive Looking for invariants Wr [%] ~40% Total IO, but absolute values vary # Tasks/Job, ratio M:(M+R) Tasks, vary Understanding workload evolution Trace IDTotal IO [MB]Rd. [MB]Wr [%] HDFS Wr[MB] CWA-0110,9346,80538%1,538 CWA-0275,54647,53937%8,563

6. Cloud Computing Research at TU Delft – A. Iosup 6 More Research on Clouds at TU Delft Scheduling cloud workloads MapReduce Scientific applications Cloud-based gaming infrastructure Massive Gaming is multi-billion business Driver for cloud resource management research Driver for distributed systems research CAMEO: MMOG Analytics POGGI: Game Content Generation A. Iosup et al., CAMEO: Enabling Social Networks for Massively Multiplayer Online Games through Continuous Analytics and Cloud Computing, ACM NetGames (2010) A. Iosup, POGGI: generating puzzle instances for online games on grid infrastructures. CCPE 23(2): 158-171 (2011)

7. Cloud Computing Research at TU Delft – A. Iosup 7 Take Home Message: 3-TU Research in Clouds Understanding how real clouds work Modeling cloud infrastructure (performance, availability) and workloads Compare clouds with other platforms (grids, parallel production env., p2p,…) The Cloud Workloads Archive: easy to share cloud workload traces and research associated with them Complement the Grid Workloads Archive Scheduling: making clouds work eScience and gaming applications MapReduce Massive Gaming: services on clouds CAMEO: Massive Game Analytics Toolkit for Online Social Network analysis POGGI: game content generation at scale  See Pascal van Eck’s presentation Publications 2008: ACM SC 2009: ROIA, CCGrid, NetGames, EuroPar (Best Paper Award) 2010: IEEE TPDS, Elsevier CCPE,… 2011: ICPE, CCGrid, Book Chapter CAMEO, IEEE TPDS, IJAMC, …

8. Cloud Computing Research at TU Delft – A. Iosup 8 Thank you for your attention! Questions? Suggestions? Observations? Alexandru Iosup A.Iosup@tudelft.nl http://www.pds.ewi.tudelft.nl/~iosup/ (or google “iosup”) Parallel and Distributed Systems Group Delft University of Technology A.Iosup@tudelft.nl http://www.pds.ewi.tudelft.nl/~iosup/ - http://www.pds.ewi.tudelft.nl/publications.phphttp://www.pds.ewi.tudelft.nl/publications.php - http://www.st.ewi.tudelft.nl/~iosup/research_cloud.htmlhttp://www.st.ewi.tudelft.nl/~iosup/research_cloud.html More Info: Do not hesitate to contact me…