1. SODA : Service-On-Demand Architecture for Application Service Hosting Utility Platforms Dongyan Xu, Xuxian Jiang Lab FRIENDS (For Research In Emerging Network and Distributed Services) Department of Computer Sciences Center for Education and Research in Information Assurance and Security (CERIAS) Purdue University

2. Outline  Motivations and goals  Related work  Research components of SODA  Summary and on-going work

3. Motivations  Vision of utility computing  Computation utility  Storage utility  Application service hosting  Conference management  e-Campaign  Digital government  Serving the underserved communities  IT function shadowing for disaster recovery  Virtual enterprise, collaboratory, and community

4. Our Goal  To build a value-added application service hosting platform based on shared infrastructure, achieving:  On-demand creation and provisioning  Virtualization  Isolation  Protection  Accountability  Privacy

5.  Utility computing architectures  VERITAS, HP UDC, IBM Oceano  Grid platforms  Computation: Globus, Condor, Legion, NetSolve, Harness, Cactus  Storage and data: SRB, NeST, Data Grid, OceanStore  Shared infrastructure  PlanetLab, Emulab  Active services  Active Service Grid, Berkeley Active Service Framework, CANS (NYU), Darwin, WebOS Related Work

6.  Resource isolation  GARA, QLinux (UMass), Virtual service (UMich), Resource Container, Cluster Reserves (Rice)  Virtualization technologies  Virtual super computer (aggregation): NOW, HPVM  Virtual OS, isolation kernel (slicing): VMWare, Xen (Cambridge), Denali (UW), UML, UMLinux, Virtual Private Server (Ensim)  Grid computing on VM: Virtuoso (Northwestern), Entropia  Virtual cluster: Cluster-on-Demand (Duke) Related Work

7. SODA  Service-On-Demand Architecture for application service hosting utility platforms  Research components of SODA  General architecture  Protection, intrusion detection, logging  Confined and VM-based overlay  Market-driven planning and management

8. Outline  Research components of SODA:  General architecture  Security and protection  Confined VM-based overlay  ‘Property’ planning and management

9. Detailed Information  Xuxian Jiang, Dongyan Xu, "SODA: a Service-On-Demand Architecture for Application Service Hosting Utility Platforms", Proceedings of The 12th IEEE International Symposium on High Performance Distributed Computing (HPDC-12), Seattle, WA, June 2003."SODA: a Service-On-Demand Architecture for Application Service Hosting Utility Platforms"HPDC-12

10. Overview of SODA SODA Host (physical) AS AS’ Virtual service node

11. Virtualization: Key Technique  Two-level OS structure  Host OS  Guest OS  Strong isolation  Administration isolation  Installation isolation  Fault / attack Isolation  Recovery, migration, and forensics  Virtual service node  Application service (AS)  Guest OS  Internetworking enabled One SODA host Host OS … Guest OS AS 1 AS n

12. SODA Master SODA Agent Host OS Guest OS Service S SODA Daemon Host OS Guest OS Service S SODA Daemon Host OS Guest OS Service S’ SODA Daemon Guest OS Service S’ Service Switch for S Service Switch for S’ Service Requests From Clients Service Creation Requests From ASP Virtual service node

13. On the Same SODA Host WWW service Honeypot

14. Host OS and Guest OS  Guest OS: based on User-Mode Linux (UML), an open-source virtual OS ( different from UMLinux and VServer )  By Jeff Dike, http://user-mode-linux.sourceforge.net http://user-mode-linux.sourceforge.net  Running in user space of host OS  Separate kernel address space  Physical memory usage limit  Host OS: Linux (linux-2.4.19, enhanced)  CPU fair share scheduler (for CPU isolation between virtual service nodes)

15. Experiment: CPU Isolation Original Linux SchedulerEnhanced Linux Scheduler VM 1 : CPU-intensive VM 2 : IO-intensive VM 3 : Web

16. On-Demand Service Priming  Performed by SODA Daemon  Customization of guest OS (“cook to order” )  Active service image downloading  Automatic bootstrapping of virtual service node

17. Service Bootstrapping Time Linux Configuration Image size Time (seattle) Time (tacoma) Rootfs_tomrtbt_ 1.7.205 15 MB2.0 sec.3.0 sec. Rootfs_base_1.0 29.3 MB3.0 sec.4.0 sec. Root_fs_lfs_4.0 400 MB4.0 sec.16.0 sec. Root_fs.rh-7.2- server.pristine.200 21012 253 MB22.0 sec.42.0 sec.

18. Slow-Down (w/o optimization) 1,368 37,004 gettimeofday 1,200 27,044 munmap 1,208 27,864 mmap 1,084 26,904 dup2 1,064 26,648 geteuid 1,208 27,276 getpid Linux UMLSystem call System call level (clock cycles) Application level

19. Outline  Research components of SODA:  General architecture  Security and protection  Confined VM-based overlay  ‘Property’ planning and management

20. Detailed Information  Xuxian Jiang, Dongyan Xu, Rudolf Eigenmann, "Protection Mechanisms for Application Service Hosting Platforms", Proceedings of IEEE/ACM Int'l Symposium on Cluster Computing and the Grid (CCGrid 2004), Chicago, IL, April 2004."Protection Mechanisms for Application Service Hosting Platforms"CCGrid 2004  Xuxian Jiang, Dongyan Xu, "Collapsar: A VM-Based Architecture for Network Attack Detention Center", to appear in Proceedings of the 13 th USENIX Security Symposium (Security '04), San Diego, CA, August 2004."Collapsar: A VM-Based Architecture for Network Attack Detention Center"Security '04

21. Security and Protection  Virtual switching and firewalling  IDS in guest OS kernel  Untamperable logging (‘blackbox’-ing) Host OS … Guest OS AS 1 AS n

22. Virtual Switching and Firewalling Virtual machine (with IP addr.) SODA host (Invisible on Internet) Guest OS Host OSFirewall

23. Kernort: IDS in Guest OS Kernel  Problems with traditional IDS  Encrypted traffic (e.g. ssh) makes NIDS less effective  App-level IDS process will be “killed”, once a machine is compromised  Log may be tampered with  Fail-open  Related projects  Backtracker (Michigan)  VMM-based retrospection (Stanford)  Forensix (OHSU)  ESP (Purdue CERIAS)  Open-source projects: Snort, Saint Jude

24. Kernort  VM-based IDS  Deployed in each VM  Inside guest OS kernel: a unique vista point  Customizable without affecting host OS  Clearer view  Untamperable logging (saved to SODA host)  Renewable signature (read from SODA host)  Fail-close instead of fail-open

25. Kernort: IDS in Guest OS Kernel Guest OS

26. Kernort  Components  Kernort sensor  Event-driven (system call and packet reception)  Renewable signature set  Matching against a small signature set (“Top 20 most wanted”)  Kernort blackbox  Untamperable logging  Privacy preservation of ASes  Analyzer  Exhaustive signature matching  Detection of complex attack patterns  Session replay

27. Kernort Virtual machine Host OS Kernort (shaded areas: logs)

28. Real-Time Alert

29. Session Re-play

30. Impact on Performance

32. Outline  Research components of SODA:  General architecture  Security and protection  Confined VM-based overlay  ‘Property’ planning and management

33. Detailed Information  Xuxian Jiang, Dongyan Xu, "vBET: a VM-Based Emulation Testbed", Proceedings of ACM Workshop on Models, Methods and Tools for Reproducible Network Research (MoMeTools, in conjunction with ACM SIGCOMM 2003), Karlsruhe, Germany, August 2003."vBET: a VM-Based Emulation Testbed"MoMeToolsACM SIGCOMM 2003  Xuxian Jiang, Dongyan Xu, "VIOLIN: Virtual Internetworking on OverLay INfrastructure", Department of Computer Sciences Technical Report CSD TR 03-027, Purdue University, July 2003."VIOLIN: Virtual Internetworking on OverLay INfrastructure"  Xuxian Jiang, Dongyan Xu, “A Middleware Architecture for Confined Virtual Machine Overlays", in preparation, March 2004.“A Middleware Architecture for Confined Virtual Machine Overlays"

34. Traditional Overlay Network  Problems with traditional overlays:  Open for attacks  Attacks from the outside (i.e. Internet) against overlay nodes  Attacks from an overlay node against the outside  Difficult to manage  An overlay across multiple administration domains  A host participate in multiple overlays  Difficult to enforce overlay topology and traffic volume  VPN does not solve the problems

35. Traditional Overlay Network Firewall

36. VM-based Overlay  The case for VM-based overlay  Multiple overlays on shared infrastructure  On-demand creation  Confinement and isolation  VM introduces new network administration complexity  “What is this new machine that has suddenly appeared in my domain?”  “Where is the machine that was in my domain yesterday?”  “How much network connectivity should a VM have?”  “How many IP addresses for VMs?”

37. Confined VM-based Overlay  In addition to VM, we need VN for VMs  VN: a highly overloaded term (VPN, X-bone…)  What is new: Confined and VM-based overlays  Applications  Multi-institutional collaborations  Philanthropic (volunteer) computing systems  Network emulations

38. Confined VM-based Overlay Firewall VM ≤1Mbps ≤2Mbps Virtual infrastructure

39. Key Properties  Confined overlay topology and traffic  No attack possible from inside the overlay to the outside world  Virtual IP address space  No need for application modification and re-compilation

40. A More Generic Picture VIOLIN: Virtual Internetworking on OverLay INfrastructure

41. vBET: an Example of Confined Overlays on Demand  An education tool for network and distributed system emulation  Fidelity-preserving setup  Maneuverable network entities  Real-world network software  Strict confinement (network security experiment)  Flexible configuration  Not constrained by device/port availability  No manual cable re-wiring or hardware setup  Simultaneous experiments  Cost-effective

42. vBETvBET Features  Can be deployed in n ≥ 1 vBET servers  Efficient startup and tear-down of emulated entities  Strong network virtualization  IP address space  Virtual routers, switches, firewalls, end-hosts, links  Communications confined by virtual topology  Dynamic addition, deletion, migration, configuration of network entities

43. vBET GUI

44. Sample Emulation: OSPF Routing

45. Emulation of OSPF Routing Demo video clip at: http://www.cs.purdue.edu/~jiangx/vBET/videos/vbet_ospf.avi

46. Sample Emulation: Distributed Firewalls

47. Screenshot

48. Sample Emulation: Chord P2P Network

49. Screenshot

50. Outline  Research components of SODA:  General architecture  Security and protection  Confined VM-based overlay  ‘Property’ planning and management

51. Property Planning and Management  Tenant selection:  Among a set of potential tenants (ASes), which ones to host? (for maximum revenue, resource utilization, security…)  SODA provider selection:  Among a set of SODA providers, which one should be chosen to host an AS?

52.  Examples of bad planning:  Many PDA transcoding ASes in an area with a small PDA user population  AS not requiring client registration and log-in (potential DDoS attacks)  Majority of ASes exhibiting similar demand characteristics such as: Property Planning and Management Load Time

53. Property Planning and Management  AS profiling  Resource requirement  Security/authentication  Demand characteristics  Market analysis  Competing ASes, market size/growth/expected share  ASes correlation (“80% of clients requesting AS X also request AS Y” )  Trading/pricing of SODA machine slices

54. Property Planning and Management  Forming alliance of SODA providers

55. Property Planning and Management  Forming alliance of SODA providers

56. Summary  Virtualization: a key enabling technology in realizing utility computing vision  Hosting utility is more complex than computation utility (host – tenants – clients)  SODA achieves:  On-demand service creation  Service virtualization, isolation and confinement  Protection, accountability, privacy  Overlay isolation and confinement

57. Ongoing Work  VM/service migration, shadowing, recovery  Service profiling, accounting, auditing (resources, security)  Market-driven planning, provisioning, and management (SODA ecology)  Deployment and evaluation (Purdue Bindley Bioscience Center)

58. Thank you. For more information: {dxu, jiangx}@cs.purdue.eduxu, jiangx}@cs.purdue.edu http://www.cs.purdue.edu/~dxu AOL keywords “Purdue SODA Friends”