1. Cloud Federations Patrizio Dazzi (ISTI-CNR) [Overall Presentation] patrizio.dazzi@isti.cnr.it Gaetano Anastasi (ISTI-CNR) [Hands-On] gaetano.anastasi@isti.cnr.it contrail is co-funded by the EC 7th Framework Programme under Grant Agreement nr. 257438 http://contrail-project.eu

2. Presentation Outline Cloud Federations Contrail Federations Contrail Federation Architecture Federations and Resources A few details about the current status of Contrail Cloud Federations Introduction to the hands-on session

3. Cloud Federations

4. What is Cloud Computing ? Computing as a public utility – Clouds are the computing “power plants” Do not manage resources, just use them Choose the interaction model: SaaS, PaaS, IaaS 4

5. Principal Advantages of Cloud Comuting Advantages for Tenants: – Pay just what you get – Pay only when you get – Reduce costs for maintenance Advantages for Providers – maximizing the effectiveness of the shared resources. – dynamically re-allocation depending on actual usage 5

6. Anyway… Providers have: – A limited amount of resources – A limited range of resource types – Resources placed only in same countries Providers want: – to avoid that their customers leave them – To make (more) money Users want – To have all the resources they need (possibly more) – To pay less…. How can we manage this ??? 6

7. Cloud Federations A Federation of Clouds is (maybe) the answer – By federating, providers can offer more resources – More resources means… To be able to run bigger applications To be more elastic To provide (potentially more) different resources – A federation (may) allow users to Choose among a wider range of providers Seamlessly use more providers for the same application Migrate from a provide to another 7

8. Federation of IaaS providers Infrastructure as a Service – provide computation, storage and network IaaS Providers – distinct Clouds may have different interfaces, access rules, capabilities, prices, … Special Providers – Peculiar computational supports – Specific Storage capabilities – High performance networking 8

9. How Federation of IaaS impacts on PaaS and SaaS The background: – Platform and Software as a service offer an even more complex landscape – PaaS and SaaS provider may be distinct from IaaS ones IaaS Federation can – Allow properly instrumented PaaS and SaaS to run on top of a huge amount of resources – Need to translate PaaS and SaaS requirements To address heterogeneity of providers Enforce QoS guarantees 9

10. Unified Identity and Billing management Federations should be almost invisible from the point of view of users but: – User’s identity should be automatically managed In a cross-cloud fashion Creating and managing proper identities in each cloud (if needed) Map by keeping proper capabilities and permissions – Users have to be properly billed Unified monitoring Unified accounting 10

11. Security Mechanisms Authentication – Allow the access of users to Clouds – Exploit existing identity infrastructures E.g. OpenID – Enact an identity federation Authorization – Access Control – Usage Control Particularly useful for long lasting applications 11

12. Cross-Cloud management of Service Level Agreements Cloud Applications have QoS requirements – Performance, reliability, security – Typically expressed as SLAs SLA management is a complex activity also in a single Cloud – Violations may occur and need to be managed Even more complex in a Federation of Clouds – Not all providers provide the same degree of guarantees strictness penalties Compensation – An overall coordination activity shall be performed 12

13. Last but not Least: Brokering Cloud Federations also behave as brokers – To find for each user’s application the best Cloud(s) depending on Application structure and requirements Cloud reputation Cost – Splitting the user’s application in order to Choose the best Cloud for each part of the application Allow a better exploitation of Specialized Clouds 13

14. Contrail Federations 14

15. Contrail Iaas Federation A Contrail Federation integrates in a common platform multiple Clouds, both public and private ones. To perform this task it provides: A common support for authentication, authorization and billing Mechanisms for policy definition, monitoring, and enforcing for all the QoS-related aspects An automated selection of providers depending on the user applications A support for resource provisioning to applications able to deal with heterogeneous sets of resources

16. Contrail development methodology Do not rebuild and reinvent – Contrail exploits Standards OVF OCCI OpenID – Exploit existing platforms and functionalities Open Nebula SLA@SOI Etc. 16

17. Develop a Federation support that integrates and actively coordinates SLA management provided by single Cloud providers Do not disrupt provider’s business model Allow exploiting a Federation seamlessly Federation Support must be scalable Exploit Providers’ SLA support

18. Main Contrail Federation Innovations (1) Federation = more than a simple broker, or portal, or decentralized cloud-bursting – Interoperability – Heterogeneous providers – Dynamically choosing best providers – At deploy time and at runtime – Allow to combine providers – Migration, elasticity – Security and privacy framework

19. Main Contrail Federation Innovations (2) 19 Sophisticated SLA/QoS – QoS via SLA – Via provider selection and integration – Enforcement mechanisms – Federation as a mediator and a 3 rd party – Federation also acts as a coordinator

20. Contrail Federation Architecture 20

21. Simplified Blocks Architecture Final Users ConPaaSCloud Federation Cloud Provider 1 Cloud Provider 2 … Cloud Provider N 21

22. Distributed Federation Access Points Several Federation access points (FAPs) FAP act as brokers, but share a common view – Security, status of resources, users and providers reputation F F F F F F F F Hierarchical structure Common Policies Detailed resource allocation is on providers AP may be hosted by providers

23. Federation Interfaces SLASLA AuthAuth AuthZAuthZ Federation Core Coarse-grain view of Contrail Federation Architecture Functionalities which extend horizontally in the platform Provisioning Manager

24. Concrete Architecture In the next slide Module View of a single Access point Interactions, some modules not shown Interfaces sit on top of this core Auth/AutZ mechanisms included 24

25. Contrail federation 25 Provisioning Manager

26. Federations and Resources 26

27. Basic concepts What’s an application for IaaS? – A set of software entities which need to be deployed on a suitable set of resources for execution The parties involved – User Who submit applications – Provider Any entity managing physical resources which may be used to run applications – Federation The union of many providers under a common set of APIs and functionalities, which can be exploited as a single Cloud

28. Application Description The requirements so far can be expressed as a Task Interaction Graph – An undirected graph G(N,E) can model a distributed application – Each Node in N is a task (needs a resource) – Edges imply relations between tasks (need links) – Heavily labeled graph Nodes state all resource constraints and SLA measures Edges labeled with communication needs – Cloud applications can get hard to describe

29. Types of resources Computation resources – Available VMs slots on top of physical hardware Storage resources – Shared filesystems – Shared Databases Networking resources – Virtual networks connect machine within an application – Behaviour of the joining points between the internet and the federation is important

30. How resources are measured ? Static specification by analogy with the physical counterparts – Memory size, CPU type and speed (peak) – Size, FS of storage – Nominal bandwidth and latency of networks Dynamic specification – Available computing power, memory – Actual (average, peak, used) storage speed – Observed in-Cloud bandwidth and latency – Observed bw/lat to the outside – Application-specific metrics

31. Provider’s View on resources Providers know exactly the layout and state of all physical (virtual) resources – Dedicated link bandwidths, physical memory… They can greatly optimize resource management – Turn off unused resources, exploit cheapest – They have their own goal (revenues)

32. QoS properties of resources Extend the set of characteristics to be measured on the platform Protection – Type of security mechanisms which are in place Auth. Protocols, Encryption mechanisms, Isolation Privacy – Guarantees offered by storage holder, network infrastructure Geo-localization – Can have deep legal implications Power consumption – Overall power, power efficiency

33. QoS expressed via SLAs As the SLA is signed, the user should be able to trust resources from the platform – But not all Providers may offer the same reliability How reliability can be measured ? – failures, SLA violations with different providers – lengthy task with poor reward for single users

34. Provider Reputation Management information – Available resources per kind – Features granted – Amount of users/apps ongoing – State of SLAs controlled by the federation – Static level of “trust” given from federation to the provider Past History – History of SLA violation per user / type of app – Average level of satisfaction of SLA

35. Contrail approach to SLA – Reuse SLA@SOI framework as a starting point Integration with Contrail internal interfaces and components Integration with domain-specific reasoning/monitoring plugins – Extend SLA@SOI with: Federation support Integration with external providers (and their SLA management systems) Reputation model for providers Cost-based QoS enforcement

36. Federation-level SLA Enforcing Federation will act as a SLA coordinator over providers – As much as possible the single providers is in charge of the local SLA Reduce reaction delay – Federation evaluates the provider and the app – Extend the SLA@SOI monitoring infrastructure to the federation Keep track of main application parameters – Receive SLA violations from providers – Reallocate some resources on a provider basis May require a new negotiation between federation and provider

37. What if… a SLA is violated in a single provider scenario ? The application is deployed on a single provider, which may still violate SLA Actions: – The provider resize the set of allocated resources – If the application is no longer violating SLAs The application will be up and running and that’s it – Otherwise A renegotiation phase is conducted and if will not be successfully the application will be terminated

38. …and what if the violation occurs in a multi- provider scenario ? The application has been sent by the federation to a provider for the execution and a SLA violation occurs Actions: – Previous slide shows what happens when the provider is able to manage the violation by itself – If it is not able, the federation can still migrate (part of) the application to a different provider 38Contrail S. School 2012 – P. Dazzi- Cloud Federations

39. Applications Running in Multiple Providers Some applications could be also decomposed in parts and each deployed in a different provider Actions – resize part(s) managed by providers – migrate part(s) may need to stop the application – violate constraints By leveraging the whole federation we push away the limit where a violation is unavoidable – rebalance constraints By renegotiating with more providers, overall SLO may be achieved Increase resource availability where they are cheaper/more available at the moment

40. Anyway SLA splitting is still an open issue Necessary to leverage SLA management at single providers How to derive a combination of SLA for separate parts of the application which allow to manage the application overall Not yet addressed in literature Not hard if providers are specialized – Compute, network, storage = SLA aspects As the user expresses SLA terms on parts of the application (task groups) this will become easier

41. Contrail Federation, the 3 rd Player A Contrail Federation sits in the middle Aims – Serving the users – Exploit efficiently the providers If economic gain is sought, it comes form intermediation Can efficiently gather provider information – Gathers from all users and all providers – Can make better informed choices than users – Can afford to launch directed tests in doubt – Can trigger corrective actions impossible to single providers

42. A few details about the current status of Contrail Cloud Federations 42

43. How it is used Federation Interfaces use REST The Web portal and command line tools access the REST layer Different roles – federation coordinator – cloud administrator – end user Main steps – account creation – SLA formation – Application upload (VMs and descriptor) – SLA negotiation and agreement – Deploy 43

44. What is implemented today User management – Registration, mapping are working – some integration activities are still ongoing SLA Management – Negotiation with providers counterpart is working – A complete integration is expected in the next release Application management – Applications can be submitted and executed – So far only one provider at a time can be used for a single application 44

45. Aligning with standard formats OVF (Open Virtualization Format) – Open format for describing application – Allows to describe structured applications – Directly expresses only HW constraint and deployment information Partially overlaps with full SLA specification – OVF is extendable – As of v1.1.0 it does not target Application Management SLA@SOI provides a general formalization – Includes monitoring hierarchy and negotiation Exploit a combination of OVF and SLA@SOI mechanisms

46. “Securing” the implementation The current implementation of federation authentication – Support for OpenID – Additional details in Jens’s keynote The current implementation of federation authorization – Advanced support to access control – Ongoing work to finalize the support to usage control 46

47. Ongoing work about advanced features Multi-criteria mapping algorithms – Genetic algorithms for application mapping Evolving mapping plans to achieve better allocations Application splitting – Interplay with SLA splitting Monitoring data aggregation and filtering 47

48. Introduction to the hands-on session Contrail S. School 2012 – P. Dazzi- Cloud Federations48

49. Sample Application OVF Representation 2 Appliances 49Contrail S. School 2012 - M. Coppola - Cloud Federations Number of virtual CPUs 1 virtual CPU 1 3 1 byte * 2^20 Memory Size 512 MB of memory 2 4 512

50. Simplified Deployment Chain Users submit an OVF Federation mapping phase: – Which provider(s) for the application? Federation Application Submission Provider Deployment – Provisioning Manager: receives requests coming from the federation – VEP: manages provider resources for deployment For simplicity do not consider SLAs 50Contrail S. School 2012 - M. Coppola - Cloud Federations

51. Thanks for you attention! 51 Questions?

52. Funded under: FP7 (Seventh Framework Programme) Area: Internet of Services, Software & Virtualization (ICT- 2009.1.2) Project reference: FP7-IST-257438 Total cost: 11,29 million euro EU contribution: 8,3 million euro Execution: From 2010-10-01 till 2013-09-30 Duration: 36 months Contract type: Collaborative project (generic) contrail is co-funded by the EC 7th Framework Programme