Tech Arch Proposal for a Federated Cloud Building the Helix Nebula System by Tech Arch Group

Tech Arch Mission Tech Arch is built from cloud technology and architecture experts from supply-side partners Provide, from a technical perspective: Requirements Solution and Architecture Drive the Roadmap – incremental releases Reference technologies as building blocks Share lessons learned and knowledge in running PoCs

Helix Nebula Architecture Document Objective Bring together past and current knowledge Set a stake in the ground Elicit feedback Devise an incremental roadmap No implementation constraints (e.g. budget, time)

Helix Nebula Architecture Document Requirements (demand-side) Helix Nebula Service Enabling Framework Cross-analysis Roadmap Conclusion Current report : 1.5 on Helix Nebula Workshop Timetable

Requirements Actors Supply-side – Providers (e.g. Providers, Consumers, SMEs) Demand-side – Consumers Broker – Middle men (blue box) SME Supplier - Independent Service Providers SME Consumer - Independent Service Providers

User Story Categories Simple Provisioning Contextualisation Image Management Data Transfer and Placement Image Marketplace Data Marketplace Security Networking Cross Cloud / Federated Provisioning Hybrid Cloud Deployment Service Discovery

Requirements Sample Identifies actors (e.g. Providers, Consumers, SMEs) Captures functional requirements in the form of User Stories: “As a demand-side system administrator, I can install a Helix Nebula connector, such that my cloud provisioning solution can provision resources on the public HN cloud, alongside local cloud resources”

Demand-Side Functional Requirements Phase 1: until end 2012Phase 2: until Q Portability / management of the images: "upload once with internal conversion" -- user story Harmonize QoS, cost model, Service management / monitoring & accounting 2. Transparent accounting of resources; monitoring / management of the service & payment according to agreed principles 2. Simple system allowing timely advanced knowledge about scale of cost 3. Standardize the interfaces to ICT provisioning as in PoC (compute, network, storage) - common API ? - dynamic feature: system division & VM scaling 3.Data storage in the cloud, with transparent on-line accounting (costs) 4. Contextualization - auth credentials - arbitrary user data 4. Interconnectivity of Suppliers for data exchange 5. Security, fulfilling agencies security requirements

Demand-Side Specific Requirements Phase 1: until end 2012Phase 2: until Q ESA- Individual accounts for selected scientists having access to data, processors and tools - Online accounting of usage of resources - SSO should be working for each provider - "devpay" support - web based exploitation platform (workbench) providing 3 working areas: 1. private with sharing functionality 2. Public area where results, data and tools can be posted 3. Quality controlled area, only peer reviewed results, data, tools can be issued CERN - Thin provisioning: booting of N machines should be fast (O(minutes)) - Public IP address with reversible DNS - Upload image once -- internal conversion and distribution to specific HN providers - Ephemeral scratch disk at boot time EMBL - EC2 API compliance

Service Enabling Framework So-called “Blue Box” Provides API services and a Web Portal Provide users with unified interfaces Provide suppliers with clear container/interfaces

The “blue box”

Cross Analysis Methodology by reviewing of existing cloud provisioning solutions (Open source or commercial) What are the good candidates considered for general tradeoff analysis ? What would be missing for the HN BlueBox 0.9 as defined August 2012 ? What is currently well supported by HN partners ? What is relevant for the HN BlueBox next versions ? Technical analysis from user stories to “blue box” components

Features analysis Identified Cloud FeaturesBlue Box Phase 1 component(Existing Solution) Cloud Interfaces (front-end)APIEC2, jclouds, OGF OCCI, … Virtual Image ManagementImage factoryVMI supported, converter, Service ContextualizationProvisioning ServiceYes/No (ISO, user data, …) BrokeringProvisioning ServiceYes/No SchedulingProvisioning ServiceYes/No Administration InterfacePortalYes Virtualization Adapter (to Hypervisor) Cloud Adapter (to IaaS provider) API Mediation Layer Supported API to -hypervisor (e.g. XEN, KVM, VMware, …) -cloud provider (OCCI, EC2, …) Hybrid CloudFederationYes/No (mobility and service wrapping) Computing ServiceMarketplace / Catalogue Specific service provisioning (MapReduce, GRID, Databases, …) Accounting and billingBilling/paymentsYes/No (model, … ) Virtual Network ManagementNetwork LayerSupported networks (VLAN, VPN, …) SecurityIdentity Management Authentication methods (e.g. password, SSH, X.509) + SSO drivers Flexibility and Extensibility (plugins development) Plugin framework? License Open source license or commercial conditions

Roadmap Build the Helix Nebula vision incrementally What would be the key features to make the “blue box” relevant? For users and providers? Identified 3 releases with corresponding features

Roadmap Phase 1 – User I/F unification Federation – SSO Service catalog Image Factory & Marketplace (no conversion) Provisioning (single cloud at a time), with contextualisation and placement criteria Monitoring/metering – vendor specific, but harmonised Data management – ingest, transfer, export and replicate Billing/Payment – vendor specific, but harmonised Termination – de-provisioning, quarantine Security – full firewall support, key rotation Networking/connectivity – all provider connected on GEANT

Roadmap Phase 2 – Cloud Provider unification Image conversion between different vendor’s infrastructure Combined Metering and Billing Focus on automating the various security aspects within Helix Nebula

Roadmap Phase 3 – Added value services Improved monitoring and metering Image and Data marketplace with ‘devpay’ like service Multi-cloud and hybrid cloud provisioning

Conclusion One year on… Reference Architecture Requirements analysis against exisiting Roadmap … “y a plus qu’à”!!

Thanks!! Paul Parsons, The Server Labs Phil Evens, Logica (DE) Hervé Caumont, Terradue Emmanuel Mathot, Terradue Micheal Higgins, CloudSigma Gunnar Billing, Atos Marc-Elian Bégin, SixSq … and all other Tech Arch members