Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009

OOI CI Kick-Off Meeting, Sept 9-11, Outline Subsystem Architecture Overview Scope of Release 1 Selected Components –Elastic Computing –Global Network Strategy

3 OOI CI Kick-Off Meeting Sept 9-11, 2009 Common Execution Infrastructure Provides the execution and deployment environment for CI services and user processes

Deployment Scenario 4 OOI CI Kick-Off Meeting Sept 9-11, 2009

Scope of Release 1 Virtual Process Execution Management –Definition of target execution environment independent processes –Support of multiple cloud execution environments –Generation of target specific executables (images) Elastic Computing –Elastic scheduling, provisioning, execution –Monitoring and management Resource Management –Compute Nodes and Target Environments –Process Repository OOI CI Kick-Off Meeting, Sept 9-11,

Elastic Provisioning Example 6 OOI CI Kick-Off Meeting Sept 9-11, 2009 Imperative to separate mechanism from (smart, specialized) policy

Cloud Provisioning Architecture 7 OOI CI Kick-Off Meeting Sept 9-11, 2009

CEI Functional Components

Operational Unit

Execution Engine

Example: Data Exchange

Deployment Concepts 12 OOI CI Kick-Off Meeting Sept 9-11, 2009 Operational Architecture Deployment Architecture Implementation Architecture

Execution Engines 13 OOI CI Kick-Off Meeting Sept 9-11, 2009

Deployment - Execution 14 OOI CI Kick-Off Meeting Sept 9-11, 2009

Cloud Provisioning Basics 15 OOI CI Kick-Off Meeting Sept 9-11, 2009

Cloud Instance Startup 16 OOI CI Kick-Off Meeting Sept 9-11, 2009

Observe–Decide–Act Pattern 17 OOI CI Kick-Off Meeting Sept 9-11, 2009

Network Logical Deployment Model 18 OOI CI Kick-Off Meeting Sept 9-11, 2009

Component Deployment Model 19 OOI CI Kick-Off Meeting Sept 9-11, 2009

Hardware Deployment Model 20 OOI CI Kick-Off Meeting Sept 9-11, 2009

Network Architecture

22 OOI CI Kick-Off Meeting Sept 9-11, 2009 Network Deployment

Academic Global Gigabyte Network

Academic National Gigabit Network

OOI Operational Locations Microsoft Amazon West Amazon East Woods Hole Seattle Washington DC San Diego Portland Corvallis Management Point Acquisition Point Distribution Point Execution Point

Technology Mapping Cloud Execution Sites –AWS/EC2, Science Clouds (Nimbus) –GoGrid, FlexiScale (optional) National Grid Resource Interface –Condor for high throughput jobs –Teragrid Gateways

OOI CI Kick-Off Meeting, Sept 9-11, Thanks !

Cloud Computing Strategy Requirements –a secure, scalable, fault-tolerant network and computing infrastructure that spans the OOI marine instrument networks to the national (academic & commercial) computing/data facilities –a participation model that allows individual research teams to integrate the computing environments at their intuitions into this infrastructure Strategy –provide an effective development and operational process that allow the OOI members and its operational staff to transparently use and effectively manage the OOI specific computing resources as a coherent cloud resource Definition –style of computing where ICT capabilities are provided “as a Service”, –allowing users to access technology-enabled services without knowledge of, expertise with, or control over the technology infrastructure that supports them

Cloud Strategy Benefits Attributes –Effective coupling of resource acquisition to user demand –Streamlining the preparation, deployment and management of observatory applications –Removing the management and ownership of physical computing resources from the direct concern of the Observatory Results –Eliminate centralized capital assets associated with networking storage, and computing as well as associated staffing –Continual transparent tracking of the technology advancements –Ability to respond to changes in usage demand in minutes and changes in market costs within weeks

Architectural Concerns Operational –Enterprise of Applications, Services and the deployed Modules to support the Enterprise Deployment (concern of CEI) –Secure, Scalable, Self-healing Execution Platform Implementation (concern of CEI) –Configuration framework to compose and condition specific classes deployable execution units (Capability Containers) Functional –Decomposition of the system based on functional and construction concerns

CEI Components 40 min: Elastic scaling and provisioning –Load graph: Start story. From there to CPE drawings or DX drawing. System scales according to load. –Domain models development, deployment, operation architecture OV7 overview, execution engine OV7 –CPE use case 20 min Network: Global site selection (Matthew) –Deployment architecture –Detailed network diagrams (Brian) –National backbone infrastructure –GLIF (Tom) for network environment we’re deploying into. –Story: National DIF framework using Solace technology, link to COI/Messaging. Higher density computing per enclosure, diversity of deployment. Possibility for inner core messaging architecture (very efficient on the backbone); commodity equipment OOI CI Kick-Off Meeting, Sept 9-11,