1. Ocean Observatories Initiative OOI Cyberinfrastructure Overview Matthew Arrott VMware Presentation March 5, 2010

2. OOI CI LCO Review, Feb 2010 2 Goals for this Presentation

3. OOI CI LCO Review, Feb 2010 3 Overview The OOI Integrated Observatory OOI Network Architecture OOI Logical Architecture Conclusions

4. OOI CI LCO Review, Feb 2010 4 OOI Program Overview

5. OOI CI LCO Review, Feb 2010 5 OOI Integrated Observatory

6. OOI CI LCO Review, Feb 2010 6 Integrated Observatory

7. OOI CI LCO Review, Feb 2010 7 Science & Education Operations Interactive Ocean Observing Interactive Ocean Modeling & Data Assimilation Automated Data Product Generation Discipline-Driven Semantic Organization of Data Interactive Instrument Network Integrated Observatory Management User-Driven Integration of Resource Observatory Activity Model

8. OOI CI LCO Review, Feb 2010 8 Scope Release 1: Data Distribution Network (2011) –“Ingest” data (products) from external data sources; both streaming and cataloged –Characterize data sources with their metadata attributes (format, structure, meaning) –Transform external data into internal OOI data formats, for stream-based distribution and persistence –Distribute data via streaming and DAP servers to data analysts and numerical modelers –Provide initial instrument integration, control and sensor data acquisition –Provide instrument and platform direct access –Provide service integration and execution platform

9. OOI CI LCO Review, Feb 2010 9 Scope of Future Releases Release 2: Managed Instrument Network (2012) –Fully managed instrument activation and control; ready for deployment on OOI moorings and cable infrastructure –Qualified data products based on automated data processing Release 3: On-Demand Measurement Processing (2013) –Observatory management and resource scheduling –On demand and event driven measurements –Data processing workflows Release 4: Integrated Modeling Network (2014) –Interactive analysis and synthesis –Numerical model integration Release 5: Interactive Ocean Observatory (2015) –Full control of data, processes, models, instruments –Closed loop, adaptive sensing and platform control

10. 10 Goal after Release 5

11. OOI CI LCO Review, Feb 2010 11 Release Schedule

12. OOI CI LCO Review, Feb 2010 12 Network Architecture

13. OOI CI LCO Review, Feb 2010 13 Network Architecture

14. OOI CI LCO Review, Feb 2010 14 International Research Networks

15. OOI CI LCO Review, Feb 2010 15 National Research Networks

16. OOI National Network OOI CI LCO Review, Feb 2010 16

17. OOI CI LCO Review, Feb 2010 17 Network Deployment

18. OOI CI LCO Review, Feb 2010 18 OOI Logical Architecture

19. OOI CI LCO Review, Feb 2010 19 Release-1 OOI CI Functions Instrument and platform interfaces Instrument and platform control from the observatory Direct access to instruments and platforms Data acquisition from sensors and external observatories Management of science data and other observatory information Data inventory and metadata annotation Data ingestion, i.e. canonicalization and metadata annotation Data discovery, cataloging and presentation Data distribution, replication, streaming Data persistence, assurance, retrieval Process and service execution Virtualization of computing Elastic scaling to demand High availability services Service-orientation Reliable message based asynchronous communication Security: identity and policy management Multi facility system integration Sensing & Acquisition Data Management Common Operating Infrastructure Common Execution Infrastructure Interface Adapters Application Infrastructure

20. OOI CI LCO Review, Feb 2010 20 The Capability Container Ingredients: Capability: function, service, component Plumbing: Messaging, security Container: drop it all in as needed Packaging: label it, tape it shut… done!

21. OOI CI LCO Review, Feb 2010 21 Secure Reliable Messaging Capability Container Capability Container Capability Container Capability Container Capability Container Capability (Service) Capability (Service) Capability (Service) Capability (Service) Capability (Service)

22. OOI CI LCO Review, Feb 2010 22 The “Exchange” Applications communicate through Exchange Spaces Exchange Spaces are namespaces of “communicators” Applications need to enroll in Exchange Spaces Governance applies within Exchange Spaces

23. OOI CI LCO Review, Feb 2010 23 Exchange Space and Points Exchange Points are the message routing and queuing resources of Exchange Spaces Communicators play the role of Producer, Consumer, and Distributer (Broker)

24. OOI CI LCO Review, Feb 2010 24 Exchange Space Exchange Space is comprised of –Distributed Application Facility (DAF) –Distributed IPC Facility (DIF)

25. OOI CI LCO Review, Feb 2010 25 Messaging Abstraction Capability Container Capability Container Capability Container Send Message From: “name2” To: “name4” In: Exchg-Space1 Action: “invoke-service”

26. OOI CI LCO Review, Feb 2010 26 Common Message Format Capability Container –Provides message handling through interceptors After a service sends a message, before it receives a message Message signing and validation Policy enforcement Governance tracking –Provides a common message format for all CI messages Based in ACL FIPA message structure Content, encoding, ontology

27. OOI CI LCO Review, Feb 2010 27 Services and Resources as Agents Resource Agent Resource Agent Proxy Agent Capability Container Capability Container Capability Container Capability Container

28. OOI CI LCO Review, Feb 2010 28 Scenario An instrument, a physical resource, is represented by an agent to the system and its users Users request control of the instrument Capabilities are projected into another domain of authority by a proxy agent

29. OOI CI LCO Review, Feb 2010 29 Back-End Infrastructure

30. OOI CI LCO Review, Feb 2010 30 Exchange Points and the DIF

31. OOI CI LCO Review, Feb 2010 31 Message Brokers over DIF

32. DIF Member Process OOI CI LCO Review, Feb 2010 32

33. OOI CI LCO Review, Feb 2010 33 COI Infrastructure Services The Common Operating Infrastructure (COI) is the integration & communication environment for all the other subsystem services

34. OOI CI LCO Review, Feb 2010 34 Integration Strategy Capability Container Secure reliable messaging “Exchange” Service-oriented architecture Non-central governance COI Infrastructure Services

35. OOI CI LCO Review, Feb 2010 35 CI Interfaces and Subsystems

36. OOI CI LCO Review, Feb 2010 36 Preparing Services for Deployment Instrument Point Acquisition Point Ingest Point Application Point Storage Point Access Portal

37. OOI CI LCO Review, Feb 2010 37 Deployment Scenario

38. OOI CI LCO Review, Feb 2010 38 Deployment Plan and Strategy Deployment of capability containers in multiple execution environments by leveraging virtualization –The CEI will make it work CyberPoP (Cyberinfrastructure Point of Presence): physical deployment of one or multiple capability containers –Different types

39. OOI CI LCO Review, Feb 2010 39 Available, Elastic Execution

40. OOI CI LCO Review, Feb 2010 40 CEI Service Interactions Planner Controller (“Executive”) Provisioner Fault Monitoring and Compensation Registry Resource (Taskable) 2970-00001 OV2 CEI

41. OOI CI LCO Review, Feb 2010 41 Scenario: Planner and Controller The pattern Planner- Controller-Resource can repeat itself at multiple (nested levels) 2970-00018 OV2 CEI

42. OOI CI LCO Review, Feb 2010 42 CEI Architecture Overview OOI CI LCO Review, Feb 2010 42 EPU EPU Worker (Operational Unit) EPU Worker (Operational Unit) EPU Worker (Operational Unit) EPU Worker (Operational Unit) EPU Worker (Operational Unit) EPU Worker (Operational Unit) HA Service (OOI Application) VM (Deployable Unit) VM (Deployable Unit) VM (Deployable Unit) Application Software (Deployable Type)

43. OOI CI LCO Review, Feb 2010 43 CEI Architecture Overview OOI CI LCO Review, Feb 2010 43

44. OOI CI LCO Review, Feb 2010 44 CEI Service Interaction OOI CI LCO Review, Feb 2010 44 Planner ControllerProvisioner Monitoring Information Deployable Type Registry Context Broker IaaS AdapterIaaS Exchange VM info Proc Info Other Info

45. OOI CI LCO Review, Feb 2010 45 Elastic Processing Unit Elastic Processing Unit (EPU): High- Availability Services with policy based scaling to demand OOI CI LCO Review, Feb 2010 45 EPU Controller Register HA service EPU Worker N1 EPU Worker N1 Msg to N EPU Worker N2 EPU Worker N2 EPU Worker NX EPU Worker NX …

46. OOI CI LCO Review, Feb 2010 46 EPU: Prototype 1 EPU design and prototype: –Policy-configurable planner/scheduler –Scaling policy: size of work queue –All components integrated with Magnet/AMQP OOI CI LCO Review, Feb 2010 46

47. OOI CI LCO Review, Feb 2010 47 Governance

48. OOI CI LCO Review, Feb 2010 48 Motivating Governance Administering collaborations –Based on framing normative relationships among peers –Abstracting away from low-level details OOI, broadly: many stakeholders; many resources; longevity of decades Exchange spaces, narrowly: abstractions for communicating; assembly of multiple topologies for messaging; analogous to traditional enterprise integration patterns

49. OOI CI LCO Review, Feb 2010 49 Elements of a Service Engagement Enactment: doing the domain work – what the end user cares most about Administration: captured via contracts –Partnerships –Rules of encounter Identity Enforcement

50. OOI CI LCO Review, Feb 2010 50 What is Governance? Broadly, administering service engagements IT Governance: How IT resources are administered SOA Governance: How services are created, deployed, removed, … Currently, governance is manual –Low productivity –Poor scalability for fine-grained, real time governance decisions –Hidden, implicit considerations yield low confidence and poor maintainability

51. OOI CI LCO Review, Feb 2010 51 Why Governance? Stakeholders using resources to best serve individual and collective needs –Share resources in a controlled manner –Configure and reconfigure dynamically –Enable unanticipated uses for resources –Respect human organizational needs In a nutshell, stakeholders administer themselves

52. OOI CI LCO Review, Feb 2010 52 Separation of Concerns Protocol: specifying the interactions among autonomous parties Policy: specifying the decision making of each autonomous party as it participates in various protocols Behavior: specifying the implementation that realizes the interactions

53. OOI CI LCO Review, Feb 2010 53 Exchange Space Use Case OOI CI LCO Review, Feb 2010 53

54. OOI CI LCO Review, Feb 2010 54 Messaging View of Enrollment OOI CI LCO Review, Feb 2010 54

55. OOI CI LCO Review, Feb 2010 55 Community Affiliation Use Case OOI CI LCO Review, Feb 2010 55

56. OOI CI LCO Review, Feb 2010 56 Combined Scenario, Schematically OOI CI LCO Review, Feb 2010 56

57. OOI CI LCO Review, Feb 2010 57 IdM Architecture Overview

58. OOI CI LCO Review, Feb 2010 58 Conclusions

59. OOI CI LCO Review, Feb 2010 59 Summary

60. OOI CI LCO Review, Feb 2010 60 Next Steps

61. OOI CI LCO Review, Feb 2010 61 Thanks !