1. OOI CI R2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 Ocean Observatories Initiative 1 OOI CI Overview and Status Matthew Arrott DMAC-ST Washington DC, Jan 18-19, 2012

2. OOI CI R2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 2 Agenda Review of the OOI CI Construction Objectives Introduction of Release 2 Functional Objectives Computational Infrastructure Objectives External Observatory Integration What will OOI-CI mean to me

3. OOI CI R2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 3 OOI CI Construction Objectives

4. OOI CI R2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 4 OOI Project Scope Building/deploying science infrastructure for understanding a changing water planet Multi Dimensional Engineering Concern Social Organizational Functional Procedural Technical Cyberinfrastructure fulfills the integrative mechanism across these dimensions

5. OOI CI R2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 5 The Hubble Telescope for the Oceans

6. OOI CI R2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 6 The Hubble Telescope for the Oceans

7. OOI CI Release 2 Kickoff May 2, 2011 7 Core Science & Education Capabilities Interactive Ocean Observing (R3 - R4) Interactive Ocean Modeling & Data Assimilation (R3 - R4) Discipline-Driven Semantic Organization of Data (R3 - R4) Automated Data Product Generation (R1 - R3) Interactive Instrument Network (R1 - R3) Integrated Observatory Management (R1 - R2) User-Driven Integration of Resource (R1 - R3) Observatory Activity Model

8. OOI CI R2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 8 OOI Integrated Observatory Product The fully operational research observatory will meet the following Goals: Continuous observations at time scales of seconds to decades Spatial measurements from millimeters to kilometers Sustained operation during storms and other severe conditions Real-time or near-real-time data as appropriate Two-way transmission of data and remote instrument control Power delivery to sensors between the sea surface and the seafloor Standard plug-n-play sensor interface protocol Autonomous underwater vehicle dock for data download/battery recharge Access to deployment and maintenance vehicles that satisfy the needs of specific observatories Facilities for instrument maintenance and calibration A management system that makes data publicly available An effective education and outreach program

9. OOI CI R2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 9 Extending the Traditional Data Access Model

10. 10 Multi-Scale Research Specific Ocean Laboratories

11. OOI Review Year 2 May 16-20, 2011 11 Release Schedule Inception Phase Elaboration Phase Construction Phase Transition Phase

12. OOI Review Year 2 May 16-20, 2011 12 Release Schedule by Subsystem Inception Phase Elaboration Phase Construction Phase Transition Phase

13. OOI Review Year 2 May 16-20, 2011 13 CI Components Developed for R1 Python Capability Container Java Capability Container Access Library Web UI Platform Exchange Messaging System Distributed State Infrastructure Resource Registry Framework Science Data Persistence and Transport Format Data Publish-Subscribe Framework Event Notification Framework Instrument Agent Framework Elastic Processing Unit Virtualized Cloud Management Tools

14. OOI Review Year 2 May 16-20, 2011 14 Capabilities of the Managed Instrument Network Operate Marine Observatories Operate Platforms and Instruments Manage Instrument Lifecycle Test and Troubleshoot Instruments Acquire Data and Generate Data Products Search Data Visualize Data Manage the Integrated Observatory Network

15. OOI Review Year 2 May 16-20, 2011 15 R1 Integrated Observatory Network Web UI https://confluence.oceanobservatories.org/display/CIPUB/OOI+Review+2011+Demo s

16. OOI CI R2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 16 Release 2 Functional Objectives

17. OOI CI R2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 17 Observatory Operations & Administration

18. OOI Review Year 2 May 16-20, 2011 18 CI Release-1: Data Distribution Network Provide framework for discovery and interaction based on data-intensive users’ needs CI Release-2: Managed Instrument Network Provide support for operations and maintenance of marine networks based on RSN and CGSN needs CI Release-3: On Demand Measurement Processing Provide dynamic analysis and visualization tools based on science communities’ user needs CI Release-4: Interactive Ocean Observatory Provide mission control and collaboration support based on science communities’ needs User Experience Incremental Development Plan Scientific Collaboration General Science Users Science Experts Resource Providers

19. OOI Review Year 2 May 16-20, 2011 19 UX R2 Product Description: List of Use Cases UC.R2.01 Acquire External Data SourceUC.R2.02 Derive Data ProductUC.R2.03 Produce Real-Time QC DataUC.R2.04 Browse to Get Data ProductUC.R2.05 Register and Connect InstrumentUC.R2.06 Command InstrumentsUC.R2.07 Direct Instrument Access IIUC.R2.08 Manage Instrument LifecycleUC.R2.09 Activate Instrument DriverUC.R2.10 Manage Marine PlatformUC.R2.11 Operate Marine ObservatoryUC.R2.12 Deploy Agents On Remote PlatformUC.R2.13 Acquire Data From InstrumentUC.R2.14 Monitor an InstrumentUC.R2.15 Qualify Instrument InterfaceUC.R2.16 Install Instrument Automatically UC.R2.17 Define Visualization MethodUC.R2.18 Visualize Data ProductUC.R2.19 Produce Matlab Visualization UC.R2.20 Annotate ResourcesUC.R2.21 Transform Data in WorkflowUC.R2.22 Version ResourceUC.R2.23 Ingest Dataset SupplementUC.R2.24 Search for ResourceUC.R2.25 Advanced Resource SearchUC.R2.26 Navigate Resources and MetadataUC.R2.27 Manage Replicated ArchiveUC.R2.28 Manage Resource MetadataUC.R2.29 Integrate External Data Source UC.R2.30 Define InteractionUC.R2.31 Define New ServiceUC.R2.32 Conduct NegotiationUC.R2.33 Enroll in an OrgUC.R2.34 Share an Org ResourceUC.R2.35 Share Affiliated Orgs' ResourcesUC.R2.36 Create an OrgUC.R2.37 Control Service InteractionsUC.R2.38 Define Resource Life CycleUC.R2.39 Manage ION UsersUC.R2.40 Monitor ION ResourcesUC.R2.41 Recover Failed ProcessUC.R2.42 Define Resource PolicyUC.R2.43 Operate Message Brokers UC.R2.44 Put Services Anywhere EasilyUC.R2.45 Replicate Activated ServiceUC.R2.46 Operate Integrated SystemUC.R2.47 Deploy Versioned User ProcessUC.R2.48 Schedule User-Defined ProcessUC.R2.49 Deploy Distributed ProcessesUC.R2.50 Define Scaling PolicyUC.R2.51 Define Execution EngineUC.R2.52 Manage ION Processes UC.R2.53 View Modeler-Submitted ProductsUC.R2.54 Access NEPTUNE CA Data UC.R2.55 Manage Help TicketUC.R2.56 Monitor ION NetworkUC.R2.57 Configure Start Page UC.R2.58 Display Arbitrary ResourceUC.R2.59 Generate New Screen DM S&A A&S COI CEI EOI Ops

20. OOI CI R2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 20 Managed Instrument Network

21. OOI Review Year 2 May 16-20, 2011 21 Sensor Set #1 Instrument Agents Class#FamilyInstrument Class Long Name ADCPS16Water VelocityVelocity_profile_600m CTDGV24Conductivity, Temperature, DepthCTD_glider CTDMO135Conductivity, Temperature, DepthCTD_mooring CTDPF30Conductivity, Temperature, DepthCTD_profiler DOSTA61Dissolved Oxygenoxygen_dissolved_stable FDCHP7Direct Covariance Fluxflux_direct_cov_HP FLORD35FluorometerFluorometer_two_wavelength METBK18MeteorologyMeteorology_bulk NUTNR22Nitratenutrient_Nitrate OPTAA35Optical Propertiesattenuation_absorption_optical PCO2A10Partial Pressure CO2pCO2_air-sea PCO2W25Partial Pressure CO2pCO2_water PHSEN33Partial Pressure CO2pH_stable PREST3Seafloor Pressurepressure_SF_tidal SPKIR22Optical Propertiesspectral_irradiance VEL3D32Water VelocityVelocity_point_3D_turb VELPT21Water VelocityVelocity_point WAVSS10Surface Wave Spectrawave_spectra_surface ZPLSG4Zooplankton Sonarplankton_ZP_sonar_global

22. OOI Review Year 2 May 16-20, 2011 22 Sensor Set #2 Instrument Agents Class#FamilyInstrument Class Long Name BOTPT3Seafloor Pressurepressure_bottom_tilt CAMDS12Cameras xcamera_digital_still_strobe CAMHD1Cameras xcamera_digital_video_HD HYDLF5HydrophoneHydrophone_LF_passive MASSP2Mass Spectrometermass_spectrometer OBSBB3Ocean Bottom SeismometerSeismometer_BB_triaxial_accel OBSBK2Ocean Bottom SeismometerSeismometer_BB_triaxial_keck OBSSP8Ocean Bottom SeismometerSeismometer_shortperiod PPSDN1Seafloor Fluid and DNADNA_particulate RASFL1Seafloor Fluid and DNAwatersample_chem_trace_H2S_pH THSPH1Seafloor PropertiesTemp_H2_H2S_pH TMPSF1Seafloor PropertiesTemperature_seafloor TRHPH1Seafloor PropertiesTemp_resist

23. OOI Review Year 2 May 16-20, 2011 23 Sensor Set #3 Instrument Agents Class#FamilyInstrument Class Long Name ADCPA15Water Velocityvelocity_profile_mobile_asset ADCPT17Water VelocityVelocity_profile_300m CTDAV3Conductivity, Temperature, DepthCTD_AUV CTDBP18Conductivity, Temperature, DepthCTD_bottom_pumped DOFST22Dissolved Oxygenoxygen_dissolved_fastresp FDCLP3Direct Covariance Fluxflux_direct_cov_LP FLORT42FluorometerFluorometer_three_wavelength HPIES2Inverted Echo SounderIES_pressure_velocity HYDBB6HydrophoneHydrophone_BB_passive NUTR43Nutrientsnutrient_four_channel OSMOI2Seafloor Propertieswatersample_chem_trace OTISF1Seafloor Propertiesflow_benthic PARAD30Optical PropertiesPAR PRESF9Seafloor Pressurepressure_SF VADCP2Water VelocityVelocity_profile_50m_turb ZPLSC8Zooplankton Sonarplankton_ZP_sonar_coastal

24. OOI Review Year 2 May 16-20, 2011 24 CI Architecture for Instrument Integration

25. 25 Data Processing and Product Generation

26. OOI Review Year 2 May 16-20, 2011 26 COL OOI's Sensor Life Cycle OOI Final System Design Core Sensor List Core Sensor Specs Sensor Working Group Core Sensor RFPs COL CGSNRSN Common Sensor Selections COL CGSNRSN CI Procure Common Sensors CGSNRSN Develop QC/Xform Algorithms CGSNRSN Develop Sensor Drivers CI Integrate w/ Sensor Agent Test w/Instrumen t Test Kit CI Integration Test System Integration Test CI Encode Algorithms CI Deploy at Sea RSNCGSN

27. OOI Review Year 2 May 16-20, 2011 27 Instrument Development Kit Scheduled for Release-2 Includes: Logical Test Facility Workbench (dry testing) Marine Specific System Test Facility (wet testing) Configurations for RSN and CGSN observatories Access and Management Portals for Interactive access

28. OOI CI R2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 28 Computational Infrastructure Objectives

29. OOI Review Year 2 May 16-20, 2011 29 National Network Deployment (Year 3)

30. OOI Review Year 2 May 16-20, 2011 30 CyberPoPs and Network Infrastructure Acquisition Point CyberPoPs: Portland, OR Woods Hole, MA Distribution Point CyberPoPs: Seattle, WA Chicago, IL San Diego, CA McLean, VA (optional) Engineering Center: San Diego, CA Network Infrastructure Dedicated 10GE loop San Diego, Portland, Seattle, Chicago; branch to McLean (optional) Dedicated 1GE connection to Woods Hole

31. OOI Review Year 2 May 16-20, 2011 31 San Diego Engineering Center Purchased and deployed San Diego CyberPoP and network equipment (see rack drawing) Located in UCSD’s Atkinson Hall server room 1101 (secure and protected) In use for Release-1 production and QA Management tools (Confluence, Jira, etc)

32. OOI Review Year 2 May 16-20, 2011 32 San Diego CyberPoP and Network Equipment

33. OOI Review Year 2 May 16-20, 2011 33 Acquisition Point CyberPoP Deployments (Y3) Sites Portland Woods Hole Function Data acquisition from marine observatories Real-time data processing and OOI data product generation Data preservation High availability

34. OOI Review Year 2 May 16-20, 2011 34 CyberPoP Physical Layout (Year 3 Deployments) Sites Seattle Chicago Function Content distribution (web servers, data servers, messaging) Interconnects with major national and international network providers (layer 2 peering) Links to compute clouds

35. OOI CI R2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 35 Network Peering Integration ChicagoSeattle

36. OOI CI R2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 36 Multi-Tier Messaging Federation

37. 37 Multi-Site Service Network Deployment

38. OOI CI R2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 38 External Observatory Integration

39. OOI CI R2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 39 IOOS Integration Model

40. OOI Review Year 2 May 16-20, 2011 40 External Observatory Integration Progress Identification of datasets and data sources needed to support early adopter group (ESPRESSO/Rutgers, HiOOS/UHawaii) numerical model workflows Classification of datasets by type and representation Development of Dataset Agents for automatic data and metadata ingestion Translation of data/metadata into the Common Science Data Format Streaming of data packets and metadata update notifications Integration of real-time data sources in early-adopter on- site numerical workflow generation processes, using the Integrated Observatory Network Release-1

41. OOI Review Year 2 May 16-20, 2011 41 External Observatory Integration Exemplar: MARACOOS (Rutgers University) Experimentally substitute part of the scientists’ data assimilation workflow preceding numerical model execution

42. OOI Review Year 2 May 16-20, 2011 42 External Observatory Integration: Master Dataset List Datasets NAVY : NRLSSC (DAP) : 1 NOAA : NDBC SOS (HTML) : 34 NOAA : NDBC SDF (DAP) : 1 NOAA : PFEG (DAP) : 1 Rutgers : tashtego (DAP) : 2 UCSD : HFRNET (DAP) : 1 UH : SOEST (DAP) : 1 USGS : WaterService (HTML) : 26

43. OOI CI R2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 43 “What will OOI-CI mean to me”

44. OOI CI Release 2 Kickoff May 2, 2011 44 Regional Data Provider & Manager Effective mechanism to reliably at scale distribute your data to your user community and data management/preservation facility Independently publish and receive data in a wide variety of syntactic formats and community vocabularies Publish and receive as near-real-time data streams and as historical data sets Effective mechanism to reliably at scale aggregate and disseminate data from your publishers and to your end user communities

45. OOI CI Release 2 Kickoff May 2, 2011 45 Participating Agency Participate as providers, managers and consumers Anticipated some requirements for publishing on the OOI as a separate Agency will require coordination with NSF Requirements are being addressed as a part of the IOOS OOI Integration effort

46. OOI CI Release 2 Kickoff May 2, 2011 46 How does one “join” – approval / certification? Open data policy network for consumers of data Anticipate publishers required to register and certified Data policies and rights management being established Open data policies and content limitations to science and education data content Publishing requires acceptance of some form of open use policy Consuming requires some form of “as is”; “at your on risk”; “with attribution of source if republished” policies

47. OOI CI Release 2 Kickoff May 2, 2011 47 Relationship with IOOS, NEPTUNE and others observatories External Observatory Integration subsystem within the OOI Specifically focused on providing bi-directional integration interfaces to IOOS, NEPTUNE-Canada and WMO (aka GTS) Integration continues on into the Operations and Maintenance phase of the OOI as a user/observatory “on boarding” capability Provide the training and support to communities that wish to integrate with OOI

48. OOI CI Release 2 Kickoff May 2, 2011 48 OOI define standards for them to “plug in”?, e.g SensorML OOI provides a wide variety of syntactic formats and community vocabularies by which to interface with the OOI OOI and the user community will be able to extend these interfaces Capability facilitated by architectural choice of using controlled set of canonical data models into and out of which all data are transformed New transformations are added, “plug ins”, to the network for a new format and/or vocabulary to one of the controlled data models

49. OOI CI Release 2 Kickoff May 2, 2011 49 Will a region be able to host an instance of the OOI “stack”? The OOI “Integrated Observatory Network” is a federation architecture Comprising participants from multiple domains of authority operating from their local domain of authority The OOI “stack” exists at a couple of levels: It is a messaging protocol It is a component software implementation that implements the messaging protocol and process management containers (think modern web server) that publish and consume services “Capability Containers”, can be and are written in multiple languages achieve their interoperability through use of the common OOI messaging protocol OOI message protocol and any of the OOI supplied Capability Containers will be provided under one of the standard open source licenses

50. OOI CI R2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 50 Thanks!