1. OOI CI LCA REVIEW August 2010 Ocean Observatories Initiative External Observatory Integration: Christopher Mueller Life Cycle Objectives Review La Jolla, CA

2. OOI CI LCA REVIEW August 2010 Agenda Strategy Subsystem Purpose Release 1 Product Description Use Case Overview Architectural Overview Status of Progress Use Cases Address, Use Cases Demonstrated Technology Challenges and Achievements Plan for Elaboration Significant Risks

3. OOI CI LCA REVIEW August 2010 Agenda Strategy Subsystem Purpose Release 1 Product Description Use Case Overview Architectural Overview Status of Progress Use Cases Address, Use Cases Demonstrated Technology Challenges and Achievements Plan for Elaboration Significant Risks

4. OOI CI LCA REVIEW August 2010 OOI-CI – IOOS DMAC Strategy OOI Cyberinfrastructure (CI): a transformational integrated observing infrastructure, providing a link between ocean research and discovery Integrated Ocean Observing System: a federal, regional, and private-sector partnership working to enhance our ability to collect, deliver, and use ocean information IOOS Data Management and Communications (DMAC) Subsystem : a subsystem capable of delivering real-time, delayed-mode, and historical data for in-situ and remotely-sensed physical, chemical and biological observations, as well as, model-generated outputs, including forecasts, to IOOS® users and of delivering all forms of information to and from secure archive facilities. (http://ioos.gov/library/dmac_implementation_2010.pdf) What is our strategy for aligning OOI and IOOS? "OOI is NSF's contribution to IOOS” (then) (now) “One cyberinfrastructure (implementation) shared between two organizations” (now)

5. OOI CI LCA REVIEW August 2010 External Observatory Integration Subsystem: Purpose 1. integrates external observatories (i.e. IOOS) with the OOI Integrated Observatory 2. facilitates inclusion of external data 3. delivers data in community specific forms 4. allows immediate full scale testing of the OOI-CI System 5. introduces OOI to a community of early adopters

6. OOI CI LCA REVIEW August 2010 Release 1 Product Description Use Case Overview IDTitleDescription UC.R1.21Merge Data *Provide data unified from many sources UC.R1.23Put to External Repository *Send data to special-purpose repository IDTitleDescription UC.R1.04Ingest and Describe DataExternally Provided Data read and distributed UC.R1.22Present as CatalogPresent organized set of resources externally IDTitleDescription UC.R1.03Hello Data SourceData source is registered and connected UC.R1.06Distribute Data ProductData made available to many consumers UC.R1.07Subscribe to DataUser finds data, asks for update notifications UC.R1.13Transform DataData process produces new data from old UC.R1.24Version a ResourceResource is supplanted by changed version UC.R1.33Assert Access PolicyDefine access policy for a given resource * May be out of scope for R1 Responsible For Supports Uses

7. OOI CI LCA REVIEW August 2010 Release 1 Product Description Use Case Overview Scenario for "Merge Data" Use Case 1. Scientist creates algorithm to create unified dataset from multiple data sources, one or more algorithm(s), and perhaps server-side processing (e.g., Ferret, GridFields). 2. Scientist registers for notifications of new available data for data sets 3. Scientist defines a new data set for the algorithm's output 4) Scientist defines new data process for the algorithm (Define New Service) 4. Scientist receives notification of new available data and executes the algorithm 5. The Integrated Observing Network ingests the new data packet 6. Output from algorithm is created as any other new data. Will Replace with graphical representation

8. OOI CI LCA REVIEW August 2010 Release 1 Product Description Use Case Overview Scenario for ”Put to External Repository" Use Case 1. An organized external repository for data of particular type(s) is identified 2. An adapter process is developed that interfaces with the external repository 3. The adapter process subscribes to data of interest 4. Services to Transform Data to the format required by the custom repository are set up and enabled 5. Data and metadata, in the necessary format, arrive recurringly at the adapter process, which makes them available to the external repository 6. Any final transformations relative to the Repository's structure are performed by the external Repository Will Replace with graphical representation

9. OOI CI LCA REVIEW August 2010 Architectural Overview Researcher’s Process OOI-CI Process Apply CF conventions LDM THREDDS Catalog CDM IDD GTS NOAA Port NetCDF Java Library Model Analysis NDBC: Observing System Monitoring Center (OSMC) PULL Unidata Or NDBC 1 2 3 4 NCO & User Scripts PULL DAP & CF Matlab (test CDM) 5 Navy/NASA/Other Model output 1A 2A CI-DMAC_project_20100128.doc OOI-CI & IOOS-DMAC Intersection Development Project- FY2010 *Improve the process by which researchers get data to drive models

10. OOI CI LCA REVIEW August 2010 Architectural Overview http://www.oceanobservatories.org/spaces/display/syseng/CIAD+OV+04+External+Interfaces

11. OOI CI LCA REVIEW August 2010 Architectural Overview External Observatories are parallel to our concept of Marine Observatories Exist as data stores with a interaction mechanisms External Observator y InstrumentPlatform OOI Marine Observator y IOOSWMONeptun e CA Realized Operational Specification http://www.oceanobservatories.org/spaces/display/syseng/CIAD+OV+04+External+Interfaces

12. OOI CI LCA REVIEW August 2010 Status of Progress Established the Rutgers modeling team as the initial scientific “client” Started developing interaction patterns between OOI-CI, IOOS, & Scientists Compiled lists of data sources, outputs and workflows required by the Rutgers modeling team

13. OOI CI LCA REVIEW August 2010 Status of Progress – List of Rutgers Data Sources Providers: PFEG, Rutgers, NAVY, USGS, RADS, NCEP, AOML Acquisition Protocols: OPeNDAP, curl, wget, urlread Prospective CDM Types: Grid, Station, Trajectory, Profile Data Variables: Sea Surface Height, Currents, Winds, Salinity, Temperature, River Discharge, … Acquisition Forms: NetCDF, ASCII

14. OOI CI LCA REVIEW August 2010 EOI Technology List IDD/LDM Matlab NetCDF Java

15. OOI CI LCA REVIEW August 2010 Technology Challenges and Achievements Achievements: Functional prototypes of NetCDF  DAP translation in both JAVA & Python Challenges: Potential structural differences between DAP objects generated by pyDAP and Java OPeNDAP libraries

16. OOI CI LCA REVIEW August 2010 Plan for Elaboration Continue to work with selected scientific modeling groups Register each data source as a Provider in the OOI-CI System Translation/transformation to achieve CDM/CF compliance Leverage IOOS data sources wherever possible Work with scientists to utilize the OOI-CI System for acquiring data Develop client-side tool that receives OOI-CI syndication notifications to fetch data and start a science user process locally

17. OOI CI LCA REVIEW August 2010 Use Cases at Risk for Release 1 IDTitleDescription UC.R1.21Merge DataProvide data unified from many sources

18. OOI CI LCA REVIEW August 2010 EOI Technologies at Risk for Release 1 None at this time

19. OOI CI LCA REVIEW August 2010 Thanks ! Questions ?