Ocean Observatories Initiative Serving Ocean Model Data on the Cloud M. Meisinger, C. Farcas, E. Farcas, C. Alexander, M. Arrott, J. de La Beaujardière, P. Hubbard, R. Mendelssohn, R. Signell IEEE/MTS Oceans09 Biloxi October 29, 2009

Outline OOI Cyberinfrastructure Overview OOI and IOOS/NOAA DIF Collaboration ERDDAP/DDN Prototype Enabling Technologies –Deployment in the Cloud –Message-based communication Data Exchange Prototype Benefits and Summary

OOI Overview

Science Activity Model OOI Scientific Investigation

OOI CI Data Management Service-Oriented Architecture Message-based Communication (via the Exchange)

OOI – IOOS/DIF Collaboration Collaboration Projects –Data Distribution Network Prototype (ERDDAP) –Data Exchange Prototype Duration –ERDDAP/DDN Prototype: Sept 2008 – Jan 2009 –Data Exchange: March 2009 – March 2010 Data Exchange Purpose –Create an operational, community usable “Data Exchange” prototype combining a dataset catalog and server-side processing engine as infrastructure –Deploy on elastic resources in the Amazon cloud

ERDDAP Prototype

OOI CI Messaging Scenario

What is the Cloud? Infrastructure as a Service –Compute Nodes –Storage –Present resources as infinite Advantages –No capitalization of compute infrastructure –No operations of compute infrastructure –Very useful in development phase and during peak load –Easy to use, proven, quick turn around Disadvantages –Higher costs during the operations phase –Reduced control of data, location, authority, security etc. Existing Clouds –Commercial clouds: “Pay per use” and “allocation” model –Examples: Amazon EC2/EBS, Microsoft Azure, Google AppEngine etc.

Cloud Deployment Architecture

Could Provisioning

Data Exchange Screenshots

Data Exchange

Prototype Technologies ERDDAP (NOAA) –Data transformation and distribution web frontend F-TDS (Ferret THREDDS data server) –THREDDS: Dataset catalog and OpenDAP server –Ferret: Server-side processing extensions for THREDDS for structured grids GridFields library –Manipulation of unstructured grids (server-side) MatLab –End user client application accessing cloud based F-TDS –Execution environment for server-side processing Amazon Elastic Cloud (EC2) –Cloud execution environment

Benefits Data Exchange Infrastructure –Promotes standards based interoperability –Ready to use data distribution infrastructure for ocean modeling communities –Provides adoption strategy for OOI and DIF –Operational test platform Platform to test web strategies and encodings Platform to test cloud computing technologies Cloud Deployment –Horizontal and vertical scaling on demand –Easier to deploy large-scale distributed applications Pathway to future OOI – IOOS collaboration

Thanks!

Science & Education Operations Interactive Ocean Observing Interactive Ocean Modeling & Data Assimilation Automated Data Product Generation Discipline-Driven Semantic Organization of Data Interactive Instrument Network Integrated Observatory Management User-Driven Integration of Resource Observatory Activity Model

IOOS Data Integration Framework

Data Processing and Availability