A New Generation of Data Services for Earth System Science Education and Research: Unidata’s Plans and Directions AGU Fall Meeting San Francisco, CA 6 December 2005 Dr. Mohan Ramamurthy Director, Unidata Program Center UCAR Office of Programs Boulder, CO

Science Drivers Environmental problems like global change & water cycle transcend disciplinary as well as geographic boundaries, requiring multidisciplinary approaches and global teams for solving them; Environmental problems like global change & water cycle transcend disciplinary as well as geographic boundaries, requiring multidisciplinary approaches and global teams for solving them; Rapid advances in observational technologies, especially in remote sensing; Rapid advances in observational technologies, especially in remote sensing; Increasing use of complex, coupled modeling systems; Increasing use of complex, coupled modeling systems; Research studies on societal impact of hurricane-related flooding involve integrating data from atmospheric sciences, oceanography, hydrology, geology, geography, and social sciences.

Science Drivers: Examples

Emergency Response Ensemble Predictions Coastal Environments GIS Integration End to End Information Services Need integrated services

Education Drivers A “holistic” Earth- system science approach to education A “holistic” Earth- system science approach to education Active, student- centered learning. i.e., learning science by doing science Active, student- centered learning. i.e., learning science by doing science Observations (data)Observations (data) Tools (models, visualization)Tools (models, visualization) DiscoveryDiscovery

Technology Trends Enabling a New Generation of Data Services Internet & the World Wide WebInternet & the World Wide Web Commodity microprocessorsCommodity microprocessors Object-oriented programmingObject-oriented programming Open standardsOpen standards Web servicesWeb services Extensible Markup Language (XML)Extensible Markup Language (XML) Global, high-bandwidth and wireless networksGlobal, high-bandwidth and wireless networks Digital librariesDigital libraries CollaboratoriesCollaboratories Grid Computing/e-ScienceGrid Computing/e-Science Data Portals and Federated, distributed ServersData Portals and Federated, distributed Servers Geographic Information SystemsGeographic Information Systems Knowledge environmentsKnowledge environments Ontologies and Semantic webOntologies and Semantic web Data mining and knowledge discoveryData mining and knowledge discovery

Data Services: An Evolution   An evolution from proprietary data systems towards more open standards-based data services – i.e. web services.   Data services should address the myriad applications and needs of the community: research, education, outreach, collaboration, etc.   This transition to web services poses many challenges.

Service-Oriented Science Source: Ian Foster, Science, 6 May 2005 Web Services are self-contained, self- describing, modular applications that can be published, located, and invoked across the Web. XML based Web Services are emerging as tools for creating next generation distributed systems that facilitate program-to-program interaction without the user-to-program interaction. Besides recognizing the heterogeneity as a fundamental ingredient, these web services, independent of platform and environment, can be packaged and published on the internet as they can communicate with other systems using the common protocols. Emerging web services standards are enabling much easier system-to-system integration.

Google Maps and Personal Weather Data How is this magic performed? Answer: DHTML, JavaScript, CSS, XML, and XSLT

Google Earth

A Partial List of Data Services Collection Collection Transport Transport Notification Notification Cataloging and metadata generation Cataloging and metadata generation Metadata submission Metadata submission Subsetting Subsetting Aggregation Aggregation Decoders/format converter Decoders/format converter Querying Querying Visualization Visualization Collaboration Collaboration Ontology Ontology Data mining Data mining Weblogging or blogging Weblogging or blogging …

End–To–End Data Service Development at Unidata IDD Data LDM THREDDS Catalog Browse Access THREDDS Data Server (TDS) THREDDS Data Repository (TDR) Put Data TDS Client Notify Application (e.g. IDV) Service Data Storage TDR Client Locally Generated Data Locally Generated Data

Internet Data Distribution Radar Model Satellite

File Formats Local/Remote Services Underlying Interfaces Primary Interfaces THREDDS catalog TDS (THREDDS Data Server Interface) OGC WCS (Web Coverage Server) NetCDF interfaceOPeNDAP OpenDAPADDE GRID Station AREA IOsp GRIB GINI NIDS netCDF via HTTP Jgoffs DMSP NetCDF TDS: A Collection of Services

TDR: Another View

Data Services for Education THREDDS Middleware For effective incorporation of data into digital libraries, we need a range of data services tailored for education:   integrate data, models, viz. tools with learning objects and other curricular materials

NCSA OU Unidata OU User running local analysis and display tools Data ServiceDecoder Service Assimilation Service Prediction Service User Orchestrates Web Services to Create Regional Forecast Product Generation & Mining Service LEAD: Data Services for NWP IU Orchestration Service UAH

Data Services for Field Projects

GALEON IE: Data services for GIS Integration

Concluding Remarks At Unidata, we are in the process of building many of these data-related services and technologies; At Unidata, we are in the process of building many of these data-related services and technologies; Despite the significant progress, much work still lies ahead; Despite the significant progress, much work still lies ahead; Other organizations and communities are engaged in similar exercises (e.g., NOAA, IOOS, and CUAHSI) ; Other organizations and communities are engaged in similar exercises (e.g., NOAA, IOOS, and CUAHSI) ; We will partner with those groups where necessary and leverage each other’s work; We will partner with those groups where necessary and leverage each other’s work; We are diligently building bridges with other communities for mutual benefit. We are diligently building bridges with other communities for mutual benefit.