Architecture and Technologies for an Agile, User-Oriented Air Quality Data System Rudolf B. Husar Washington University, St. Louis Presented at the workshop The User and the GEOSS Architecture Applications for North America July 30, 2006, Denver Outline Highlight Trends of Air Quality Sensing and Management Describe an Agile IS Architecture for Air Quality Decision Support Show Their Application Through Two Use Cases Smoke Event ppt, flashpptflash AQ Policy ppt, flashpptflash

Changes in Air Quality Management Command & Control Weight of Evidence Flexible NAAMS Rigid Monitoring

Real-time Air Pollution Sensing and Reporting High Resolution Satellite DataSurface PM25 and Ozone Data Smoke Plumes

Generic Decision Support for Air Quality Decisions GEOSS Architecture Framework Knowledge into the Minds of Regulatory Analysts Knowledge into the Minds of Technical Analysts Observations Reports: Model Forecasts, Obs. Evidence Models Decisions Knowledge into the Minds of Decision- making managers Decision Support System

Key Technical Challenge: Characterization Pollutant characterization requires many different instruments and analysis tools. Each sensor/network covers only a fraction of the 6-8 dimensional data space. Other sensors provide only integral measures of the pollution, e.g. satellite - vertical integral. Satellite-Integral

Data are distributed geographically by autonomous providers Emission Ambient Satellite Model EPA NOAA NASA Other Content | Agency | Form Data includes emissions Emission Ambient Satellite Model EPA NOAA NASA Other Content | Agency | Form Information Providers: Geography, Content, Agency, Form Data includes emissions, ambient data, Ambient Emission Ambient Satellite Model EPA NOAA NASA Other Content | Agency | Form Data includes emissions, ambient data, satellite data Satellite Ambient Emission Ambient Satellite Model EPA NOAA NASA Other Content | Agency | Form Data includes emissions, ambient data, satellite data and model output Model Satellite Ambient Emission Ambient Satellite Model EPA NOAA NASA Other Content | Agency | Form Data are provided by multiple agencies: EPA, NOAA, NASA and others NASA Mission NOAA GASP NASA IDEA NASA DAACs NOAA ASOS EPA-AQS DataMart EPA AIRNow RPO VIEWS FS FireInv State/Local Emission EPA NEISGEI EPA NEI NOAA WeaMod EPA AQModel NOAA Forecast Emission Ambient Satellite Model EPA NOAA NASA Other Content | Agency | Form NASA DAACs NOAA GASP NASA IDEA NASA Missions EPA NEI EPA NEISGEI FS FireInv State/Local Emission NOAA ASOS RPO VIEWS EPA AIRNow EPA-AQS AIRS NOAA WeaMod EPA AQModel NASA GloModel NOAA Forecast Furthermore, data are provided in varied formats and access protocols Emission Ambient Satellite Model EPA NOAA NASA Other Content | Agency | Form Data on Internet are geography-independent and can be ‘linearized’ Internet NASA DAACs EPA R&D Model EPA AIRNow others

Users are distributed geographically EPA NOAA NASA Other Stakeholder | Agency | Form Policy Manager Public Scientist EPA NOAA NASA Other Stakeholder | Agency | Form Policy Manager Public Scientist Policy Users includes policy makers EPA NOAA NASA Other Stakeholder | Agency | Form Policy Manager Public Scientist Users includes policy makers, the public Policy Public EPA NOAA NASA Other Stakeholder | Agency | Form Policy Manager Public Scientist Users includes policy makers, the public, AQ managers Policy Public Manager EPA NOAA NASA Other Stakeholder | Agency | Form Policy Manager Public Scientist and scientist Policy Public Manager Scientist EPA NOAA NASA Other Stakeholder | Agency | Form Policy Manager Public Scientist Users are affiliated with multiple agencies: EPA, NOAA, NASA, as well as others Policy Public Manager Scientist EPA NOAA NASA Other Stakeholder | Agency | Form Policy Manager Public Scientist Users: By Types, Agency, Info Needs Furthermore, users need various types of information provided in multiple formats Policy Manager Policy Scientist ManagerScientist Policy Public EPA NOAA NASA Other Stakeholder | Agency | Form Policy Manager Public Scientist Since the users are also on the Internet, their geographic location is irrelevant Public Manager Scientist Internet other

The data life cycle consists of the acquisition and the usage parts Usage ActivitiesData Acquisition Data Acquisition and Usage Activities (Select View Show, click to step through PPT) The acquisition part processes the sensory data by firmly linked procedures The focus is on data usage activities The usage activities are more iterative, dynamic procedures The collected and cleaned data are stored in the repository Data Repository The usage cycle transform data into knowledge for decision making Decisions

Scientist Science DAACs Current info systems are project/program oriented and provide end-to-end solutions Info UsersData ProvidersInfo System AIRNow Public AIRNow Model Compliance Manager ‘Stovepipe’ and Federated Usage Architectures Landscape Part of the data resources of any project can be shared for re-use through DataFed Through the Federation, the data are homogenized into multi-dimensional cubes Data processing and rendering can then be performed through web services Each project/program can be augmented by Federation data and services

The Network Effect: Less Cost, More Benefits through Data Multi-Use Program Public Data Organization Data Program Organization Data Program Data Orgs Develop Programs Programs ask/get Data Public sets up Orgs Pay only once Richer content Data Re-Use Network Effect Data are costly resource – should be reused (recycled) for multiple applications Data Reuse Less Prog. Cost More Knowledge Data reuse saves $$ to programs and allows richer knowledge creation Less Soc. Cost More Soc. Benefit Data reuse, like recycling takes some effort: labeling, organizing, distributing

Providers NASA DAACs EPA R&D Model EPA AIRNow others Public Manager Scientist Users other The info system transforms the data into info products for each user In the first stage the heterogeneous data are prepared for uniform access Uniform Access Agile Information System: Data Access, Processing and Products The second stage performs filtering, aggregation, fusion and other operations Data Processing Web Service Chain Custom Processing SciFlo DataFed Info Products Reports, Websites Forecasting Compliance Other Sci. Reports The third stage prepares and delivers the needed info products

Decision Support System Event Knowledge into the Minds of EPA Analysts Knowledge into the Minds of State Analysts DSS for Exceptional Event Decisionsapping of Observations Event Reports: Model Forecasts, Obs. Evidence Models Decisions Event Knowledge into the Minds of EPA Regulators Decision Support System Data Sharing Std. Interface Data Obs. & Models Characterization Std. Interface ReportingDomain Processing Control Reports

Stages of AQ Data Flow and Value-Adding Processes Domain ProcessingData Sharing Std. Interface Gen. Processing Std. Interface Data Control Reports Reporting Obs. & ModelsDecision Support System Analyzing Filter/Integrate Aggregate/Fuse Custom Analysis Organizing Document Structure/Format Interfacing Characterizing Display/Browse Compare/Fuse Characterize Value-Adding Processes Reporting Inclusiveness Iterative/Agile Dynamic Report

Loosely Coupled Data Access through Standard Protocols The next three slides describe the key technologies used in the creation of an adaptable and responsive air quality information system. OGC data access protocols and standard formats facilitate loose coupling between data on the internet and processing services. For air quality, the Web Coverage Service (WCS), provides a universal simple query language for requesting data as where, when, what. That is: geographic (3D bounding box), time range and parameter. The Web Map Service (WMS) and Web Feature Service (WFS) are also useful. The use of standard data physical data formats and naming conventions elevates the syntactic and semantic interoperability. Within DataFed all data access services are implemented as WCS or WMS and optionally WFS. General format adapter components permit data request in a variety of standard formats. GetCapabilities GetData Capabilities, ‘Profile’ Data Where? When? What? Which Format? Server Back End Std. Interface Client Front End Std. Interface QueryGetData Standards Where?BBOXOGC, ISO When?TimeOGC, ISO What?TemperatureCF FormatnetCDF, HDF..CF, EOS, OGC T2T1 Domain ProcessingData Sharing Std. Interface Gen. Processing Std. Interface Data Control Reports Reporting Obs. & ModelsDecision Support System

Web Services and Workflow for Loose Coupling Service Broker Service Provider Publish Find Bind Service User Web Service Interaction Service Chaining & Workflow Domain ProcessingData Sharing Std. Interface Gen. Processing Std. Interface Data Control Reports Reporting Obs. & ModelsDecision Support System Web Services Triad: Publish – Find – Bind Workflow Software: Dynamic Programming

Collaborative Reporting and Dynamic Delivery Co Writing - Wiki ScreenCast Analysis Reports: Information supplied by many Needs continuous program feedback Report needs many authors Wiki technologies are for collaborative writing Dynamic Delivery: Much of the content is dynamic Animated presentations are compelling Movies and screencasts are for dynamic delivery Domain ProcessingData Sharing Std. Interface Gen. Processing Std. Interface Data Control Reports Reporting Obs. & ModelsDecision Support System

Summary The current challenges for air quality information systems include data delivery in real time, pollution characterization through the integration of multi-sensory data and providing agile support to regulatory management. The talk describes the architecture and implementation of a standards based system for accessing and processing air quality data to satisfy the above needs. The agile federated data system, DataFed, system has been in use for science and management support since DataFedis composed of distributed data and web services and integrated by user-configurable workflow software.The use of DataFed is illustrated through two use cases: (1) A real time monitoring example of a smoke event uses surface,satellite and model forecast information to inform air quality managers and the public. (2) Hemispheric aerosol transport model is compared to surface monitoring data to estimate the uncertainty and to improve themodel estimates.

Links PowerpointFlash Background800800/ DataFed Architecture800800/ Smoke Event pptppt800800/ AQ Policy pptppt800800/

Acknowledgements The presentation on Air Quality Background and Information Architecture benefited greatly from ideas, and challenges posed by a number of experienced individuals, from EPA (Rich Scheffe, Steve Young, Terry Keating), NASA (Lawrence Friedl, Kathy Fontaine). The participation in the NASA Information Technology Infusion workgroup (Karen Moe, Bran Wilson, Liping Di and others) was an intense collective learning experience. At CAPITA, Kari Hoijarvi engineered and implemented DataFed; Stefan Falke contributed datasets and application software. This presentation was prepared with help from Erin Robinson.