Integrated Air Quality Information System: Challenges Posed by Key Community Members EPA Rich Scheffe, EPA OAQPS Steve Young, EPA OEI Terry Keating, EPA ORD NASA L. Friedl, K. Fontaine, NASA HQ Frank Lindsey, NASA HQ WMO/IGOS Len Barrie, WMO GEOSS

Scheffe Challenge

GEOSS Eco-informatics Accountability/ indicators SIPs, nat.rules designations PHASE PM research Risk/exposure assessments AQ forecasting Programs NAAQS setting EPA NOAA NASA NPS USDA DOE Private Sector States/Tribes RPO’s/Interstate Academia NARSTO NAS, CAAAC CASAC, OMB Enviros Organizations CDC Supersites IMPROVE, NCore PM monit, PAMS CASTNET Lidar systems NADPSatellite data Intensive studies PM centers Other networks: SEARCH, IADN.. Data sources CMAQ GEOS-CHEM Emissions Meteorology Health/mort. records The Scheffe Challenge: ‘Organizations - Programs – Data Mess’ Info System Challenges: What’s the overall dependency Information Flow Forces and Controls on Data Flow Cooperation, Competition, Co-Opetition Information System of Systems

Air Quality Management System: Components and Functions Public Analyzing Interpreting Evaluating Separating Synthesizing Organizing Quality control Formatting Documenting Displaying Deciding Evaluate options Matching goals Compromising Choosing Data Manager, Organizer Technical Analysts, Program Manager Policy Analysts, Decision Maker Value Adding Processes Human Agents Decision Support System (DSS) The primary purpose of data systems is to serve programs/projects Programs perform analysis for Orgs., the DSS is within programs The big decisions of societal importance are Organizations (This needs more wisdom from the practioners)

Relationship Between Organizations - Programs – Data Version 0.1 Goals $$ Info needs, $$ Data need, $$ Judge, Decide, ActAnalyze, Report Actionable Knowledge Decision, Action Public Measure, Organize Organized Data Flow of Information Data systems organize the measurements and models and provide them to programs. Programs analyze the data and provide actionable knowledge to organizations. Organizations evaluate multiple information sources, make decisions and act. Flow of Control Public and special interest groups set up organizations and provides them with funding Organizations develop programs, define their scope, governance and funding Programs satisfy their information needs by monitoring or by using other’s data Data sources acquire the data for their parent programs and also expose them for reuse

Information System Components for AQ Programs Integrated DataDatasets Std. Interface Data Views Std. Interface Data Control Reports Public Human Information SystemMachine Information System Obs. & ModelsDecision Support System

Flow of Data and Usage Control Flow of Control Reports are commissioned by programs Analysts select, explore, and process data for a report Flow of Data Providers expose data to analysts who extract the needed subset The data is pulled into data exploration or processing software Integrated DataDatasets Std. Interface Data Views Std. Interface Data Control Reports Obs. & ModelsDecision Support System

Flow of Data and Usage Control Data Control Requesting Information Providing Information Sensors Acquisition processing User Agencies User Programs NAAQS SIPs Forecast GEOSS … Info System hhh Integrated DataDatasets Std. Interface Data Views Std. Interface Data Control Reports Obs. & ModelsDecision Support System

Steve Young Challenge

The Steve Young Challenges Integrated DataDatasets Std. Interface Data Views Std. Interface Data Control Reports Obs. & ModelsDecision Support System Real-time monitoring & watch for surprises Inform decision-makers & assess outcomes Support for adaptive management

S. Young Challenge #1: Real-time monitoring & watching for surprises Integrated DataDatasets Std. Interface Data Views Std. Interface Data Control Reports Obs. & ModelsDecision Support System Choose Data & ToolsBrowse, ExploreTrigger Response

S. Young Challenge #2: Inform decision-makers & assess outcomes Integrated DataDatasets Std. Interface Data Views Std. Interface Data Control Reports Obs. & ModelsDecision Support System This needs more work Ideas? How could the info-system help here? Air Quality Assessment Compare to Goals Plan Reductions Track Progress Controls (Actions) Monitoring (Sensing) Close the sensory-motor loop!

S. Young Challenge #3. Support for Adoptive AQ Management From ‘Stovepipe’ to Workflow Software Integrated DataDatasets Std. Interface Data Views Std. Interface Data Control Reports Obs. & ModelsDecision Support System Flow of Data Flow of Control Air Quality Data Meteorology Data Emissions Data Informing Public AQ Compliance Status and Trends Network Assess. Tracking Progress Data to Knowledge Transformation Loosely Coupled Workflow

Terry Keating Challenge

The Terry Keating Challenges Facilitate data-model comparison, assimilation Domain ProcessingData Sharing Std. Interface Gen. Processing Std. Interface Data Control Reports Reporting Obs. & ModelsDecision Support System Include global perspective (observation, model, analysis)

Architechture Challenge Friedl Fontaine Barrie Lindsey

GEOSS Architecture Mapped to Air Quality The L. Friedl, K. Fontaine Challenge Integrated DataDatasets Std. Interface Data Views Std. Interface Data Control Reports Obs. & ModelsDecision Support System Not part of DataFed Policy, Management, Personal Decisions Special Architectural Features: Implied data assimilation into models Decision Support black box Focus on model predictions, not ‘simulation’ Pollutant ‘Characterization’ is implicit

IGOS Architecture Mapped to DataFed The Len Barrie Challenge Integrated DataDatasets Std. Interface Data Views Std. Interface Data Control Reports Obs. & ModelsDecision Support System Not part of DataFed Quality assurance, CalVal? Integrated Data Archives? - Special Architectural Features: Real-time data assimilation into AQ models Characterization of global pattern

F. Lindsey, NASA: Air Quality Collaborative Consortium Goal: Cross-leverage the shared partner resources and activities, while Maintaining partner’s autonomy in capabilities and activities Approach: Exchange the resources at the web ‘interfaces’, like portals. Infuse cutting-edge technologies and resources into Agencies as needed. Use ESIP portal as the linker and the Air Quality cluster as the mediator. Outcome: Sharing and integration could form next-generation capabilities. Savings by re-use and better information by broader perspectives Better air quality through more informed management NOAAEPA Priv DoI Air Quality Consortium Data, Tools, Methods Shared Private NASA Other FedS Applications AQ Policy Regulation Research

DataFed Challenge

Value-Adding Processes Integrated DataDatasets Std. Interface Data Views Std. Interface Data Control Reports Obs. & ModelsDecision Support System Analyzing Filter/Integrate Aggregate/Fuse Assimilate Organizing Document Structure Format Exploring Display Browse Compare Value-Adding Processes Data ManagerTechnical Analysts DB SystemAnalysis SystemReporting System

Loosely Coupled Data Access through Standard Protocols OGC Web Coverage Service (WCS) Client request Capabilities Server returns Capabilities and data ‘Profile’ Client requests data by ‘where, when, what’ query Server returns data ‘cube’ in requested format GetCapabilities GetData Capabilities, ‘Profile’ Data Where? When? What? Which Format? Server Back End Std. Interface Client Front End Std. Interface QueryGetData Standards Where?BBOXOGC, ISO When?TimeOGC, ISO What?TemperatureCF FormatnetCDF, HDF..CF, EOS, OGC T2T1 Integrated DataDatasets Std. Interface Data Views Std. Interface Data Control Reports Obs. & ModelsDecision Support System

Web Services and Workflow for Loose Coupling Integrated DataDatasets Std. Interface Data Views Std. Interface Data Control Reports Obs. & ModelsDecision Support System Service Broker Service Provider Publish Find Bind Service User Web Service Interaction Service Chaining & Workflow

Collaborative Reporting and Dynamic Delivery Integrated DataDatasets Std. Interface Data Views Std. Interface Data Control Reports Obs. & ModelsDecision Support System Co Writing - Wiki Screencast Analysis Reports: Information supplied by many Needs continuous program feedback Report needs many authors Wiki technologies are for collaborative writing Dynamic Delivery: Much of the content is dynamic Animated presentations are compelling Movies and screencasts are for dynamic delivery

The Network Effect: Less Cost, More Benefits through Data Reuse Program Public Data Organization Data Program Organization Data Program Data Orgs Develop Programs Programs ask/get Data Public sets up Orgs Pay only once Richer content Less Prog. Cost More Knowledge Less Soc. Cost More Soc. Benefit Data Re-Use Network Effect Data are costly resource – should be reused (recycled) for multiple applications Data reuse saves $$ to programs and allows richer knowledge creation Data reuse, like recycling takes some effort: labeling, organizing, distributing

GEOSS Challenge 1: Jose Achache

GEOSS/AMI Domain of Actions Based on ideas of P. Senge: Architecture of Learning OrganizationsArchitecture of Learning Organizations Possible DataFed Roles: 1.Guiding Idea: Refine, solidify, evangelize the System of Systems idea 2.Methods and Tools: Develop, promote, implement standards; Coordinate SoS use cases 3.Infrastructure: Maintain the DataFed middleware for distributed data access; Supply Agency champions with SoS ‘sales’ material P. Senge et. al, 1994: Fifth Discipline Fieldbook (Link)Link Guiding Idea: System of Systems Methods, Tools: Standards, Use Cases Infrastructure: Web 2.0, Local Champs Domain of Action Organizational Architecture

Key Agencies for Air Quality Management and Science Flexible NAAMS Advanced Monitoring Initiative Programmatic Path to GEOSS

Goal: Advance air quality model-observation complex to level of meteorological FDDA systems Build Organizational frameworks : IGACO-EMEP efforts Data base, IT standards: data unification center? Practical LRTAP task? Standardization/QAQC Reference material, or method (aerosols) Adopt model evaluation/fusion as a design principle Support linkage of ground based and satellite observation platforms through development of a sustainable vertical profiling system (aircraft and ground based lidar) Address integration of disparate data bases –QA/QC: provide requirements for data standards/metadata descriptions –Data base unification Harness the communities around major tools, platforms and programs. –Air quality modeling platforms (GEOS-chem, MOZART, CMAQ,….more) –Satellite Instruments (MODIS, OMI) –Existing routine surface (AIRNow) and aircraft programs –Integration efforts (AEROCOM)

The Architecture of SoS Networks SoS Architecture – Form and Function Different aspects of the network Right Level of Networking?

One of the key factors dictating the achievement of system-of-systems configurations that support network centric operations is the availability of mechanisms that promote information sharing among systems. Direct system-to-system linkages presuppose that the systems know a priori about each system that might benefit from any other system. Many combat situations have disproved this assumption. Emergent behavior in a network that is only created by the combination, not by the individual members. Emergent behavior: novel and coherent (logically connected, purposeful, meaningful) structures, patterns and properties arising from self-organization (mashers are self-organizers) in complex systems WikipediaWikipedia Emergent behavior is created by mashing Mashing occurs when open on both ends of network.- diffusion –Input side – grow by assimilation –Output side – Harvest (i.e. DataFed harvests datasets)

Stages of Self-Organization Stage 1: Expose goods. Sharing was doable, but lots of work. Need glue, duct tape, shop-based approach. Value Shop. Burden: user pays Stage 2: Mediated sharing. DataFed, where wrappers/adapters used. Loose coupling automate the wrapper and mediator Burden: Mediator pays. Users gain. Stage 3: Creating new things together, service chaining, mashing, collaboration – Emergent behavior where network produces value, non- linearly. Burden: creators Tim Burners-Lee: The web allows creating new things together. Howard Rheingold: The computer as mind amplifiers. The ultimate goal of all this is to both amplify the mind of individuals and also connect the minds.