DataFed Challenge

Value-Adding Processes Integrated DataDatasets Std. Interface Data Views Std. Interface Data Control Reports Obs. & ModelsDecision Support System Analyzing Filter/Integrate Aggregate/Fuse Assimilate Organizing Document Structure Format Exploring Display Browse Compare Value-Adding Processes Data ManagerTechnical Analysts DB SystemAnalysis SystemReporting System

Loosely Coupled Data Access through Standard Protocols OGC Web Coverage Service (WCS) Client request Capabilities Server returns Capabilities and data ‘Profile’ Client requests data by ‘where, when, what’ query Server returns data ‘cube’ in requested format GetCapabilities GetData Capabilities, ‘Profile’ Data Where? When? What? Which Format? Server Back End Std. Interface Client Front End Std. Interface QueryGetData Standards Where?BBOXOGC, ISO When?TimeOGC, ISO What?TemperatureCF FormatnetCDF, HDF..CF, EOS, OGC T2T1 Integrated DataDatasets Std. Interface Data Views Std. Interface Data Control Reports Obs. & ModelsDecision Support System

Web Services and Workflow for Loose Coupling Integrated DataDatasets Std. Interface Data Views Std. Interface Data Control Reports Obs. & ModelsDecision Support System Service Broker Service Provider Publish Find Bind Service User Web Service Interaction Service Chaining & Workflow

Collaborative Reporting and Dynamic Delivery Integrated DataDatasets Std. Interface Data Views Std. Interface Data Control Reports Obs. & ModelsDecision Support System Co Writing - Wiki Screencast Analysis Reports: Information supplied by many Needs continuous program feedback Report needs many authors Wiki technologies are for collaborative writing Dynamic Delivery: Much of the content is dynamic Animated presentations are compelling Movies and screencasts are for dynamic delivery

The Network Effect: Less Cost, More Benefits through Data Reuse Program Public Data Organization Data Program Organization Data Program Data Orgs Develop Programs Programs ask/get Data Public sets up Orgs Pay only once Richer content Less Prog. Cost More Knowledge Less Soc. Cost More Soc. Benefit Data Re-Use Network Effect Data are costly resource – should be reused (recycled) for multiple applications Data reuse saves $$ to programs and allows richer knowledge creation Data reuse, like recycling takes some effort: labeling, organizing, distributing

Scheffe Challenge

GEOSS Eco-informatics Accountability/ indicators SIPs, nat.rules designations PHASE PM research Risk/exposure assessments AQ forecasting Programs NAAQS setting EPA NOAA NASA NPS USDA DOE Private Sector States/Tribes RPO’s/Interstate Academia NARSTO NAS, CAAAC CASAC, OMB Enviros Organizations CDC Supersites IMPROVE, NCore PM monit, PAMS CASTNET Lidar systems NADPSatellite data Intensive studies PM centers Other networks: SEARCH, IADN.. Data sources CMAQ GEOS-CHEM Emissions Meteorology Health/mort. records The Scheffe Challenge: ‘Organizations - Programs – Data Mess’ Info System Challenges: What’s the overall dependency Information Flow Forces and Controls on Data Flow Cooperation, Competition, Co-Opetition Information System of Systems

Air Quality Management System: Components and Functions Public Analyzing Interpreting Evaluating Separating Synthesizing Organizing Quality control Formatting Documenting Displaying Deciding Evaluate options Matching goals Compromising Choosing Data Manager, Organizer Technical Analysts, Program Manager Policy Analysts, Decision Maker Value Adding Processes Human Agents Decision Support System (DSS) The primary purpose of data systems is to serve programs/projects Programs perform analysis for Orgs., the DSS is within programs The big decisions of societal importance are Organizations (This needs more wisdom from the practioners)

Relationship Between Organizations - Programs – Data Version 0.1 Goals $$ Info needs, $$ Data need, $$ Judge, Decide, ActAnalyze, Report Actionable Knowledge Decision, Action Public Measure, Organize Organized Data Flow of Information Data systems organize the measurements and models and provide them to programs. Programs analyze the data and provide actionable knowledge to organizations. Organizations evaluate multiple information sources, make decisions and act. Flow of Control Public and special interest groups set up organizations and provides them with funding Organizations develop programs, define their scope, governance and funding Programs satisfy their information needs by monitoring or by using other’s data Data sources acquire the data for their parent programs and also expose them for reuse

Information System Components for AQ Programs Integrated DataDatasets Std. Interface Data Views Std. Interface Data Control Reports Public Human Information SystemMachine Information System Obs. & ModelsDecision Support System

Flow of Data and Usage Control Flow of Control Reports are commissioned by programs Analysts select, explore, and process data for a report Flow of Data Providers expose data to analysts who extract the needed subset The data is pulled into data exploration or processing software Integrated DataDatasets Std. Interface Data Views Std. Interface Data Control Reports Obs. & ModelsDecision Support System

Flow of Data and Usage Control Data Control Requesting Information Providing Information Sensors Acquisition processing User Agencies User Programs NAAQS SIPs Forecast GEOSS … Info System hhh Integrated DataDatasets Std. Interface Data Views Std. Interface Data Control Reports Obs. & ModelsDecision Support System

Architechture Challenge

GEOSS Architecture Mapped to Air Quality The L. Friedl, K. Fontaine Challenge Integrated DataDatasets Std. Interface Data Views Std. Interface Data Control Reports Obs. & ModelsDecision Support System Not part of DataFed Policy, Management, Personal Decisions Special Architectural Features: Implied data assimilation into models Decision Support black box Focus on model predictions, not ‘simulation’ Pollutant ‘Characterization’ is implicit

IGOS Architecture Mapped to DataFed The Len Barrie Challenge Integrated DataDatasets Std. Interface Data Views Std. Interface Data Control Reports Obs. & ModelsDecision Support System Not part of DataFed Quality assurance, CalVal? Integrated Data Archives? - Special Architectural Features: Real-time data assimilation into AQ models Characterization of global pattern

Young Challenge

The Steve Young Challenges Integrated DataDatasets Std. Interface Data Views Std. Interface Data Control Reports Obs. & ModelsDecision Support System Real-time monitoring & watch for surprises Inform decision-makers & assess outcomes Support for adaptive management

S. Young Challenge #1: Real-time monitoring & watching for surprises Integrated DataDatasets Std. Interface Data Views Std. Interface Data Control Reports Obs. & ModelsDecision Support System Choose Data & ToolsBrowse, ExploreTrigger Response

S. Young Challenge #2: Inform decision-makers & assess outcomes Integrated DataDatasets Std. Interface Data Views Std. Interface Data Control Reports Obs. & ModelsDecision Support System This needs more work Ideas? How could the info-system help here? Air Quality Assessment Compare to Goals Plan Reductions Track Progress Controls (Actions) Monitoring (Sensing) Close the sensory-motor loop!

S. Young Challenge #3. Support for Adoptive AQ Management From ‘Stovepipe’ to Workflow Software Integrated DataDatasets Std. Interface Data Views Std. Interface Data Control Reports Obs. & ModelsDecision Support System Flow of Data Flow of Control Air Quality Data Meteorology Data Emissions Data Informing Public AQ Compliance Status and Trends Network Assess. Tracking Progress Data to Knowledge Transformation Loosely Coupled Workflow