L inked E nvironments for A tmospheric D iscovery Linked Environments for Atmospheric Discovery (LEAD) Kelvin K. Droegemeier School of Meteorology and Center for Analysis and Prediction of Storms University of Oklahoma Jay Alameda National Center for Supercomputing Applications University of Illinois at Urbana-Champaign

L inked E nvironments for A tmospheric D iscovery Geosciences CI Challenges Enormously complex human-natural system –Vast temporal (sec to B yrs) and spatial (microns to 1000s of km) scales –Highly nonlinear behavior Massive data sets –physical and digital –static/legacy and dynamic/streaming –geospatially referenced –multidisciplinary and heterogeneous –open access

L inked E nvironments for A tmospheric D iscovery Geosciences CI Challenges Massive computation –weather, space weather, climate, hydrologic modeling –seismic inversion –coupled physical system models Inherently field-based, visual disciplines with the need to manage information for long periods of time Bringing advanced CI capabilities to education at all levels Connecting the last mile to operational practitioners

L inked E nvironments for A tmospheric D iscovery Each year, mesoscale weather – floods, tornadoes, hail, strong winds, lightning, and winter storms – causes hundreds of deaths, routinely disrupts transportation and commerce, and results in annual economic losses > $13B. Where ALL These Elements Converge: Mesoscale Weather

L inked E nvironments for A tmospheric D iscovery

What Would You Do???

L inked E nvironments for A tmospheric D iscovery What Weather Technology Does… Forecast Models NEXRAD Radar Decision Support Systems

L inked E nvironments for A tmospheric D iscovery What Weather Technology Does… Forecast Models NEXRAD Radar Decision Support Systems Absolutely Nothing!

L inked E nvironments for A tmospheric D iscovery The LEAD Goal Provide the IT necessary to allow People (scientists, students, operational practitioners) and Technologies (models, sensors, data mining) TO INTERACT WITH WEATHER

L inked E nvironments for A tmospheric D iscovery The Roadblock The study of mesoscale weather is stifled by rigid IT frameworks that cannot accommodate the –real time, on-demand, and dynamically-adaptive needs of mesoscale weather research; –its disparate, high volume data sets and streams; and –its tremendous computational demands, which are among the greatest in all areas of science and engineering Some illustrative examples…

L inked E nvironments for A tmospheric D iscovery Analysis/Assimilation Quality Control Retrieval of Unobserved Quantities Creation of Gridded Fields Prediction/Detection PCs to Teraflop Systems Product Generation, Display, Dissemination End Users NWS Private Companies Students Traditional Methodology STATIC OBSERVATIONS Radar Data Mobile Mesonets Surface Observations Upper-Air Balloons Commercial Aircraft Geostationary and Polar Orbiting Satellite Wind Profilers GPS Satellites

L inked E nvironments for A tmospheric D iscovery Analysis/Assimilation Quality Control Retrieval of Unobserved Quantities Creation of Gridded Fields Prediction/Detection PCs to Teraflop Systems Product Generation, Display, Dissemination End Users NWS Private Companies Students Traditional Methodology STATIC OBSERVATIONS Radar Data Mobile Mesonets Surface Observations Upper-Air Balloons Commercial Aircraft Geostationary and Polar Orbiting Satellite Wind Profilers GPS Satellites The Process is Entirely Serial and Static (Pre-Scheduled): No Response to the Weather! The Process is Entirely Serial and Static (Pre-Scheduled): No Response to the Weather!

L inked E nvironments for A tmospheric D iscovery The Consequence: Model Grids Fixed in Time – No Adaptivity

L inked E nvironments for A tmospheric D iscovery Analysis/Assimilation Quality Control Retrieval of Unobserved Quantities Creation of Gridded Fields Prediction/Detection PCs to Teraflop Systems Product Generation, Display, Dissemination End Users NWS Private Companies Students STATIC OBSERVATIONS Radar Data Mobile Mesonets Surface Observations Upper-Air Balloons Commercial Aircraft Geostationary and Polar Orbiting Satellite Wind Profilers GPS Satellites The LEAD Vision: No Longer Serial or Static Models Responding to Observations

L inked E nvironments for A tmospheric D iscovery 10 km 3 km 1 km 20 km Model Dynamic Adaptivity t = t o

L inked E nvironments for A tmospheric D iscovery t = t o + 6 Hours 10 km 3 km 10 km 20 km

L inked E nvironments for A tmospheric D iscovery Today’s Standard Computer Forecast 12-hour National Forecast (coarse grid) Radar (Tornadoes in Arkansas)

L inked E nvironments for A tmospheric D iscovery Today’s Standard Computer Forecast 12-hour National Forecast (coarse grid) Radar (Tornadoes in Arkansas)

L inked E nvironments for A tmospheric D iscovery Radar (Tornadoes in Arkansas) 6-hour Mesoscale Forecast (medium grid) Radar Experimental Mesoscale Window Radar

L inked E nvironments for A tmospheric D iscovery Radar (Tornadoes in Arkansas) 6-hour Mesoscale Forecast (medium grid) Radar Experimental Mesoscale Window Radar

L inked E nvironments for A tmospheric D iscovery Radar6-hour Local Forecast (fine grid) Experimental Storm-Scale Window Xue et al. (2003)

L inked E nvironments for A tmospheric D iscovery Dynamic Adaptivity in Action

L inked E nvironments for A tmospheric D iscovery 11 h Forecast 20 June 2001 (6 km) Courtesy Weather Decision Technologies, Inc.

L inked E nvironments for A tmospheric D iscovery 9 h Forecast 20 June 2001 (6 km) Courtesy Weather Decision Technologies, Inc.

L inked E nvironments for A tmospheric D iscovery 5 h Forecast 20 June 2001 (6 km) Courtesy Weather Decision Technologies, Inc.

L inked E nvironments for A tmospheric D iscovery 3 h Forecast 20 June 2001 (6 km) Courtesy Weather Decision Technologies, Inc.

L inked E nvironments for A tmospheric D iscovery Mesoscale Weather LEAD: Users INTERACTING with Weather

L inked E nvironments for A tmospheric D iscovery Mesoscale Weather NWS National Static Observations & Grids LEAD: Users INTERACTING with Weather

L inked E nvironments for A tmospheric D iscovery Mesoscale Weather NWS National Static Observations & Grids LEAD: Users INTERACTING with Weather Local Observations

L inked E nvironments for A tmospheric D iscovery Mesoscale Weather NWS National Static Observations & Grids Users ADaM ADAS Tools LEAD: Users INTERACTING with Weather Local Observations

L inked E nvironments for A tmospheric D iscovery Mesoscale Weather NWS National Static Observations & Grids Users ADaM ADAS Tools Local Physical Resources Remote Physical (Grid) Resources Virtual/Digital Resources and Services MyLEAD Portal LEAD: Users INTERACTING with Weather Local Observations

L inked E nvironments for A tmospheric D iscovery Mesoscale Weather NWS National Static Observations & Grids Users ADaM ADAS Tools Local Physical Resources Remote Physical (Grid) Resources Virtual/Digital Resources and Services MyLEAD Portal LEAD: Users INTERACTING with Weather Interaction Level I Local Observations

L inked E nvironments for A tmospheric D iscovery Analysis/Assimilation Quality Control Retrieval of Unobserved Quantities Creation of Gridded Fields Prediction/Detection PCs to Teraflop Systems Product Generation, Display, Dissemination End Users NWS Private Companies Students Traditional Methodology STATIC OBSERVATIONS Radar Data Mobile Mesonets Surface Observations Upper-Air Balloons Commercial Aircraft Geostationary and Polar Orbiting Satellite Wind Profilers GPS Satellites Observing Systems Operate Largely Independent of the Weather – Little Adaptivity Observing Systems Operate Largely Independent of the Weather – Little Adaptivity

L inked E nvironments for A tmospheric D iscovery NEXRAD Doppler Weather Radar Network

L inked E nvironments for A tmospheric D iscovery The Limitations of NEXRAD

L inked E nvironments for A tmospheric D iscovery The Limitations of NEXRAD #1. Operates largely independent of the prevailing weather conditions

L inked E nvironments for A tmospheric D iscovery The Limitations of NEXRAD #2. Earth’s curvature prevents 72% of the atmosphere below 1 km from being observed #1. Operates largely independent of the prevailing weather conditions

L inked E nvironments for A tmospheric D iscovery The Limitations of NEXRAD #2. Earth’s curvature prevents 72% of the atmosphere below 1 km from being observed #1. Operates largely independent of the prevailing weather conditions #3. Operates entirely independent from the models and algorithms that use its data

L inked E nvironments for A tmospheric D iscovery Source: NWS Office of Science and Technology The Consequence: 3 of Every 4 Tornado Warnings is a False Alarm

L inked E nvironments for A tmospheric D iscovery Analysis/Assimilation Quality Control Retrieval of Unobserved Quantities Creation of Gridded Fields Prediction/Detection PCs to Teraflop Systems Product Generation, Display, Dissemination End Users NWS Private Companies Students The LEAD Vision: No Longer Serial or Static DYNAMIC OBSERVATIONS Models and Algorithms Driving Sensors

L inked E nvironments for A tmospheric D iscovery New NSF Engineering Research Center for Adaptive Sensing of the Atmosphere (CASA) UMass/Amherst is lead institution Concept: inexpensive, dual-polarization phased array Doppler radars on cell towers – existing IT and power infrastructures! Adaptive dynamic sensing of multiple targets (“DCAS”)

L inked E nvironments for A tmospheric D iscovery New NSF Engineering Research Center for Adaptive Sensing of the Atmosphere (CASA) UMass/Amherst is lead institution Concept: inexpensive, dual-polarization phased array Doppler radars on cell towers – existing IT and power infrastructures! Adaptive dynamic sensing of multiple targets (“DCAS”)

L inked E nvironments for A tmospheric D iscovery New NSF Engineering Research Center for Adaptive Sensing of the Atmosphere (CASA) UMass/Amherst is lead institution Concept: inexpensive, dual-polarization phased array Doppler radars on cell towers – existing IT and power infrastructures! Adaptive dynamic sensing of multiple targets (“DCAS”)

L inked E nvironments for A tmospheric D iscovery Users ADaM ADAS Tools NWS National Static Observations & Grids Mesoscale Weather Local Observations MyLEAD Portal Local Physical Resources Remote Physical (Grid) Resources Virtual/Digital Resources and Services LEAD: Users INTERACTING with Weather

L inked E nvironments for A tmospheric D iscovery Experimental Dynamic Observations Users ADaM ADAS Tools NWS National Static Observations & Grids Mesoscale Weather Local Observations MyLEAD Portal Local Physical Resources Remote Physical (Grid) Resources Virtual/Digital Resources and Services LEAD: Users INTERACTING with Weather

L inked E nvironments for A tmospheric D iscovery Experimental Dynamic Observations Users ADaM ADAS Tools NWS National Static Observations & Grids Mesoscale Weather Local Observations MyLEAD Portal Local Physical Resources Remote Physical (Grid) Resources Virtual/Digital Resources and Services LEAD: Users INTERACTING with Weather Interaction Level II

L inked E nvironments for A tmospheric D iscovery The LEAD Goal Restated To create an integrated, scalable framework that allows analysis tools, forecast models, and data repositories to be used as dynamically adaptive, on-demand systems that can –change configuration rapidly and automatically in response to weather; –continually be steered by new data (i.e., the weather); –respond to decision-driven inputs from users; –initiate other processes automatically; and –steer remote observing technologies to optimize data collection for the problem at hand; –operate independent of data formats and the physical location of data or computing resources

L inked E nvironments for A tmospheric D iscovery CS Challenges/Barriers Workflow –Dynamic/agile/reentrant Data –Synchronization, fault-tolerance, metadata, cataloging, interchange, ontologies Monitoring and performance estimation –Detection of vulnerabilities, recovery, autonomy Mining –Grid functionality, scheduling, fault tolerance

L inked E nvironments for A tmospheric D iscovery Meteorology Challenges/Barriers “Packaging” of complex systems (WRF, ADAS) Fault tolerance Continuous model updating for effective use of truly streaming observations Storm-scale ensemble methodologies Hazardous weather detections based upon gridded analyses versus use of “raw” sensor data alone Dynamically adaptive forecasting (models and observations) – how good compared to current static methodologies?

L inked E nvironments for A tmospheric D iscovery LEAD Architecture Distributed Resources Resource Access Services User Interface Desktop Applications LEAD Portal Portlets Crosscutting Services Configuration and Execution Services Application Resource Broker (Scheduler) Application & Configuration Services Client Interface Data Services Workflow Services Catalog Services

L inked E nvironments for A tmospheric D iscovery LEAD Architecture Distributed Resources Computation Specialized Applications Steerable Instruments Storage Data Bases Resource Access Services GRAM Globus SSH Scheduler LDM OPenDAP Generic Ingest Service User Interface Desktop Applications IDV WRF Configuration GUI LEAD Portal Portlets Visualization Workflow Education Monitor Control Ontology Query Browse Control Crosscutting Services Authorization Authentication Monitoring Notification Configuration and Execution Services Workflow Service Control Service Query Service Stream Service Ontology Service MyLEAD Workflow Engine/Factories VO Catalog THREDDS Application Resource Broker (Scheduler) Host Environment GPIR Application Host Execution Description Applications (WRF, ADaM, IDV, ADAS) Application Description Application & Configuration Services Client Interface Observations Streams Static Archived Data Services Workflow Services Catalog Services Decoder/Resolver Service RLS OGSA- DAI

L inked E nvironments for A tmospheric D iscovery Key System Components and Technologies Capability/ResourcePrincipal Technologies Atmospheric, Oceanographic, Land- Surface Observations CONDUIT, CRAFT, MADIS, IDD, NOAAPort, GCMD, SSEC, ESDIS, NVODS, NCDC Operational Model GridsCONDUIT, NOMADS Data Assimilation SystemsADAS, WRF 3DVAR Atmospheric Prediction SystemsWRF, ARPS VisualizationIDV Data MiningADaM NSF NMI ProjectGlobus Tool Kit Semantic Interchange and FormattingESML, NetCDF, HDF5 Adaptive Observing Systems (Radars)CASA OK Test Bed, V-CHILL LEAD PortalNSF NMI Project (OGCE) Workflow OrchestrationBPEL4WS MonitoringAutopilot Data Cataloging/ManagementTHREDDS, MCS, SRB

L inked E nvironments for A tmospheric D iscovery The Driver: Canonical Research & Education Problems The LEAD Research Process Fundamental Scientific and Technological Barriers System Functional Requirements and Capabilities System Architecture and Definition of Services Building Blocks Technology Generations End User Focus Group Testing and Deployment The End Game: Canonical Research & Education Problems Basic Research Prototypes Test Beds

L inked E nvironments for A tmospheric D iscovery Generation 2 Dynamic Workflow Generation 3 Adaptive Sensing Generation 3 Adaptive Sensing Generation 2 Dynamic Workflow Generation 1 Static Workflow Generation 1 Static Workflow Generation 2 Dynamic Workflow Generation 1 Static Workflow Year 1 Year 2 Year 3 Year 4 Year 5 LEAD Technology Generations Technology & Capability Generation 1 Static Workflow Generation 1 Static Workflow Look-Ahead Research

L inked E nvironments for A tmospheric D iscovery In LEAD, Everything is a Service Finite number of services – they’re the “low-level” elements but consist of lots of hidden pieces…services within services. Service A (ADAS) Service B (WRF) Service C (NEXRAD Stream) Service D (MyLEAD) Service E (VO Catalog) Service F (IDV) Service G (Monitoring) Service H (Scheduling) Service I (ESML) Service J (Repository) Service K (Ontology) Service L (Decoder) Many others…

L inked E nvironments for A tmospheric D iscovery Start by Building Simple Prototypes to Establish the Services/Other Capabilities… Service C (NEXRAD Stream) Service F (IDV) Service L (Decoder) Prototype X

L inked E nvironments for A tmospheric D iscovery Start by Building Simple Prototypes to Establish the Services/Other Capabilities… Service C (NEXRAD Stream) Service F (IDV) Service L (Decoder) Prototype Y Service D (MyLEAD) Service E (VO Catalog)

L inked E nvironments for A tmospheric D iscovery Start by Building Simple Prototypes to Establish the Services/Other Capabilities… Service C (NEXRAD Stream) Service F (IDV) Service L (Decoder) Prototype Z Service A (ADAS) Service I (ESML) Service J (Repository) Service D (MyLEAD) Service E (VO Catalog)

L inked E nvironments for A tmospheric D iscovery Service B (WRF) Service A (ADAS) Service C (NEXRAD Stream) Service D (MyLEAD) Service L (Mining) Service L (Decoder) Service J (Repository) …and then Solve General Problems by Linking them Together in Workflows

L inked E nvironments for A tmospheric D iscovery Service B (WRF) Service A (ADAS) Service C (NEXRAD Stream) Service D (MyLEAD) Service L (Mining) Service L (Decoder) Service J (Repository) …and then Solve General Problems by Linking them Together in Workflows Note that these services can be used as stand-alone capabilities, independent of the LEAD infrastructure (e.g., portal)

L inked E nvironments for A tmospheric D iscovery

Feedback from Application Scientists Benefits Single sign-on feature is very handy Secured access to compute resources from a browser, is increasing productivity Difficulties Grid authentication is not trivial to use - important feature needed by an application scientist Hard to keep track of continuously evolving grid middleware System needs continuous development as middleware on production machines moves forward and is not backward compatible

Canonical Problem #3 Problem #3: Dynamically Adaptive, High-Resolution Nested Ensemble Forecasts Goal: For the continental United States (CONUS), automatically generate a 1-km grid spacing ADAS analysis every 30 minutes, and a 6-hour, 2-km grid spacing CONUS forecast every 3 hours. Automatically launch finer-grid spacing nested WRF ensemble forecasts when data mining algorithms – applied to both the CONUS analyses and forecasts – detect features indicative of storm potential (e.g., convergence lines, strong instability, incipient convection) or actual storm development. Conduct rigorous post-mortem assessment of statistical forecast skill and compare the high- resolution nested grid forecasts with the single-grid CONUS run at coarser resolution.

Canonical Problem #3 START Define Data Requirements and Query for Desired Data

Canonical Problem #3 START Define Data Requirements and Query for Desired Data Allocate Storage and Move/Stream Data to Appropriate Location (e.g., PACI Center)

Canonical Problem #3 START Define Data Requirements and Query for Desired Data Allocate Storage and Move/Stream Data to Appropriate Location (e.g., PACI Center) Allocate Computational Resources

Canonical Problem #3 START Define Data Requirements and Query for Desired Data Allocate Storage and Move/Stream Data to Appropriate Location (e.g., PACI Center) Allocate Computational Resources Data Surface Observations Upper-Air Observations Commercial Aircraft Data NEXRAD Radar Data Satellite Data Wind Profiler Data Land Surface Data Terrain Data Background Model Fields and Previous Forecasts

ESML & Decoding Canonical Problem #3 START Define Data Requirements and Query for Desired Data Allocate Storage and Move/Stream Data to Appropriate Location (e.g., PACI Center) Allocate Computational Resources Data Surface Observations Upper-Air Observations Commercial Aircraft Data NEXRAD Radar Data Satellite Data Wind Profiler Data Land Surface Data Terrain Data Background Model Fields and Previous Forecasts

ESML & Decoding Remapping, Gridding, Conversion ADAS Quality Control Canonical Problem #3 START Define Data Requirements and Query for Desired Data Allocate Storage and Move/Stream Data to Appropriate Location (e.g., PACI Center) Allocate Computational Resources Data Surface Observations Upper-Air Observations Commercial Aircraft Data NEXRAD Radar Data Satellite Data Wind Profiler Data Land Surface Data Terrain Data Background Model Fields and Previous Forecasts

ESML & Decoding Remapping, Gridding, Conversion ADAS Quality Control ADAS Analysis Processing Canonical Problem #3 START Define Data Requirements and Query for Desired Data Allocate Storage and Move/Stream Data to Appropriate Location (e.g., PACI Center) Allocate Computational Resources Data Surface Observations Upper-Air Observations Commercial Aircraft Data NEXRAD Radar Data Satellite Data Wind Profiler Data Land Surface Data Terrain Data Background Model Fields and Previous Forecasts

ESML & Decoding Remapping, Gridding, Conversion ADAS Quality Control ADAS Analysis Processing ADAS Analysis (3D Gridded Fields) + Background Fields Canonical Problem #3 START Define Data Requirements and Query for Desired Data Allocate Storage and Move/Stream Data to Appropriate Location (e.g., PACI Center) Allocate Computational Resources Data Surface Observations Upper-Air Observations Commercial Aircraft Data NEXRAD Radar Data Satellite Data Wind Profiler Data Land Surface Data Terrain Data Background Model Fields and Previous Forecasts

ESML & Decoding Remapping, Gridding, Conversion ADAS Quality Control ADAS Analysis Processing ADAS Analysis (3D Gridded Fields) + Background Fields ADAS-to-WRF Converter Canonical Problem #3 START Define Data Requirements and Query for Desired Data Allocate Storage and Move/Stream Data to Appropriate Location (e.g., PACI Center) Allocate Computational Resources Data Surface Observations Upper-Air Observations Commercial Aircraft Data NEXRAD Radar Data Satellite Data Wind Profiler Data Land Surface Data Terrain Data Background Model Fields and Previous Forecasts

ESML & Decoding Remapping, Gridding, Conversion ADAS Quality Control ADAS Analysis Processing ADAS Analysis (3D Gridded Fields) + Background Fields ADAS-to-WRF Converter 3D Gridded Fields in WRF Mass Coordinate + Suite of Ensemble Initial Conditions Canonical Problem #3 START Define Data Requirements and Query for Desired Data Allocate Storage and Move/Stream Data to Appropriate Location (e.g., PACI Center) Allocate Computational Resources Data Surface Observations Upper-Air Observations Commercial Aircraft Data NEXRAD Radar Data Satellite Data Wind Profiler Data Land Surface Data Terrain Data Background Model Fields and Previous Forecasts

ESML & Decoding Remapping, Gridding, Conversion ADAS Quality Control ADAS Analysis Processing ADAS Analysis (3D Gridded Fields) + Background Fields ADAS-to-WRF Converter 3D Gridded Fields in WRF Mass Coordinate + Suite of Ensemble Initial Conditions Multiple Copies of WRF Forecast Model Running Simultaneously Canonical Problem #3 START Define Data Requirements and Query for Desired Data Allocate Storage and Move/Stream Data to Appropriate Location (e.g., PACI Center) Allocate Computational Resources Data Surface Observations Upper-Air Observations Commercial Aircraft Data NEXRAD Radar Data Satellite Data Wind Profiler Data Land Surface Data Terrain Data Background Model Fields and Previous Forecasts

ESML & Decoding Remapping, Gridding, Conversion ADAS Quality Control ADAS Analysis Processing ADAS Analysis (3D Gridded Fields) + Background Fields ADAS-to-WRF Converter 3D Gridded Fields in WRF Mass Coordinate + Suite of Ensemble Initial Conditions WRF Gridded Output Multiple Copies of WRF Forecast Model Running Simultaneously Canonical Problem #3 START Define Data Requirements and Query for Desired Data Allocate Storage and Move/Stream Data to Appropriate Location (e.g., PACI Center) Allocate Computational Resources Data Surface Observations Upper-Air Observations Commercial Aircraft Data NEXRAD Radar Data Satellite Data Wind Profiler Data Land Surface Data Terrain Data Background Model Fields and Previous Forecasts

ESML & Decoding Remapping, Gridding, Conversion ADAS Quality Control ADAS Analysis Processing ADAS Analysis (3D Gridded Fields) + Background Fields ADAS-to-WRF Converter 3D Gridded Fields in WRF Mass Coordinate + Suite of Ensemble Initial Conditions WRF Gridded Output myLEAD Storage Multiple Copies of WRF Forecast Model Running Simultaneously Canonical Problem #3 START Define Data Requirements and Query for Desired Data Allocate Storage and Move/Stream Data to Appropriate Location (e.g., PACI Center) Allocate Computational Resources Data Surface Observations Upper-Air Observations Commercial Aircraft Data NEXRAD Radar Data Satellite Data Wind Profiler Data Land Surface Data Terrain Data Background Model Fields and Previous Forecasts

ESML & Decoding Remapping, Gridding, Conversion ADAS Quality Control ADAS Analysis Processing ADAS Analysis (3D Gridded Fields) + Background Fields ADAS-to-WRF Converter 3D Gridded Fields in WRF Mass Coordinate + Suite of Ensemble Initial Conditions WRF Gridded Output myLEAD Storage Multiple Copies of WRF Forecast Model Running Simultaneously Canonical Problem #3 Meta Data Creation and Cataloging START Define Data Requirements and Query for Desired Data Allocate Storage and Move/Stream Data to Appropriate Location (e.g., PACI Center) Allocate Computational Resources Data Surface Observations Upper-Air Observations Commercial Aircraft Data NEXRAD Radar Data Satellite Data Wind Profiler Data Land Surface Data Terrain Data Background Model Fields and Previous Forecasts

ESML & Decoding Remapping, Gridding, Conversion ADAS Quality Control ADAS Analysis Processing ADAS Analysis (3D Gridded Fields) + Background Fields ADAS-to-WRF Converter 3D Gridded Fields in WRF Mass Coordinate + Suite of Ensemble Initial Conditions WRF Gridded Output myLEAD Storage Multiple Copies of WRF Forecast Model Running Simultaneously Canonical Problem #3 Meta Data Creation and Cataloging Visualization & Data Mining START Define Data Requirements and Query for Desired Data Allocate Storage and Move/Stream Data to Appropriate Location (e.g., PACI Center) Allocate Computational Resources Data Surface Observations Upper-Air Observations Commercial Aircraft Data NEXRAD Radar Data Satellite Data Wind Profiler Data Land Surface Data Terrain Data Background Model Fields and Previous Forecasts

ESML & Decoding Remapping, Gridding, Conversion ADAS Quality Control ADAS Analysis Processing ADAS Analysis (3D Gridded Fields) + Background Fields ADAS-to-WRF Converter 3D Gridded Fields in WRF Mass Coordinate + Suite of Ensemble Initial Conditions WRF Gridded Output myLEAD Storage Multiple Copies of WRF Forecast Model Running Simultaneously Canonical Problem #3 Meta Data Creation and Cataloging Visualization & Data Mining STOP START Define Data Requirements and Query for Desired Data Allocate Storage and Move/Stream Data to Appropriate Location (e.g., PACI Center) Allocate Computational Resources Data Surface Observations Upper-Air Observations Commercial Aircraft Data NEXRAD Radar Data Satellite Data Wind Profiler Data Land Surface Data Terrain Data Background Model Fields and Previous Forecasts

ESML & Decoding Remapping, Gridding, Conversion ADAS Quality Control ADAS Analysis Processing ADAS Analysis (3D Gridded Fields) + Background Fields ADAS-to-WRF Converter 3D Gridded Fields in WRF Mass Coordinate + Suite of Ensemble Initial Conditions WRF Gridded Output myLEAD Storage Multiple Copies of WRF Forecast Model Running Simultaneously Canonical Problem #3 Meta Data Creation and Cataloging Visualization & Data Mining STOP START Define Data Requirements and Query for Desired Data Allocate Storage and Move/Stream Data to Appropriate Location (e.g., PACI Center) Allocate Computational Resources Data Surface Observations Upper-Air Observations Commercial Aircraft Data NEXRAD Radar Data Satellite Data Wind Profiler Data Land Surface Data Terrain Data Background Model Fields and Previous Forecasts

ESML & Decoding Remapping, Gridding, Conversion ADAS Quality Control ADAS Analysis Processing ADAS Analysis (3D Gridded Fields) + Background Fields ADAS-to-WRF Converter 3D Gridded Fields in WRF Mass Coordinate + Suite of Ensemble Initial Conditions WRF Gridded Output myLEAD Storage Multiple Copies of WRF Forecast Model Running Simultaneously Canonical Problem #3 Meta Data Creation and Cataloging Visualization & Data Mining STOP Adjust Forecast Configuration and Schedule Resources START Define Data Requirements and Query for Desired Data Allocate Storage and Move/Stream Data to Appropriate Location (e.g., PACI Center) Allocate Computational Resources Data Surface Observations Upper-Air Observations Commercial Aircraft Data NEXRAD Radar Data Satellite Data Wind Profiler Data Land Surface Data Terrain Data Background Model Fields and Previous Forecasts

ESML & Decoding Remapping, Gridding, Conversion ADAS Quality Control ADAS Analysis Processing ADAS Analysis (3D Gridded Fields) + Background Fields ADAS-to-WRF Converter 3D Gridded Fields in WRF Mass Coordinate + Suite of Ensemble Initial Conditions WRF Gridded Output myLEAD Storage Multiple Copies of WRF Forecast Model Running Simultaneously Canonical Problem #3 Meta Data Creation and Cataloging Visualization & Data Mining STOP Adjust Forecast Configuration and Schedule Resources START Define Data Requirements and Query for Desired Data Allocate Storage and Move/Stream Data to Appropriate Location (e.g., PACI Center) Allocate Computational Resources Data Surface Observations Upper-Air Observations Commercial Aircraft Data NEXRAD Radar Data Satellite Data Wind Profiler Data Land Surface Data Terrain Data Background Model Fields and Previous Forecasts

L inked E nvironments for A tmospheric D iscovery How Would One Go About Setting This Up in LEAD?? The “First LEAD Commandment” –Thou shalt not use unintelligible computer science jargon in the portal for describing options/tasks to end users –Foo, portlet, ontology, widget, daemon, worm, hash…

L inked E nvironments for A tmospheric D iscovery

Select/Search for Data Data Environment Select Region of Interest

L inked E nvironments for A tmospheric D iscovery

Tools Environment Select Tools IDV Visualizer ADAS Assimilator WRF Predictor ADaM Data Miner Decoders

L inked E nvironments for A tmospheric D iscovery

Experiments Environment new load saved

L inked E nvironments for A tmospheric D iscovery Grid Resources Environment Select Resource

ESML & Decoding Remapping, Gridding, Conversion ADAS Quality Control ADAS Analysis Processing ADAS Analysis (3D Gridded Fields) + Background Fields ADAS-to-WRF Converter 3D Gridded Fields in WRF Mass Coordinate + Suite of Ensemble Initial Conditions WRF Gridded Output myLEAD Storage Multiple Copies of WRF Forecast Model Running Simultaneously Canonical Problem #3 Meta Data Creation and Cataloging Visualization & Data Mining STOP Adjust Forecast Configuration and Schedule Resources START Define Data Requirements and Query for Desired Data Allocate Storage and Move/Stream Data to Appropriate Location (e.g., PACI Center) Allocate Computational Resources Data Surface Observations Upper-Air Observations Commercial Aircraft Data NEXRAD Radar Data Satellite Data Wind Profiler Data Land Surface Data Terrain Data Background Model Fields and Previous Forecasts