Sponsored by the National Science Foundation GENI Alpha Demonstration Nowcasting: UMass/CASA Weather Radar Demonstration David Irwin November 3,

Sponsored by the National Science Foundation 2 November 3, 2010 Problem CASA (an NSF ERC) is studying experimental networks of small controllable weather radars –Better data is the foundation of better hazardous weather detection and earlier warnings –Complex modeling to detect inclement weather requires many resources: sensors, bandwidth, storage, and computation Costly to dedicate resources for rare events –How do we generate accurate, short-term “nowcasts” using these new distributed radar systems?

Sponsored by the National Science Foundation 3 November 3, 2010 Solution Today: only a few large NEXRAD radars (100s) Tomorrow: many (1000s) smaller, less expensive radars produce data close to the ground where weather happens Requires a flexible infrastructure for coordinated provisioning of shared sensing, networking, storage, and computing resources on-demand

Sponsored by the National Science Foundation 4 November 3, 2010 Example: Puerto Rico Testbed UPRM Student Testbed –Led by Jorge Trabal, Prof. Sandra Cruz-Pol, and Prof. Jose Colom –

Sponsored by the National Science Foundation 5 November 3, 2010 Demo Background Dynamic end-to-end Nowcasting on GENI –Use GENI/Orca Control Framework (RENCI/Duke) –Reserve heterogeneous slice of resources Sensing Slice: UMass ViSE radars Networking Slice: NLR, BEN-RENCI Computation Slice: Amazon EC2 + UMass and RENCI VMs Storage Slice: Amazon S3

Sponsored by the National Science Foundation 6 November 3, 2010 Demo Data Flow Dynamic end-to-end Nowcasting –Mapping Nowcast Workflows onto GENI Archival Storage Radar Nodes “raw” live data Upstream LDM feed archived netcdf data archived netcdf data Nowcast Processing aggregated multi-radar data aggregated multi-radar data Post to Web Nowcast images for display

Sponsored by the National Science Foundation 7 November 3, 2010 Multi-radar NetCDF Data Nowcast Processing 1.DiCloud Archival Service (S3) 2.LDM Data Feed (EC2) 1.DiCloud Archival Service (S3) 2.LDM Data Feed (EC2) “raw” live data Generate “raw” live data ViSE/CASA radar nodes Generate “raw” live data ViSE/CASA radar nodes Use proxy to track usage-based spending on Amazon and enforce quotas and limits Use proxy to track usage-based spending on Amazon and enforce quotas and limits 1.Ingest mulit-radar data feeds 2.Merge and grid multi-radar data 2.Generate 1min, 5min, and 10min Nowcasts 3.Send results over NLR to Umass 4.Repeat 1.Ingest mulit-radar data feeds 2.Merge and grid multi-radar data 2.Generate 1min, 5min, and 10min Nowcasts 3.Send results over NLR to Umass 4.Repeat ViSE views steerable radars as shared, virtualized resources ViSE views steerable radars as shared, virtualized resources Nowcast images for display Nowcast images for display

Sponsored by the National Science Foundation 8 November 3, 2010 GENI Technologies and Credits UMass-Amherst –ViSE and DiCloud projects University of Puerto Rico, Mayaguez –Jorge Trabal, Prof. Cruz-Pol, and Prof. Colom –OTG Radars Colorado State University –Prof. V. Chandrasekar –Nowcasting Software RENCI/Duke –Orca Control Framework –BEN network Starlight

Sponsored by the National Science Foundation 9 November 3, 2010 Conclusion GENI is critical for next-generation applications –Enable nowcasting in experimental radar systems –GENI capabilities: “sliceability”/virtualization, federation, network programmability Provide domain scientists a new platform –Experiment with tightly integrated systems combining sensing, storage, networking, computing –Engage domain scientists in CASA and elsewhere Extend GENI network to Puerto Rico

Sponsored by the National Science Foundation 10 November 3, 2010 Wrap-up

Sponsored by the National Science Foundation 11 November 3, 2010 Demo Data Flow Dynamic end-to-end Nowcasting on GENI Archival Storage Radar Nodes Generate “raw” data ViSE/CASA radar nodes “raw” live data DiCloud Archival Service on Amazon S3 Upstream LDM feed archived netcdf data archived netcdf data Data publicly available to downstream nodes Use proxy to track usage-based spending on Amazon and enforce quotas and limits ViSE views steerable radars as shared, virtualized resources Nowcast Processing aggregated multi-radar data aggregated multi-radar data Generate Nowcasts 1. Ingest data feeds from multiple radars 2. Merge multi-radar data 3.Generate 1min, 5min, and 10min Nowcasts 4.Repeat Post to Web Nowcast images for display

Sponsored by the National Science Foundation 12 November 3, 2010 Thank you UMass: Emmanuel Cecchet Prashant Shenoy Jim Kurose Eric Lyons UPRM: Gianni Pablos José Ortiz Wilson Castellanos Melissa Acosta José Cordero Benjamín De Jesús Sandra Cruz-Pol José Colom CSU: V. Chandrasekar Evan Ruzanski Yanting Wang RENCI: Ilia Baldine Jeff Chase Anirban Mandel

Sponsored by the National Science Foundation 13 November 3, 2010 Demo Overview Dynamic end-to-end Nowcasting on GENI –Slice of sensing, networking, computing, and storage Archival Storage Radar Nodes Upstream LDM feed Nowcast Processing Post to Web Generate “raw” data ViSE/CASA radars Archive radar data Amazon S3 “raw” live data Archived data available to downstream nodes archived netcdf data archived netcdf data aggregated multi-radar data aggregated multi-radar data Nowcast images for display Generate Nowcasts

Sponsored by the National Science Foundation 14 November 3, 2010 Demo Resource Listing in Orca/GENI