1. Making Climate Change Data Easier to Find and Use Michael Corsello Seshu Vaddey Michael.Corsello@ieee.org http://Eclime.blogspot.com

2. Climate Change is a Paradigm Shift Michael.Corsello@ieee.org http://Eclime.blogspot.com

3. Climate Change is a Paradigm Shift Michael.Corsello@ieee.org http://Eclime.blogspot.com

4. Climate Change is a Paradigm Shift Michael.Corsello@ieee.org http://Eclime.blogspot.com

5. Otherwise  We are using old analytical techniques  Designed for an old paradigm  Being applied to a new paradigm of problems Michael.Corsello@ieee.org http://Eclime.blogspot.com

6. Example  You get new Climate Change data Michael.Corsello@ieee.org http://Eclime.blogspot.com

7. Example  What’s the first thing you do? Michael.Corsello@ieee.org http://Eclime.blogspot.com

8. Example  Try to put it into excel Michael.Corsello@ieee.org http://Eclime.blogspot.com

9. Take a closer look at Climate Change data  UW CIG CBCCSP  2 emission scenarios  10 GCM’s  3 downscaling methods  From available total of  6 emission scenarios  23 GCM’s  Multiple Approaches Michael.Corsello@ieee.org http://Eclime.blogspot.com

10. Take a closer look at Climate Change data Total Size of Data Produced ~32 TB % of Total Michael.Corsello@ieee.org http://Eclime.blogspot.com

11. Take a closer look at Climate Change data Total Size of Data Produced ~32 TB % of Total Individual hydrologic projection (297 sites) ~1.3 GB 0.004 % Michael.Corsello@ieee.org http://Eclime.blogspot.com

12. Take a closer look at Climate Change data Total Size of Data Produced ~32 TB % of Total Individual hydrologic projection (297 sites) ~1.3 GB 0.004 % Hydrology (297 Sites, All Projections)) ~18.5 GB 0.06 % Michael.Corsello@ieee.org http://Eclime.blogspot.com

13. Take a closer look at Climate Change data Total Size of Data Produced ~32 TB % of Total Individual hydrologic projection (297 sites) ~1.3 GB 0.004 % Hydrology (297 Sites, All Projections)) ~18.5 GB 0.06 % Temp & Precip data (2 of 21 parameters) Monthly Grids (all HD projections) Daily Grids (all HD projections) ~65 GB ~2.4 TB 0.20 % 7.5 % Michael.Corsello@ieee.org http://Eclime.blogspot.com

14. Take a closer look at Climate Change data Total Size of Data Produced~32 TB% of Total Individual hydrologic projection (297 sites) ~1.3 GB0.004 % Hydrology (297 Sites, All Projections)) ~18.5 GB0.06 % Temp & Precip data (2 of 21 parameters) Monthly Grids (all HD projections) Daily Grids (all HD projections) ~65 GB ~2.4 TB 0.20 % 7.5 % Daily total precipitation Daily average temperature Daily maximum temperature Daily minimum temperature Outgoing longwave radiation Incoming shortwave radiation Relative humidity Vapor pressure deficit Daily evapotranspiration Daily Runoff Daily Baseflow Soil Moisture, Layer 1 Soil Moisture, Layer 2 Soil Moisture, Layer 3 Snow water equivalent Snow depth Potential Evapotranspiration 1 Potential Evapotranspiration 2 Potential Evapotranspiration 3 Potential Evapotranspiration 4 (alfalfa) Potential Evapotranspiration 5 Michael.Corsello@ieee.org http://Eclime.blogspot.com

15. Working with Climate Change data  The Challenge  Volume of data swamps Cyber Infrastructure  Steep learning curves to use new tools  Tools are always changing Michael.Corsello@ieee.org http://Eclime.blogspot.com

16. Enter the Web and Cloud computing  Software as a Service  Platform as a Service  Infrastructure as a Service Michael.Corsello@ieee.org http://Eclime.blogspot.com

17. Enterprise Data Management  Move away from data living on our computers Michael.Corsello@ieee.org http://Eclime.blogspot.com

18. Enterprise Data Management  The data and tools / applications now reside on servers (Cloud)  The data is now more crucial than ever  We all “share” common sets of data “through” the cloud Michael.Corsello@ieee.org http://Eclime.blogspot.com

19. Enterprise Data Management  The data and tools / applications now reside on servers (Cloud)  The data is now more crucial than ever  We all “share” common sets of data “through” the cloud Michael.Corsello@ieee.org http://Eclime.blogspot.com

31. Summary  The need for a paradigm shift  In how we work  This new paradigm must provide for  Ease of use, and value to the organization (Return on Investment)  CRF is working towards this goal  We need users across different domains to work with us Michael.Corsello@ieee.org http://Eclime.blogspot.com

32. Questions? Blog: http://Eclime.blogspot.com Breakout Discussion Session Wednesday at 10am

33. CRF Developed Solution Michael.Corsello@ieee.org http://Eclime.blogspot.com

34. CRF Developed Solution  Develop series of database structures  Based upon “real-world things” (like flows) Michael.Corsello@ieee.org http://Eclime.blogspot.com

35. CRF Developed Solution  Organize these structures into separate databases for each “domain aspect”  Rather than a single monolithic database. Michael.Corsello@ieee.org http://Eclime.blogspot.com

36. CRF Developed Solution  Cloud Based Data Warehouse Michael.Corsello@ieee.org http://Eclime.blogspot.com

37. Maximize Value of Climate Data Michael.Corsello@ieee.org http://Eclime.blogspot.com

38. The real challenge with CC data is keeping track of metadata  Metadata is data about data  What about the metadata for the metadata?  Can the metadata be data itself?  There is no real “metadata”  It’s all about perspective  Metadata from one perspective is data in another  The data model is the key Michael.Corsello@ieee.org http://Eclime.blogspot.com

39. Metadata Examples  An important form of metadata is “chain of custody” (provenance)  Talks about the process by which data originates  What processing methods were used?  What was the source data?  Who did the work?  Another important form of metadata is descriptive  When was the sensor last calibrated?  What was the nominal error as defined by the manufacturer?  What is the temporal nature of the data (does it “expire”)?  What about licensing info?  Metadata can often be “linked” rather than “stored” Michael.Corsello@ieee.org http://Eclime.blogspot.com

41. The real Challenge with Climate Change?  We want the ONE true answer to Climate Change  The rest of the data is meaningless  Because the paradigm we work with is deterministic  We have a hard time dealing with uncertainty Michael.Corsello@ieee.org http://Eclime.blogspot.com

42. Cloud Computing Basics  Move computing from device oriented to resource oriented  Give me enough computing resources to get an answer  I don’t care where  Software as a Service  Software is delivered as an online service  Salesforce.com, Mint.com, Office 365  Platform as a Service  A software platform (e.g. Sharepoint, Drupal) is provided as a service  Your agency customizes the platform to your needs  Infrastructure as a Service  You rent “virtual machines” and set them up as you see fit  Basically a “virtual” computer  Add or remove machines “on- demand” Michael.Corsello@ieee.org http://Eclime.blogspot.com

43. Data Models Michael.Corsello@ieee.org http://Eclime.blogspot.com

45. Workflows  More data to manage as we create more data  All of our “final” data  Much of our “working” data Michael.Corsello@ieee.org http://Eclime.blogspot.com

46. Workflows  Management translates to  Ease of Access to Data  Analysis / Modeling with Data  Results & Reporting  Store Results for future use Michael.Corsello@ieee.org http://Eclime.blogspot.com

47. CRF Developed Solution  Developed Web and Desktop Tools to Access the Database(s) Michael.Corsello@ieee.org http://Eclime.blogspot.com