Presentation is loading. Please wait.

Presentation is loading. Please wait.

Data Science Publication for NSF Polar Cyberinfrastructure Dr. Brand Niemann Director and Senior Data Scientist/Data Journalist Semantic Community

Similar presentations


Presentation on theme: "Data Science Publication for NSF Polar Cyberinfrastructure Dr. Brand Niemann Director and Senior Data Scientist/Data Journalist Semantic Community"— Presentation transcript:

1 Data Science Publication for NSF Polar Cyberinfrastructure Dr. Brand Niemann Director and Senior Data Scientist/Data Journalist Semantic Community http://semanticommunity.info/ http://www.meetup.com/Federal-Big-Data-Working-Group/ http://semanticommunity.info/Data_Science/Federal_Big_Data_Working_Group_Meetup November 4, 2014 1

2 Preface Some prep work is already underway (if you scour the Open Science Codefest site you will find some) to prepare some datasets of relevance to the Polar community. We will provide some of this prepared data to interested parties ahead of the workshop in the next few weeks in case folks want to start hacking early. We will tweet under the hash tag: #nsfpolardatavis 2 Source: Chris Mattmann

3 Overview Build the knowledge base (in MindTouch) and spreadsheet (in Excel) first, which then makes the Spotfire (data browser) application easier to “storify” the results. Follow the Cross-Industry Data Mining Standard by: – 1 Business Understanding (of the Hackathon), – 2 Data Understanding (by mining the Sessions), – 3 Data Preparation (by screen scraping and downloading), – 4 Modeling (enough data for statistical significance?), – 5 Evaluation (How collected?, Where stored?, What results?, and Believe them?; and – 6 Deployment (Story and Demo). The documentation will be in the form of the Data Science Publication for NSF Polar Cyberinfrastructure. My goal is to see if I can integrate and federate these multiple data sources. 3

4 Data Science for Business: Data Mining Process 4 Source: Data Science for Business: Chapter 2, 2014

5 Data Science for NSF Polar Cyberinfrastructure: Knowledge Base 5 Data Science for NSF Polar Cyberinfrastructure

6 Possible Data Sets Scour the Open Science Codefest: – https://github.com/NCEAS/open-science- codefest/issues/26 https://github.com/NCEAS/open-science- codefest/issues/26 The Polar Data Catalogue (YES): – https://polardata.ca/ https://polardata.ca/ BCO-DMO (YES): – http://www.bco-dmo.org/ http://www.bco-dmo.org/ Polar Hub (NO): – http://polar.geodacenter.org/polarhub/ http://polar.geodacenter.org/polarhub/ The AMRC at University of Wisconsin-Madison (YES): – ftp://amrc.ssec.wisc.edu/pub/requests/DVPC/ ftp://amrc.ssec.wisc.edu/pub/requests/DVPC/ 6

7 Open Science Codefest: NASA/NSF/NSIDC Data Sets NASA Antarctic Master Directory – A master directory for arctic data sets http://gcmd.gsfc.nasa.gov/KeywordSearch/Keywords.do?Por tal=amd&KeywordPath=Parameters|CRYOSPHERE&Metadat aType=0&lbnode=mdlb2 http://gcmd.gsfc.nasa.gov/KeywordSearch/Keywords.do?Por tal=amd&KeywordPath=Parameters|CRYOSPHERE&Metadat aType=0&lbnode=mdlb2 NSF ACADIS Gateway – NSF data repository for arctic/polar data https://www.aoncadis.org/home.htm NSIDC Arctic Data Explorer – National Snow and Ice Data Center repository http://nsidc.org/acadis/search/ 7 Source: Link to presentation given at Open Science CodefestLink to presentation given at Open Science Codefest

8 Polar Data Catalogue: Home Page 8 https://polardata.ca/

9 Polar Data Catalogue: Collections 9 https://polardata.ca/pdcsearch/

10 Polar Data Catalogue: Search 10 https://polardata.ca/pdcsearch/

11 Polar Data Catalogue: Canadian Lake Ice Database 11 https://polardata.ca/pdcsearch/PDC_Metadata_Data_Download.ccin?action=downloadPDCData&ccin_ref_number=1821&fileLoc=/pdc/ccin/1821/lakeice/CCIN1821_20030925_CID_BDCG_Ver2003_1.zip 11 MB MDB

12 Polar Data Catalogue: Sea Ice Thickness in Southern Beaufort Sea 12 https://polardata.ca/pdcsearch/PDC_Metadata_Data_Download.ccin?action=downloadPDCData&ccin_ref_number=11470&fileLoc=/pdc/brea/11470/CCIN11470_20130827_GIS_DATA_Sea_Ice_Thickness_2012.zip Downloaded 5 Files 4 Text and 1 ZIP (Shape) 1.3 MB

13 Polar Data Catalogue: Spreadsheet 13 http://semanticommunity.info/@api/deki/files/31201/NSFPolarCI.xlsx?origin=mt-web Canadian Lake Ice Database Sea Ice Thickness in Southern Beaufort Sea

14 BCO-DMO 14 http://www.bco-dmo.org/ Tutorial PDF

15 Data Access Tutorial 2014 OCB PI Summer Workshop How to Submit Data Data access: TEXT-BASED SEARCH scenario 1: – You have a general idea of what you are looking for. Data access: MAP BROWSE scenario 2: – You are interested in data from a particular geographic region. Data access: MAP KEYWORD SEARCH scenario 3: – You are interested in data of a particular type from a particular geographic area. Data access: MAP SEMANTIC SEARCH scenario 4: – You have an idea what you are looking for, but you do not know the Program, Project, or Deployment name. Glossary of Terms Acknowledgments Follow BCO-DMO 15 http://www.bco-dmo.org/files/bcodmo/OCB-Tutorial.pdf My Question: Could Spotfire do all of this?

16 BCO-DMO Datasets 16 http://www.bco-dmo.org/datasets

17 BCO-DMO MapServer Geospatial Interface http://mapservice.bco-dmo.org/mapserver/maps-ol/index.php 17

18 Polar Hub: A Global Hub for Polar Data Discovery 18 http://polar.geodacenter.org/polarhub/ My Question: Where is the Data?

19 The AMRC at University of Wisconsin-Madison NameSizeDate Modified [parent directory] Ant_IR_area/9/30/14, 8:18:00 PM Ant_IR_netCDF/9/30/14, 8:37:00 PM AWS_dat_MAY_2014/Text Files: Spotfire?9/30/14, 3:23:00 PM AWS_q10_MAY_2014/Text Files: Spotfire?9/30/14, 3:24:00 PM AWS_q1h_MAY_2014/Text Files: Spotfire?9/30/14, 3:25:00 PM AWS_q3h_MAY_2014/Text Files: Spotfire?9/30/14, 3:27:00 PM AWS_r_MAY_2014/Text Files: Spotfire?9/30/14, 3:22:00 PM readme.txt4.6 kB10/6/14, 6:37:00 PM 19 Index of /pub/requests/DVPC/ ftp://amrc.ssec.wisc.edu/pub/requests/DVPC/

20 Data Science for NSF Polar Cyberinfrastructure: Spreadsheet Knowledge Base 20 http://semanticommunity.info/@api/deki/files/31201/NSFPolarCI.xlsx?origin=mt-web

21 Data Science for NSF Polar Cyberinfrastructure: Spotfire Cover Page 21 Web Player

22 Data Science for NSF Polar Cyberinfrastructure: Spotfire Visualizations 22 Web Player


Download ppt "Data Science Publication for NSF Polar Cyberinfrastructure Dr. Brand Niemann Director and Senior Data Scientist/Data Journalist Semantic Community"

Similar presentations


Ads by Google