Presentation is loading. Please wait.

Presentation is loading. Please wait.

Big Data and Cloud Computing Expert Panel

Published byRaymond Larocque Modified over 5 years ago

Similar presentations

Presentation on theme: "Big Data and Cloud Computing Expert Panel"— Presentation transcript:

1 Big Data and Cloud Computing Expert Panel
Assessing Cloud-Native Architectures for EOSDIS Applications Dan Pilone - The material is based upon work supported by the National Aeronautics and Space Administration under Contract Number NNG15HZ39C

2 Earth Science Instruments on ISS:
Formulation Implementation Primary Ops Extended Ops Landsat 9 PACE NI-SAR SWOT TEMPO JPSS-2 (NOAA) RBI, OMPS-Limb Sentinel-6A/B GRACE-FO (2) ICESat-2 Earth Science Instruments on ISS: RapidScat, CATS, LIS, SAGE III (on ISS), TSIS-1, OCO-3, ECOSTRESS, GEDI, CLARREO-PF CYGNSS ISS SORCE, TCTE (NOAA) NISTAR, EPIC (NOAA’S DSCOVR) QuikSCAT EO-1 Landsat 7 (USGS) SMAP Terra Fleet - Earth Science Division - Missions (with color key; editable) Dec. 15, 2015 Credit: NASA/J. Mottar SLIDE INFO UDATE EJD PER FREILICH DECISIONS FEB 4, 2016 Changed “Altimetry FO (Formulation in FY16) to “Sentinal-6A/B” On Landsat 9, removed comma and TIRFF Change I was going to make – “NISTAR, EPIC (NOAA’s DSCOVR) to green primary ops was already corrected Dec. 15, 2015 – Updated: TEMPO changed to Implementation phase; NISTAR, EPIC changed to Primary Ops phase; ; OCO-3 and ECOSTRESS changed to Implementation phase; Aquarius removed; slide now fully editable. JMottar May 18, 2015 – Added to SMD Multimedia Library. May 18, 2015 – Updated the outdated 2014 version: TRMM was removed; NISTAR, EPIC (NOAA's DSCOVR) were added; SMAP icon was updated; moon at top left was removed; color key moved to top left.-JMottar ID: I-EA (image) ID: SL-EA (slide) Suomi NPP (NOAA) Aqua CloudSat Landsat 8 (USGS) CALIPSO GPM Aura GRACE (2) OSTM/Jason 2 (NOAA) OCO-2

3 Major Components of the Earth Science Data Systems Program
Earth Observing System Data and Information System (EOSDIS) Core systems for processing, ingesting and archiving data for the Earth Science Division Competitively Selected Programs Making Earth System Data Records for Use in Research Environments (MEaSUREs) 27 projects selected in 2012 Advancing Collaborative Connections for Earth System Science (ACCESS) 12 projects selected in 2013 International and Interagency Coordination and Development CEOS WGISS OSTP Climate Data Initiative NASA-ESA Bilateral GEO/USGEO Tjf 2/11/16: OSTM = Ocean Surface Topography Mission NPP = National Polar-orbiting Partnership ISS = International Space Station SMAP = Soil Moisture Active Passive SAC-D =OCO-2 = Orbiting Carbon Observatory-2 1/3/2019

4 12 Discipline Oriented DAACs
To do this, we distribute that data over 12 discipline oriented DAACs. Cyrosphere, Land processes, snow and ice, socio-economic, etc. This sets the stage for our problem…

5 15+ years of Earth Science Data
Deletion of MODIS Collection 4 Products Landsat-7 Data Migrated from LPDAAC to USGS Addition of AQUARIUS and ICEBRIDGE products ALOS PALSAR & UAVSAR products ICESat MEASURES 2006 MEASURES 2012 TERRA AQUA AURA Addition of SNPP CERES&OMPS Total EOSDIS Accumulated Data Archive Volume (Petabytes) A petabyte is a lot of data: 1 PB is 20 million four-drawer filing cabinets full of text 1 PB is 13.3 years of HD-TV video 1.5 PB is the size of 10 billion photos on Facebook

6 EOSDIS Archive Growth Estimate
(Prime + Extended) Lots of assumptions in this chart. Subject to change...

7 EOSDIS Vision 2020 (cloud relevant)
Data Analysis at Scale Data and Processing Mobility Dataset Upgrading Virtual Collections Combining Data, Combining Tools

8 Evolution Review E&E Recommendation: The ESDIS Project should develop, implement and report on the outcome of prototypes to explore the advantages, risks, and costs of using commercial cloud environments for: Storage and data transfer […] Processing […] Improved data access […]

9 Existing prototyping efforts were restructured to investigate and test commercial cloud environments for essential EOSDIS and DAAC operations.

10 What does these consider?

11 Infrastructure, Scaling, Monitoring, Security, …

12 Development Model, Compliant Deployment, Operations, …

13 Application Architecture…

14 Systems Architecture

15 Outstanding Questions
Can operational NASA science data systems function effectively in a commercial cloud? Do commercial clouds offer increased efficiency within a NASA environment and/or provide additional functionality? How do costs compare to opportunities and risks? How can NASA efficiently partner with commercial cloud providers to improve services and usage of data and services?

16 Looking forward Need broader knowledge of cloud architectures and patterns (e.g. NGAP, serverless, 12 factor applications, etc.) Rapid pace of change Potentially dramatic change in data usage patterns What about Cloud Native data formats? What about Cloud Native services?

17 Dan Pilone // dan@element84.com
Thank you! Dan Pilone //

18 This material is based upon work supported by the National Aeronautics and Space Administration under Contract Number NNG15HZ39C.

Download ppt "Big Data and Cloud Computing Expert Panel"

Similar presentations

Kathy Fontaine CEOS WGISS 24, DLR Oberpfaffenhofen Munich October 19. 2007 NASA Agency Report.

Kathy Fontaine CEOS WGISS 24, DLR Oberpfaffenhofen Munich October NASA Agency Report.
Brian Killough NASA NASA Future Missions Summary CGMS-38 Meeting Delhi, India November 8-12, 2010.

Brian Killough NASA NASA Future Missions Summary CGMS-38 Meeting Delhi, India November 8-12, 2010.
Space-based Architecture for Climate Mary Kicza NOAA’s Assistant Administrator for Satellite and Information Services May 19, 2011.

Space-based Architecture for Climate Mary Kicza NOAA’s Assistant Administrator for Satellite and Information Services May 19, 2011.
BEDI -Big Earth Data Initiative

BEDI -Big Earth Data Initiative
USGS Core Data Stream Report Session 7: Baseline Global Observation Scenario SDCG-7 Sydney, Australia March 4 th – 6 th 2015.

USGS Core Data Stream Report Session 7: Baseline Global Observation Scenario SDCG-7 Sydney, Australia March 4 th – 6 th 2015.
1 NASA Weather Applications Program 2010 Program Overview John A. Haynes Program Manager, Weather Applied Sciences Program Earth Science Division Science.

1 NASA Weather Applications Program 2010 Program Overview John A. Haynes Program Manager, Weather Applied Sciences Program Earth Science Division Science.
NASA AGENCY REPORT Dr. Eric J. Lindstrom Physical Oceanography Program Scientist Earth Science Division Science Mission Directorate MARCH 12, 2007.

NASA AGENCY REPORT Dr. Eric J. Lindstrom Physical Oceanography Program Scientist Earth Science Division Science Mission Directorate MARCH 12, 2007.
1 January 17, 2007 NASA IPY Interest NASA has a strong interest in IPY themes Present environmental status of the polar regions Quantify and understand.

1 January 17, 2007 NASA IPY Interest NASA has a strong interest in IPY themes Present environmental status of the polar regions Quantify and understand.
1 Air Quality Applied Sciences Team (AQAST) Team Meeting – June 2015 John A. Haynes, MS Program Manager, Health and Air Quality Applied Sciences Program.

1 Air Quality Applied Sciences Team (AQAST) Team Meeting – June 2015 John A. Haynes, MS Program Manager, Health and Air Quality Applied Sciences Program.
Next Generation Application Platform (NGAP) Andrew Mitchell WGISS-39 Tsukuba, Japan Monday, May 11, 2015 1.

Next Generation Application Platform (NGAP) Andrew Mitchell WGISS-39 Tsukuba, Japan Monday, May 11,
Ensuring Credibility of NASA’s Earth Science Data Session Title: IN11D. Data Curation, Credibility, Preservation Implementation, and Data Rescue to Enable.

Ensuring Credibility of NASA’s Earth Science Data Session Title: IN11D. Data Curation, Credibility, Preservation Implementation, and Data Rescue to Enable.
NASA, CGMS-40, November 2012 Coordination Group for Meteorological Satellites - CGMS Status report on the current and future satellite systems by NASA.

NASA, CGMS-40, November 2012 Coordination Group for Meteorological Satellites - CGMS Status report on the current and future satellite systems by NASA.
NASA’s Earth Science Division Flight Overview Bradley D. Doorn Program Manager Earth Science Division, Science Mission Directorate PECORA 18.

NASA’s Earth Science Division Flight Overview Bradley D. Doorn Program Manager Earth Science Division, Science Mission Directorate PECORA 18.
NOAA Satellite Conference April 8-12, 2013 Mary Kicza Assistant Administrator for Satellite & Information Services.

NOAA Satellite Conference April 8-12, 2013 Mary Kicza Assistant Administrator for Satellite & Information Services.
EUM/SIR/VWG/12/0375, v1, 10 July 2012 Coordination Group for Meteorological Satellites - CGMS EUM/SIR/VWG/12/0375, v1, 10 July 2012 Coordination Group.

EUM/SIR/VWG/12/0375, v1, 10 July 2012 Coordination Group for Meteorological Satellites - CGMS EUM/SIR/VWG/12/0375, v1, 10 July 2012 Coordination Group.
Michael H. Freilich 21 April 2009 Earth Science Division Overview and Status.

Michael H. Freilich 21 April 2009 Earth Science Division Overview and Status.
NASA, CGMS-41, July 2013 Coordination Group for Meteorological Satellites - CGMS Status report on the current and future satellite systems by NASA Presented.

NASA, CGMS-41, July 2013 Coordination Group for Meteorological Satellites - CGMS Status report on the current and future satellite systems by NASA Presented.
NASA Earth Science Technology Update Presented to Doppler Wind Lidar Working Group April 28, 2015 George J. Komar Associate Director/Program Manager Earth.

NASA Earth Science Technology Update Presented to Doppler Wind Lidar Working Group April 28, 2015 George J. Komar Associate Director/Program Manager Earth.
1 GEO Energy Expert Meeting: Perspectives on Work Plan Tasks August 28-29, 2006 Richard Eckman NASA Applied Sciences Program Science Mission Directorate.

1 GEO Energy Expert Meeting: Perspectives on Work Plan Tasks August 28-29, 2006 Richard Eckman NASA Applied Sciences Program Science Mission Directorate.
Center for Satellite Applications and Research (STAR) Review 09 – 11 March 2010 Image: MODIS Land Group, NASA GSFC March 2000 Mission and Science Support.

Center for Satellite Applications and Research (STAR) Review 09 – 11 March 2010 Image: MODIS Land Group, NASA GSFC March 2000 Mission and Science Support.

Similar presentations

Ads by Google