1. A Prototype for JDEM Science Data Processing, Erik Gottschalk 1 A Prototype for JDEM Science Data Processing Erik Gottschalk Fermilab On behalf of the JDEM Team at Fermilab: Stu Fuess, Steve Kent, Jim Kowalkowski, Igor Mandrichenko, Eric Neilsen, Marc Paterno, Vince Pavlicek, Vladimir Podstavkov

2. A Prototype for JDEM Science Data Processing, Erik Gottschalk 2 Overview Joint Dark Energy Mission (JDEM)  History Baryon Acoustic Oscillations (BAO)  Science driver for JDEM prototype  Slitless spectroscopy JDEM Demonstration Data Processing System (JDDPS)  Requirements  Architecture  Software technologies Summary

3. A Prototype for JDEM Science Data Processing, Erik Gottschalk History of JDEM 1998: Cosmic acceleration (Dark Energy) discovered using Type Ia Supernovae 2000: SuperNova Acceleration Probe (SNAP) proposed by LBNL 2003: Joint Dark Energy Mission (JDEM) created by NASA & DOE  JDEM was one of three probes in NASA’s Beyond Einstein program  Three concepts (SNAP, DESTINY, ADEPT) developed using a combination of 3 techniques (Baryon Acoustic Oscillations, Weak Lensing, Supernovae) 2007: BEPAC assessment committee recommended that JDEM probe “go first” 2008: NASA created the Science Coordination Group to merge the three concepts into a single concept, which was named “JDEM-Omega” 2009: NASA created Interim Science Working Group to develop lower cost mission concepts 2010: Astro2010 selected the Wide Field Infrared Survey Telescope (WFIRST) as the highest priority space telescope to study Dark Energy and Exoplanets  WFIRST is based on the JDEM-Omega concept  Note: European Space Agency is proceeding with Cosmic Visions program, which includes the Euclid mission to study Dark Energy (BAO and Weak Lensing) 3

4. A Prototype for JDEM Science Data Processing, Erik Gottschalk BAO & Slitless Spectroscopy Baryon Acoustic Oscillations (BAO) are observed as large-scale clustering of galaxies. BAO provides a “standard ruler” used to measure length scales in cosmology to study Dark Energy. Slitless spectroscopy is a technique that is used to determine the redshift of galaxies by placing a dispersing element (e.g. a prism) in front of a telescope camera. Each source is observed as a dispersed image. JDDPS is designed to provide the data processing framework for slitless spectroscopy data analysis software. 4 BAO analysis– Obtain spectra of all objects in a field using a prism, NO slit. H-alpha emission lines can be identified in spite of confusion. (From Euclid study report)

5. A Prototype for JDEM Science Data Processing, Erik Gottschalk Slitless Spectroscopy Workflow Generate simulated spectroscopic images and object lists (e.g. stars and galaxies) and introduce noise and cosmic rays. The simulated data are processed and results are compared to input data. Slitless spectroscopy data processing workflow includes four steps:  Spectroscopic Extraction  Position Fitter  Spectral Line Detection  Combine Roll Angles 5

6. A Prototype for JDEM Science Data Processing, Erik Gottschalk Slitless Spectroscopy Workflow Generate simulated spectroscopic images and object lists (e.g. stars and galaxies) and introduce noise and cosmic rays. The simulated data are processed and results are compared to input data. Slitless spectroscopy data processing workflow includes four steps:  Spectroscopic Extraction  Position Fitter  Spectral Line Detection  Combine Roll Angles For JDDPS these are the “science algorithms” in the demonstration system. 6

7. A Prototype for JDEM Science Data Processing, Erik Gottschalk JDDPS Stakeholder Requirements JDDPS is designed to operate in a distributed computing environment, with “science algorithms” distributed across processing nodes. The design is based on 40 stakeholder requirements. The stakeholder requirements are functional requirements that are categorized according to different types of stakeholders:  Scientist algorithm developer  Scientist pipeline developer  Scientist data analyst  Operator Identifying stakeholders and defining their needs is a key feature of JDDPS. 7

8. A Prototype for JDEM Science Data Processing, Erik Gottschalk JDDPS Architecture The JDDPS architecture is designed to provide services (e.g. file I/O and database access services), and workflow design, planning and scheduling tools. 8

9. A Prototype for JDEM Science Data Processing, Erik Gottschalk JDDPS Architecture The JDDPS architecture is designed to provide services (e.g. file I/O and database access services), and workflow design, planning and scheduling tools. Quality control is an essential feature of JDDPS that addresses the need for science data quality monitoring AND execution environment monitoring. 9

10. A Prototype for JDEM Science Data Processing, Erik Gottschalk JDDPS Technology Choices Quality control system based on OpenSplice DDS  Open-source implementation of Data Distribution Service (DDS)  Publish/subscribe real-time messaging software with Quality of Service settings for reliability and fault tolerance  http://www.opensplice.com/ Workflow management system based on Kepler  Some workflow systems we evaluated (Swift, Pegasus, Askalon) did not satisfy requirements  Kepler is open source and based on the Ptolemy II execution engine  https://kepler-project.org/ Database software (PostgreSQL and Greenplum)  Greenplum is a massively parallel database system (modified PostgreSQL)  http://www.greenplum.com/ JDDPS deployed in Fermilab distributed computing environment  FermiCloud (currently running OpenNebula and Eucalyptus) 10

11. A Prototype for JDEM Science Data Processing, Erik Gottschalk JDDPS Deployment Model JDDPS will be deployed in the FermiCloud environment to run campaigns (for example, processing slitless spectra for a 24-hour period of data collection). 11 Messaging System Functions User Functions

12. A Prototype for JDEM Science Data Processing, Erik Gottschalk JDDPS Deployment Model JDDPS will be deployed in the FermiCloud environment to run campaigns (for example, processing slitless spectra for a 24-hour period of data collection). 12 Messaging System Functions User Functions

13. A Prototype for JDEM Science Data Processing, Erik Gottschalk Noteworthy Aspects of JDDPS We adopted a more rigorous systems engineering approach to software framework development compared to what is usually done for HEP computing (e.g. CDF, D0, CMS).  Identified stakeholders  Determined stakeholder needs by conducting requirements workshops  Implemented a formal requirements management and traceability process We included quality control from the beginning by addressing the need for science data quality monitoring AND execution environment monitoring as an essential feature of JDDPS. We included workflow management aspects from the beginning, so that scientists will be able to assemble, configure and run science data processing pipelines AND campaigns, all within one system. 13

14. A Prototype for JDEM Science Data Processing, Erik Gottschalk Summary We are developing a demonstration system (JDDPS) for science data processing, specifically for astrophysics. The motivation (science driver) is to provide a framework for simulations and data analyses for Baryon Acoustic Oscillations. JDDPS is designed to run in a cloud-computing environment (e.g. FermiCloud). Our technology choices reflect the need for quality control, workflow management, and large (~ 100 TB) databases. We have implemented a formal requirements management and traceability process to develop JDDPS. 14

15. A Prototype for JDEM Science Data Processing, Erik Gottschalk JDEM Slitless Spectroscopy Simulations Simulate slitless spectra and include noise and cosmic rays to generate images for analysis. 15 Perfect imageAdd photon and random noise Add cosmic rays Up-the-ramp cleaned image

16. A Prototype for JDEM Science Data Processing, Erik Gottschalk JDEM Slitless Spectroscopy Simulations 16 ● We have implemented simulations for a single NIR detector – Generate images including noise and cosmic-ray artifacts – Extract spectra and measure redshifts – Measure success rate and compare results to requirements ● We have implemented up-the-ramp sampling to remove cosmic rays (processing may need to be done on the spacecraft) ● Future: – Implement simulations for multidetector arrays – Simulate one full day of data collection