SSC SI Data Processing Pipeline Plans Tom Stephens USRA Information Systems Development Manager SSSC Meeting – Sept 29, 2009.

Slides:



Advertisements
Similar presentations
Testing Relational Database
Advertisements

The 4 T’s of Test Automation:
System Development Life Cycle (SDLC)
Requirements Specification and Management
ICT Class System Life Cycle.  Large systems development projects may involve dozens of people working over several months or even years, so they cannot.
Lecture # 2 : Process Models
Alternate Software Development Methodologies
SOFIA DCS History and Overview Ian Gatley. SOFIA March DCS Preliminary Design Review2 The South Pole CARA Project: A DCS demonstration A data.
DCS Architecture Bob Krzaczek. Key Design Requirement Distilled from the DCS Mission statement and the results of the Conceptual Design Review (June 1999):
1 Universities Space Research Association SOFIA Program Options SSSC 18 February 2010 Mina Cappuccio SOFIA Science Project Manager NASA Ames Research Center.
Slide 1 Sterling Software Peter Sharer Sterling Software.
System Development Life Cycle (SDLC)
The Software Product Life Cycle. Views of the Software Product Life Cycle  Management  Software engineering  Engineering design  Architectural design.
Design, Implementation and Maintenance
Planning. SDLC Planning Analysis Design Implementation.
Release & Deployment ITIL Version 3
MIRI software; implementation plan Peter Roelfsema.
©Ian Sommerville 2004Software Engineering, 7th edition. Chapter 17 Slide 1 Extreme Programming.
Introduction to Information System Development.
Introduction to RUP Spring Sharif Univ. of Tech.2 Outlines What is RUP? RUP Phases –Inception –Elaboration –Construction –Transition.
Chapter 2 The process Process, Methods, and Tools
-Nikhil Bhatia 28 th October What is RUP? Central Elements of RUP Project Lifecycle Phases Six Engineering Disciplines Three Supporting Disciplines.
RUP Implementation and Testing
SSSC 02/18/2010 P. Marcum Science Utilization Policies SOFIA SCIENCE UTILIZATION POLICIES Pamela M. Marcum SOFIA Project Scientist SSSC Feb 19, 2010.
Understand Application Lifecycle Management
1 ISA&D7‏/8‏/ ISA&D7‏/8‏/2013 Systems Development Life Cycle Phases and Activities in the SDLC Variations of the SDLC models.
SPACE TELESCOPE SCIENCE INSTITUTE Operated for NASA by AURA COS Pipeline Language(s) We plan to develop CALCOS using Python and C Another programming language?
1 Scope of Software Engineering Somnuk Keretho, Assistant Professor Department of Computer Engineering Faculty of Engineering, Kasetsart University
BTEC Unit 06 – Lesson 08 Principals of Software Design Mr C Johnston ICT Teacher
Extreme/Agile Programming Prabhaker Mateti. ACK These slides are collected from many authors along with a few of mine. Many thanks to all these authors.
Page No. 1 Kelvin Nichols Payload Operations and Integration Center EO50 Delay Tolerant Networking (DTN) Implementation on the International Space Station.
Rapid software development 1. Topics covered Agile methods Extreme programming Rapid application development Software prototyping 2.
Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.
CS 111 – Nov. 22 Chapter 7 Software engineering Systems analysis Commitment –Please read Section 7.4 (only pp ), Sections –Homework #2.
Systems Life Cycle A2 Module Heathcote Ch.38.
 System Development Life Cycle System Development Life Cycle  SDLC Phases SDLC Phases Phase 1: Preliminary Investigation Phase 2: Feasibility Study.
Topics Covered Phase 1: Preliminary investigation Phase 1: Preliminary investigation Phase 2: Feasibility Study Phase 2: Feasibility Study Phase 3: System.
Cmpe 589 Spring 2006 Lecture 2. Software Engineering Definition –A strategy for producing high quality software.
March 2004 At A Glance autoProducts is an automated flight dynamics product generation system. It provides a mission flight operations team with the capability.
Connecting with Computer Science2 Objectives Learn how software engineering is used to create applications Learn some of the different software engineering.
Reduce Development and Testing Time on Embedded Space Programs With Auto- Generated Code Software Engineer Northrop Grumman Electronic Systems Matthew.
Java Software Solutions Lewis and Loftus Chapter 15 Copyright 1997 by John Lewis and William Loftus. All rights reserved. 1 Software Development Process.
1 Proposal and Observation Handling Ravi Sankrit (User Support Scientist) SSSC May 11, 2011.
Requirements Engineering Requirements Engineering in Agile Methods Lecture-28.
The Implementation of BPR Pertemuan 9 Matakuliah: M0734-Business Process Reenginering Tahun: 2010.
1)History of water fall model. 2)Features of water fall model. 3)Phase of water fall model. 4)Brief description of phases. 5)Advantages. 6)Disadvantages.
ANALYSIS PHASE OF BUSINESS SYSTEM DEVELOPMENT METHODOLOGY.
An Agile Requirements Approach 1. Step 1: Get Organized  Meet with your team and agree on the basic software processes you will employ.  Decide how.
ELECTRONIC SERVICES & TOOLS Strategic Plan
1 Universities Space Research Association Martin Ruzek SSSC February 18, 2010 FIFI LS EOOP Extended Observing Opportunity Program Operating FIFI LS in.
1 Universities Space Research Association Science Instrument Pipeline Requirements SSSC 2009 September 29 Erick Young Director SOFIA Science Mission Operat.
Overview of the Final Report and Findings from the Review of Sampling Methods in Extrapolated New Base-Year Generation Studies May 11-12, 2004.
The information systems lifecycle Far more boring than you ever dreamed possible!
LECTURE 5 Nangwonvuma M/ Byansi D. Components, interfaces and integration Infrastructure, Middleware and Platforms Techniques – Data warehouses, extending.
Accounting systems design & evaluation 9434SB 18 March 2002.
 System Requirement Specification and System Planning.
MANAGEMENT INFORMATION SYSTEM
UIF INFORMATION TECHNOLOGY ENHANCEMENTS UPDATE May 9, 2005 Presented by: Bill Reynen Susan Restad
HST and JWST Pipelines and Reference Files
Calibration meeting summary
Enterprise Algorithm Change Process
System Development Life Cycle (SDLC)
System Development Life Cycle (SDLC)
Software Development Process
System Development Life Cycle (SDLC)
Observatory Science Operations
Observatory Science Operations
Extreme Programming.
Chapter 10: Software Engineering
Presentation transcript:

SSC SI Data Processing Pipeline Plans Tom Stephens USRA Information Systems Development Manager SSSC Meeting – Sept 29, 2009

Going Forward There’s more work to do –Increased scope of existing pipelines –Additional pipelines for non FSI instruments –NASA is increasing the project management requirements for the pipeline development The goal is for the Science Center to shoulder the vast majority of the burden of this new work –We will deal with the increased software development requirements –We will take on more user support functions in regards to general investigators –Allow the instrument teams to focus on the instrument and their science However, some increased involvement from the instrument teams will be required.

Centralizing the Pipeline Development Final production pipelines will be implemented at the Science Support Center (SSC) –Goal is to develop a unified system that is easier and more efficient to develop, use, maintain and manage Benefits –Standardized code base in a controlled set of languages and standards Initially C/C++, IDL, Python and JAVA Eases maintenance Reduces learning curve for new development team members –Unified code base Maximize reuse of pipeline components –Simplify development –Ease maintenance –Unified interface Probable that calibration pipelines cannot be fully automated Reduces friction when operators need to run different pipelines –Improved project management Maximize efficient use of resources Ability to focus resources on problematic/challenging areas

Roles and Responsibilities – Instrument Teams Provide algorithms for data reduction –Can be in any “reasonable” format Teams have software they are using for their own data reduction to do their science, this can be provided “as is” to help build the official pipelines Science Center staff will translate what is provided into pipeline code to run in the DCS –Includes documentation on software and algorithms –Also includes sample test data Work with SSC staff to validate implemented pipelines Work with SSC staff to support guest investigators using instrument for observing –This will proceed along the currently established plans The goal is for the instrument teams to not need to provide substantive additional support over what they are currently contracted to do

Roles and Responsibilities – Science Center Responsible for final implementation of pipeline code –Coding –Integration into the DCS –Documentation –Testing/Validation –Maintenance Responsible for ensuring suitable calibration data is taken for pipeline reduction Operates pipelines and validates output science data

SSC-SI Team Interface Points Each instrument will have a primary and backup Instrument Scientist at the SSC –Understands the instrument –Works with instrument teams to develop pipeline algorithms –Assists in validation of developed pipeline –Help support guest investigators with observation planning and archive data –Helps to define calibration observations –Provide SSC institutional knowledge about instrument One or more software developers will also work with the instrument teams, through the instrument scientist, to implement and test the data pipeline code. Implemented software will be provided back to instrument teams for their own use if desired.

Initial Impact – FSI teams Initial pipeline development will proceed as currently planned –Pipelines will be developed and delivered to SSC –SSC will take ownership and be responsible for future maintenance and changes We will want to get preliminary deliveries of your pipeline software sooner rather than later –We need to work out the mechanism for delivery –Integrate software with the DCS early and often –SSC staff are willing to assist with development if desired or needed

Initial Impact – Other Instruments Increased interaction between SSC staff and SI team –Improve SSC understanding of instrument –Improve SSC understanding of data analysis Work with SSC staff to develop algorithms/sample code for data processing –Use “whatever” is comfortable to instrument teams –Will take already existing tools/software –SSC will do the work of fitting the algorithms into the DCS pipelining system

Longer Term Impacts Focuses on maintenance aspects of instrument and pipeline development –Should not be major additional effort over current plans Instrument teams provide input to SSC during operation/maintenance period after initial pipelines developed –Improved algorithms/techniques for data reduction/calibration –Pipeline modification due to changes in instrument response or capabilities –Work with SSC staff to validate implementation and data produced by processing pipelines SSC continues to be responsible for actual implementation, operation and maintenance

Staffing This is an increase in the scope of the SSC’s pipeline development work –More and expanded pipelines –Increased software development process overhead –Additional staff will be required to accomplish this work SSC is working on a detailed staffing plan to address the development effort Details are still being worked with NASA Final development timeline and impact to SI teams will depend on the final details

Timeline Goal is to have a fully functioning pipeline implemented for each instrument as they start their official commissioning flights after the 2011 down time –However we’d like to get the existing FORCAST pipeline installed in the DCS for early science as a test for the SSC Not an Early Science Requirement Help us identify and address issues with integration Timing of commissioning will drive ramp-up of pipeline development effort and staff over the next two years –Start with re-integration of FORCAST & FLITECAM pipelines –Integrate FIFI LS pipeline –Using “lessons learned”, expand work to implementation of expanded pipelines and develop pipelines for other instruments –For a given instrument we plan to start implementation ~2 years before commissioning flights are scheduled to begin

Summary There is a lot of work to be done to implement the SMOR-II recommended pipelines The majority of this effort will fall on the SOFIA Science Center We do not want to place any additional burdens on the instrument teams than are absolutely necessary We are developing detailed plans of how to implement this work

Backup slides

High Level Development Process Pipeline/algorithm delivery from SI Team SSC Pipeline Implementation Error and Issue Reports Pipeline Validation Improvements and updates developed by SI teams and SSC Pipelines deployed to production