Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science

The Problem An expensive facility with many customers The customers have different needs Expert knowledge is needed to operate the facility Operate the facility at low cost with good service

The Need Increased Productivity Fast Response Continuous Improvement

Required Functions Job Specification Job Planning Job Execution Delivery of Results Archive

Kinds of Knowledge Technical knowledge Systems knowledge Project management knowledge Business knowledge All are necessary for a successful high-value project

Astronomical Observatories Complex and Expensive System Remote Location Shared Resource Leading-edge Technology Long System Life

Spirex Telescope with Abu Camera in Antarctica

Winterover on arrival at the pole….

Six Months Later….

Abu Pipeline Astronomer Science Center Analysis Center Data Archive Observation Center NOAO/RIT South Pole White Sands/RIT RIT

IDL, C,...XMLFITS, TIFF,... Pipeline Database Image Data CAML Documents Algorithms CAML Controller Java, ORB Operator Interface Network Interface

Large Magellanic Cloud 30 Doradus nebula

Star formation NGC 6334 Abu/Spirex 1998

Productivity Challenge 960 observing hours per year 160 observing flights per year Variety of instruments Large community of observers Conclusion: The traditional “PI” approach is unworkable. A new system for efficiency and relentless observation is needed.

The DCS Mission The DCS shall provide the infrastructure to support the operation of facility instruments on SOFIA using best current knowledge and tools, and supporting continuous improvement in an efficient, extensible, and modular architecture.

SOFIA Facility Instruments

SOFIA Team Players and Strengths IPAC IRAS, ISO, 2Mass, WIRE, SIRTF Archiving, Observation planning ARC Smart scheduling activities for instruments and telescopes for HST, Mars Rover, Deep Space 1 GSFC HST and NGST development HAWC and SAFIRE instrument architecture Object Oriented development RIT South Pole Data Pipelining HAWC development UCLA Keck FLITECAM

SOFIA Data Cycle

Science Center Activities Community Development and Support Maintain Science Data Archives General Investigator Support Maintain Center Information Infrastructure System Documentation, Observatory Performance Information, Observation Constraints Calibration Information, Atmospheric Models Maintain FI Tools Facility Software Management, Proposal Development Tools, Proposal Evaluation Tools Observation Planning Tools, Observing Tools, Flight Planning Tools Data Reduction Tools -- Pipeline analysis modules Coordinate with Observatory Operations

User Needs Archival Research Get Information on Observatory Capabilities Get Information on Instrument Capabilities Build and submit a Proposal Build an Observation Plan Build a Data Reduction Pipeline Access Observation Results Interpret Experimental Results Publish

User Access Requirements User access to all user functions shall be supported via the Web 1.Access to archived science products 2.Access to supported external archives 3.Access to observatory capabilities information 4.Access to instrument capabilities information 5.Proposal development, submission and tracking 6.Observation timing estimation 7.Atmospheric modeling resources 8.Access to observation data 9.Access to flight and housekeeping data 10.Access to flight logs 11.Data processing 12.Published document archiving

System Requirements User Access Administrator Access Data Storage and Retrieval Data Processing Planning and Scheduling Documentation Support

Data Reduction Resources FSI & MCS Interfaces Task Library DCS Data Storage User Interaction Layer prefs, context requests, results run an experiment raw and processed data AOTs, exp info, raw data, obs logs.. various searches run a pipeline SSMOC Internet

DCS Data Storage DCS Data Storage Interface DB Locator And Selector User DB Accounts Groups Privileges Contexts Preferences Experiment DB Templates Planned Accepted Completed Reviews Comments Data DB Raw Instr Data OBS Logs Processed Data Instr DB GSFC’s IML FSI Chain Lists History of Instr Use Provides common DCS interface to all storage, translates DCS data access to standard DB interface methods Convenient switch for criteria such as: load balancing, fault tolerance

The RIT Astronomy Pipeline Team Harvey Rhody, Scott Lawrence, Erika Tolar, Stephen Schaeffer, James DeFelice, Adith Chandrasekhar, J. Fernando Naveda, Bob Krzaczek (standing) and Ian Gatley. Not shown: Jim Chauvin, Brian Perry, Joel Kastner.