The Data Processing Management Architecture Professor Harvey Rhody Laboratory for Imaging Algorithms and Systems Chester F. Carlson Center for Imaging Science Rochester Institute of Technology Professor Zhigang Zhu City College Visual Computing Laboratory Center for Perceptual Robotics, Intelligent Sensors, and Machines City College of the City University, New York C C V C L

Our History CARA’s South Pole Infra-red Explorer  Limited Communication Windows  Tasking an Instrument at a Distance  Data Pipelining  Distribution to Worldwide Audience of Astronomers  Limited Communication Windows  Tasking an Instrument at a Distance  Data Pipelining  Distribution to Worldwide Audience of Astronomers SPIREX/Abu Data Pipelining

Our History NASA’s Stratospheric Observatory for Infra-red Astronomy (SOFIA)  Planning and Tasking of Many Different Instruments  Support NASA General Investigator Program  Data Pipelining  Worldwide Audience  Strong Emphasis on Capturing and Sharing Expertise of Instruments, Reduction Software, and Data  Planning and Tasking of Many Different Instruments  Support NASA General Investigator Program  Data Pipelining  Worldwide Audience  Strong Emphasis on Capturing and Sharing Expertise of Instruments, Reduction Software, and Data

Our History Wildfire Airborne Sensor Platform  Multi-band, Multi-sensor  COTS hardware  Real-time Registration, Ortho-Rectification, and Mosaic Generation  Live Data Download and Control  Distributed Software Modules, Staged in Aircraft LAN and/or Ground  Multi-band, Multi-sensor  COTS hardware  Real-time Registration, Ortho-Rectification, and Mosaic Generation  Live Data Download and Control  Distributed Software Modules, Staged in Aircraft LAN and/or Ground WASP

Our History SPIREX/Abu Data Pipelining

Typical Environment Multiple Projects, Multiple People

Typical Environment Typical Problems  How do groups share large amount of data?  How does code developed by one researcher get executed by another?  How can members shared special purpose computing resources?  How do groups share large amount of data?  How does code developed by one researcher get executed by another?  How can members shared special purpose computing resources?

DPMA Data Processing Management Architecture Entities Hosted by DPMA  Projects  Data  Algorithms  Applications  Specialized Computing Resources  Histories of Activity Entities Hosted by DPMA  Projects  Data  Algorithms  Applications  Specialized Computing Resources  Histories of Activity

DPMA Data Processing Management Architecture

DPMA Evolution of Data within a Project

DPMA The Role of Agents

DPMA Human Agents

DPMA Simple Agents

DPMA Special Agents

DPMA Agents, Projects, and Users

DPMA An Exploration Project

DPMA Identification of a Successful Approach

DPMA Focusing on the Work Performed

DPMA Capturing Success for Future Projects

DPMA Initial Development Tasks  Develop DPMA data storage, including retrieval and update mechanisms  Develop “human activity” agent to represent initial data processing tasks  First automated agents will focus on DIRSIG  Translate project tasks into DIRSIG inputs  Execute DIRSIG and harvest all outputs  DPMA project members will direct future development; attack the important needs first!  Develop DPMA data storage, including retrieval and update mechanisms  Develop “human activity” agent to represent initial data processing tasks  First automated agents will focus on DIRSIG  Translate project tasks into DIRSIG inputs  Execute DIRSIG and harvest all outputs  DPMA project members will direct future development; attack the important needs first!

Example Task