1. The Movement Towards Grid Architectures in Planetary Science
Dan Crichton, Steve Hughes and Chris Mattmann
June 28, 2005
Jet Propulsion Laboratory
California Institute of Technology
National Aeronautics and Space Administration
Pasadena, CA, USA

2. Scope of the discussion
Focus on capture, processing and distribution of scientific results from solar system exploration projects
Architecture
Current challenges
Data grid framework
Conclusions and Future Work
Standardization efforts within the international space community
16-Nov-18
Planetary-GGF

3. Characteristics of missions
Often unique, one of a kind missions
Can drive technological changes
Instruments are competed and developed by academic, industry and industrial partners
Highly distributed acquisition and processing across partner organizations
Highly diverse data sets given heterogeneity of the instruments and the targets (i.e. solar system)
Missions are required to archive and distribute data with the NASA Planetary Data System (PDS)
PDS is chartered to ensure that data are archived and available to the scientific community
Work with projects to help design, generate, and validate data products for placement in archive
Develop data standards to ensure interoperability and future usability
Provide expert scientific help to the user community.
Lead peer-review to ensure data quality
Distributed archive managed by the community
16-Nov-18
Planetary-GGF

4. Characteristics of the science data
High-quality, peer-reviewed archive of data from Solar System Exploration missions
Stored for long-term viability
Described by metadata
Distributed either online or on physical media
Defined by a set of data standards
Planetary Science Data Dictionary
Planetary Community Model
Standard Structural Specifications for products, data sets, volumes, etc supported by the model
30 years of data captured
Very, very diverse, but defined by a common conceptual model
16-Nov-18
Planetary-GGF

5. Product Information Model
Conceptual
Model
Information Object
<TITLE>
Cassini
Saturn Image
</TITLE>
<
MimeType
>
Image/JPEG
<
MimeType
>
Defines…
Implements
<Resource Location>
Data Dictionary
Based on
Dublin Core
Metadata
Data set
schema
</Resource Location>
Object
<TARGET_NAME>
Saturn
Data Elements
Vocabulary
</TARGET_NAME>
<SPACECRAFT_NAME>
Cassini
</SPACECRAFT_NAME>
Data Element Model
(ISO 11179)
Based on
ISO/IEC 11179
Describes…
Data Object
16-Nov-18
Planetary-GGF

6. End-to-end Planetary Science Systems
One or More
Instruments
One or More
Spacecraft
A Space
Tracking
Network
An Instrument
Control
Center
Commodity
Space
Communications Systems
Commodity
Space
Navigation
Systems
A Spacecraft
Control
Center
A Ground
Tracking
Network
A Science
Facility
16-Nov-18
Planetary-GGF
Source: A. Hooke, NASA/JPL

7. Generic Planetary Science System Architecture
16-Nov-18
Planetary-GGF

8. Architectural drivers for planetary science
Increasing data volumes requiring new approaches for data production, validation, processing, discovery and data transfer/distribution (E.g., scalability relative to available resources)
Increased emphasis on usability of the data (E.g., discovery, access and analysis)
Increasing diversity of data sets and complexity for integrating solar system projects (E.g., common information model for describing the data)
Increasing distribution of coordinated processing and operations (E.g., federation)
Increased pressure to reduce cost of supporting new missions
Increasing desire for PIs to have integrated tool sets to work with data products (E.g. perform their own generation and distribution)
16-Nov-18
Planetary-GGF

9. 2001 Mars Odyssey: A paradigm change
Pre-Oct 2002, no unified view across distributed operational planetary science data repositories
Science data distributed across the country
Science data distributed on physical media
Planetary data archive increasing from 4 TBs in 2001 to 300 TBs in 2007
Traditional distribution infeasible due to cost and system constraints
Mars Odyssey could not be distributed using traditional method
Current work with the OODT Data Grid Framework has provided the technology for NASA’s planetary data management infrastructure to
Support online distribution of science data to planetary scientists (up to 500 MB products)
Enable interoperability between nine institutions
Support real-time access to data products
Provided uniform software interfaces to all Mars Odyssey data allowing scientists and developers to link in their own tools
Operational October 1, 2002
2001 Mars Odyssey
16-Nov-18
Planetary-GGF

10. PDS Discipline and Data Nodes
Mars Odyssey THEMIS/ASU Data Node
Geosciences/Washington University
Rings/SETI
Radio Science/Stanford
Small Bodies/UMD
Planetary Plasma/UCLA
Imaging/JPL
Engineering/JPL
MRO-HiRISE/UofA Data Node
16-Nov-18
Imaging/USGS
Planetary-GGF
NAIF/JPL
Atmospheres/New Mexico State

11. Planetary Science Data System Distribution Architecture (Mars Odyssey Era and Beyond)
More Consistency
16-Nov-18
Planetary-GGF

12. Example: MRO Data Distribution Challenge
Science Community
Distribution
Transfer
Transfer
World-wide Deep Space Network
MRO HiRISE
Data Node
@ Univ of Arizona
NASA Deep
NSSDC
Integration
16-Nov-18
Planetary-GGF
Distributed Planetary Data System

13. The Object Oriented Data Technology Framework
Started in 1998 as a research and development task funded at JPL by the Office of Space Science to address
Application of Information Technology to Space Science
Provide an infrastructure for distributed data management
Research methods for interoperability, knowledge management and knowledge discovery
Develop software frameworks for data management to reuse software, manage risk, reduce cost and leverage IT experience
OODT Initial focus
Data archiving – Manage heterogeneous data products and resources in a distributed, metadata-driven environment
Data location and discovery – Locate data products across multiple archives, catalogs and data systems
Data retrieval – Retrieve diverse data products from distributed data sources and integrate
Runner-up NASA Software of the Year, 2003
Used in planetary, earth and biomedical sciences
Open source at
16-Nov-18
Planetary-GGF

14. OODT Architectural Principles*
Separate the technology and the information architecture
Encapsulate the messaging layer to support different messaging implementations
Encapsulate individual data systems to hide uniqueness
Provide data system location independence
Require that communication between distributed systems use metadata
Define a model for describing systems and their resources
Provide scalability in linking both number of nodes and size of data sets
Allow systems using different data dictionaries and metadata implementations to be integrated
Leverage open source
* Crichton, D, Hughes, J. S, Hyon, J, Kelly, S. “Science Search and Retrieval using XML”,
Proceedings of the 2nd National Conference on Scientific and Technical Data, National Academy of
Science, Washington DC, 2000.
16-Nov-18
Planetary-GGF

15. What does OODT do?
Tie together loosely coupled distributed heterogeneous data systems into a virtual data grid
Support critical functions
Data Production and workflow
Data Distribution
Data Discovery (including query optimization across highly distributed systems)
Data Access
An architectural approach first, an implementation second
Adapt to different distributed computing deployments
Promotes a REST-style architectural pattern for search and retrieval
Scalability in linking together large, distributed data sets
16-Nov-18
Planetary-GGF

16. OODT Data Architecture Focus
On types of and relationships among a software system’s data
Decomposition of data within a software system to its logical components and interactions
Components: Data Elements, Data Dictionary, Data Models of individual data sources
Interactions: Mappings between Data Dictionary to Data Models, Data Element structural comparison
Some standards currently exist for data architecture
ISO: ISO Standardization and Specification of Data Elements
Dublin Core Metadata Initiative: Dublin Core Data Elements to describe any electronic resource
Specifications for the Data Architecture
Common XML schema for managing information about data resources
Common XML schema for messaging between distributed services
Methods for integrating existing domain models within architecture
16-Nov-18
Planetary-GGF

17. OODT Models Request/Response Model Resource Metadata Model 16-Nov-18
XMLQuery -
resultModeId: String
propogationType: String
propogationLevels: String
maxResults: int
kwqString: String
numResults: int
mimeAccept: List
QueryHeader
id: String
title: String
description: String
type: String
statusID: String
securityType: String
revisionNote: String
dataDictID: String
QueryResult
list: List
QueryElement
role: String
value: String 1
fromSet
selectSet
whereSet
result
queryHeader
nasa.pds.xmlquery
Based on Dublin Core
Request/Response Model
Resource Metadata Model
Based on ISO/IEC 11179
16-Nov-18
Planetary-GGF

18. Conceptual Technology Architecture
Service Registry
Name Server
Name Server
Registry Server
Node 1
Profile Server
Web I/F
WSDL
WSDL
Node 1
Profile Server
Query Integration
XML Request
Node 1
Profile Server
Information Object
Product
Catalogs
XML Request
XML Request
Repository
Product Server
Desktop I/F
Information Object
Information Object
Science Products
XML Request
Repository
Product Server
Info Object
Information Object
Science Products …
XML Request
Repository/Archive
Server
Common Meta Models for Describing Space Information Objects
Common Data Dictionary end-to-end
Science
Products
16-Nov-18
Planetary-GGF

19. Consultative Committee on Space Data Systems (CCSDS)
Participation by major space agencies world-wide
Focused initially on space communications, but evolving towards end-to-end space data systems architecture
Increased dependency and interoperability challenges between space agencies
Works in conjunction with ISO
CCSDS Information Architecture WG
Part of the Systems Engineering Area
Define a reference Space Information Architecture that encompasses the capture, management and exchange of data for both flight and ground environments across the operational mission lifecycle.
Leverage existing work (adopt, adapt, develop)
16-Nov-18
Planetary-GGF

20. Communications & Navigation Systems
CCSDS Thrust Areas
Space
Internetworking
Services
Mission Operations
and Information
Management Services
Commodity
Communications & Navigation Systems
Space
Link
Services
Systems Engineering
Cross Support
Services
16-Nov-18
Planetary-GGF
Source: A. Hooke, NASA/JPL

21. Information Architecture Concept
Relay Satellite
Simple Information Object
Spacecraft and
Scientific Instruments
Spacecraft / lander
Science
Data Archive
External Science Community
Information Object
Information Object
Science Information Package
Science Information Package
Science
Data Processing
Science Products - Information Objects
Telemetry Information Package
Science Information Package
Data Analysis and Modeling
Science Information Package
Planning
Information
Object
Instrument
Planning
Information
Object
Data Acquisition
and Command
Science Team
Mission Operations
Instrument /Sensor
Operations
16-Nov-18
Planetary-GGF
Common Meta Models for Describing Space Information Objects
Common Data Dictionary end-to-end

22. Architectural Focus Establish information architectural standards
Standards based Data Architecture
Dublin Core
ISO-11179
Ontologies
Data Models
Metadata Models
Standards based Software Architecture
Components (Registries, Repositories)
COTS-based messaging framework
Configurations (legal topologies, interconnections, behaviors)
Core Architectural Principles
16-Nov-18
Planetary-GGF

23. Technical Approach (Information View)
1. Information Objects
3. Meta Models
2. Information Models
16-Nov-18
Planetary-GGF

24. IA Functional Objects Mission Planning Analysis Monitor & Control
Directive
Generation
Sci Data
Mgmt
Management
Representative
Functional
Objects
Data
Objects
Query /
Results
Metadata /
Resources
Domain
Data
Models
Query
Service
Registry
Service
Information
Management
Functional
Objects
Common
Schema &
Dictionaries
Local
Data
Models
Repository
Service
Product
Service
16-Nov-18
Planetary-GGF

25. Critical Components: Registries & Repositories
16-Nov-18
Planetary-GGF

26. TODAY: Continuing to evolve across the planetary science enterprise
Consistent distributed capabilities
Resource discovery (data, metadata, services, etc), unified repository access, simple transformations, bulk transfer of multiple products, and unified catalog access
Move towards era of “grid-ing” loosely coupled science system
Develop on-demand, shared services (E.g. processing, translation, etc)
Processing
Translation
Deploy high throughput data movement mechanisms
Move capability up the mission pipeline
Reduce local software solutions that do not scale
Increasing importance in developing an “enterprise” approach with common services
Build value-added services and capabilities on top of the infrastructure
16-Nov-18
Planetary-GGF

27. Questions? Contacts: Dan Crichton Steve Hughes THANK YOU
Steve Hughes
16-Nov-18
Planetary-GGF

28. Refereed Papers
Mattmann, C., Medvidovic, N., Ramirez, P., Jakobac, V. Unlocking the Grid. Proceedings of the 8th International ACM SIGSOFT Symposium on Component-based Software Engineering. May, 2005.
Mattmann, C., Malek, S., Beckman, N., Mikic-Rakic, M., Medvidovic, N., Crichton, D. GLIDE: A Grid-based, Light-weight Infrastructure for Data-intensive Environments. Proceedings of the 2005 European Grid Conference, Amsterdam, the Netherlands, February 2005.
Mattmann C, Crichton D, Hughes, J. S., Kelly, S., Ramirez, P. Software Architecture for Large-scale, Distributed, Data-intensive Systems. Proceedings of the 4th IEEE/IFIP Working Conference on Software Architecture, Oslo, Norway, June 2004.
Mattmann C, Ramirez P, Crichton D, and Hughes, J.S. Packaging Data Products using Data Grid Middleware for Deep Space Mission Systems. Proceedings of the 8th International Conference on Space Operations, Montreal, Canada, 2004.
Crichton D, Hughes, J.S., Kelly, S. A Science Data System Architecture for Information Retrieval. Clustering and Information Retrieval. Kluwer Academic Publishers. December 2003.  - Book Chapter
Crichton D, Hughes, J.S., Kelly, S, Rameriz, P. A Component Framework Supporting Peer Services for Space Data Management IEEE Aerospace Conference. Big Sky, Montana. March 2002. 
Crichton D, Downing G, Hughes J. S, Kincaid H, Srivistava S. An Interoperable Data Architecture for Data Exchange in a Biomedical Research Network. 14th IEEE Symposium on Computer-Based Medical Systems. July  
Crichton, D., Hughes J. S, Hardman S, Kelly S. A Distributed Component Framework for Data Product Interoperability. 17th CODATA International Conference, Baveno, Italy. October 2000.
Crichton, D., Hughes J. S, Kelly S, Hyon J. Science Search and Retrieval using XML. Second National Conference on Scientific and Technical Data, Washington D.C., National Academy of Sciences. March 2000.
16-Nov-18
Planetary-GGF

29. Remote tools to enable debugging, configuration and software upgrades
Operational Tools
Simple installation
Remote tools to enable debugging, configuration and software upgrades
With role-based access controls
Monitoring to verify health of distributed infrastructure
One of our early discoveries was that it is difficult to manage distributed environments. We therefore developed a tool that enables us to have a comprehensive view of all the distributed OODT servers that are running. This enables not only starting and stopping, but also software updates, debugging and deployment of software extensions.
OODT also provides a comprehensive documentation set through an OODT developers portal.
An obviously, as I’ve mentioned already, we are providing training sessions to developers.
16-Nov-18
Planetary-GGF