OGF-23 iRODS Metadata Grid File System Reagan Moore San Diego Supercomputer Center

OGF-23 integrated Rule-Oriented Data System Client InterfaceAdmin Interface Current State Rule Invoker Micro Service Modules Metadata-based Services Resources Micro Service Modules Resource-based Services Service Manager Consistency Check Module Rule Modifier Module Consistency Check Module Engine Rule Confs Config Modifier Module Metadata Modifier Module Metadata Persistent Repository Consistency Check Module Rule Base

OGF-23 iRODS Data Grid - System Metadata iRODS Server Rule Engine Data request goes to iRODS Server iRODS Server Rule Engine Metadata Catalog Rule Base DB Server looks up information in catalog Catalog tells which iRODS server has data 1 st server asks 2 nd for data The 2nd iRODS server applies rules User asks for data

OGF-23 Logical Name Spaces 1.Logical file names POSIX attributes - owner, location, size, creation time, access controls Grid attributes - aggregation in container, checksum, validation time Collection attributes - description, provenance, authenticity 2.Logical user names Home data grid, project, password, group membership, address, 3.Logical resource names Physical resource address, group membership, access controls 4.Logical rule names Physical rule instance, version number, condition for execution 5.Logical micro-service names Physical micro-service instance, version number, access controls 6.Logical state information Physical attribute instance, version number

OGF-23 Logical File Names

OGF-23 Logical File Name - Collections

OGF-23 File Access Controls The triplet {DATA_ID, USER_ID, DATA_ACCESS_TYPE} is used to define data access controls.

OGF-23 Logical File Name

OGF-23 Logical User Name

OGF-23 Logical User Names

OGF-23 Logical Resource Attributes

OGF-23 Logical Resource Names

OGF-23 Logical Rule Names

OGF-23 Tokens - Internal State Variables

OGF-23 Structured Information Distributed information resources Information required to interact with remote resource resides within the remote resource Mounted Collection interface accesses the information Applications now manipulate structured information Posix I/O to manipulate bit streams is no longer sufficient Generate structured information through application of micro- services at the remote storage location Transmit structured information from remote storage location to the client Maintain information structures in memory to link multiple micro- services into a server-side workflow

OGF-23 Mounted Collection Interface Mounted collection Set of standard operations for acquiring information from remote resource Containers - tar files, HDF5, XFDU, XAM, Remotely mounted file system directories Structured information driver Mapping of standard operations to the protocol used by the remote information resource Multiple standards for describing structured information Data grids Lstore, SRB, iRODS, … Digital Library METS, PREMIS metadata standards for descriptive metadata Fedora, DSpace information resources Preservation systems OAIS representation information for a record LOCKSS Commercial SNIA / XAM object-based storage interface

OGF-23 Audit Trails Log of operations performed upon a file name of rule name of person who applied the operation date file that was manipulated additional information depending on operation type Trying two approaches Store information with the file as descriptive metadata Store in separate log Log of operations performed at a storage system Recent request for tracking storage level usage and performance

OGF-23 For More Information Reagan W. Moore San Diego Supercomputer Center