Presentation is loading. Please wait.

Presentation is loading. Please wait. OGF-23 iRODS Metadata Grid File System Reagan Moore San Diego Supercomputer Center.

Similar presentations

Presentation on theme: " OGF-23 iRODS Metadata Grid File System Reagan Moore San Diego Supercomputer Center."— Presentation transcript:

1 OGF-23 iRODS Metadata Grid File System Reagan Moore San Diego Supercomputer Center

2 OGF-23 integrated Rule-Oriented Data System Client InterfaceAdmin Interface Current State Rule Invoker Micro Service Modules Metadata-based Services Resources Micro Service Modules Resource-based Services Service Manager Consistency Check Module Rule Modifier Module Consistency Check Module Engine Rule Confs Config Modifier Module Metadata Modifier Module Metadata Persistent Repository Consistency Check Module Rule Base

3 OGF-23 iRODS Data Grid - System Metadata iRODS Server Rule Engine Data request goes to iRODS Server iRODS Server Rule Engine Metadata Catalog Rule Base DB Server looks up information in catalog Catalog tells which iRODS server has data 1 st server asks 2 nd for data The 2nd iRODS server applies rules User asks for data

4 OGF-23 Logical Name Spaces 1.Logical file names POSIX attributes - owner, location, size, creation time, access controls Grid attributes - aggregation in container, checksum, validation time Collection attributes - description, provenance, authenticity 2.Logical user names Home data grid, project, password, group membership, address, 3.Logical resource names Physical resource address, group membership, access controls 4.Logical rule names Physical rule instance, version number, condition for execution 5.Logical micro-service names Physical micro-service instance, version number, access controls 6.Logical state information Physical attribute instance, version number

5 OGF-23 Logical File Names

6 OGF-23 Logical File Name - Collections

7 OGF-23 File Access Controls The triplet {DATA_ID, USER_ID, DATA_ACCESS_TYPE} is used to define data access controls.

8 OGF-23 Logical File Name

9 OGF-23 Logical User Name

10 OGF-23 Logical User Names

11 OGF-23 Logical Resource Attributes

12 OGF-23 Logical Resource Names

13 OGF-23 Logical Rule Names

14 OGF-23 Tokens - Internal State Variables

15 OGF-23 Structured Information Distributed information resources Information required to interact with remote resource resides within the remote resource Mounted Collection interface accesses the information Applications now manipulate structured information Posix I/O to manipulate bit streams is no longer sufficient Generate structured information through application of micro- services at the remote storage location Transmit structured information from remote storage location to the client Maintain information structures in memory to link multiple micro- services into a server-side workflow

16 OGF-23 Mounted Collection Interface Mounted collection Set of standard operations for acquiring information from remote resource Containers - tar files, HDF5, XFDU, XAM, Remotely mounted file system directories Structured information driver Mapping of standard operations to the protocol used by the remote information resource Multiple standards for describing structured information Data grids Lstore, SRB, iRODS, … Digital Library METS, PREMIS metadata standards for descriptive metadata Fedora, DSpace information resources Preservation systems OAIS representation information for a record LOCKSS Commercial SNIA / XAM object-based storage interface

17 OGF-23 Audit Trails Log of operations performed upon a file name of rule name of person who applied the operation date file that was manipulated additional information depending on operation type Trying two approaches Store information with the file as descriptive metadata Store in separate log Log of operations performed at a storage system Recent request for tracking storage level usage and performance

18 OGF-23 For More Information Reagan W. Moore San Diego Supercomputer Center

Download ppt " OGF-23 iRODS Metadata Grid File System Reagan Moore San Diego Supercomputer Center."

Similar presentations

Ads by Google