Presentation is loading. Please wait.

Presentation is loading. Please wait.

Data Grid: GRASP Mike Smorul. Grid Retrieval and Search Platform Based on concepts developed in the Earth Science Data Interface (ESDI) developed at the.

Similar presentations


Presentation on theme: "Data Grid: GRASP Mike Smorul. Grid Retrieval and Search Platform Based on concepts developed in the Earth Science Data Interface (ESDI) developed at the."— Presentation transcript:

1 Data Grid: GRASP Mike Smorul

2 Grid Retrieval and Search Platform Based on concepts developed in the Earth Science Data Interface (ESDI) developed at the UMIACS GLCF. Provides a graphical interface into data grid holdings. Access to entire GLCF holdings through the Storage Resource Broker(SRB)

3 Earth Science Data Interface

4 ESDI Overview Designed to allow for intuitive browsing and searching of large geospatial data sets. Tightly integrated set of web, ftp, and file servers. (customized to GLCF) Distributes over 7Tb of data per month Over 27,000 Landsat scenes and 13Tb of data available for download

5 SRB Grid Testbed (original) Modified the SRB to hold spatial data Contributed Informix port of the SRB (v3.2) Linked three ESIP sites Tested replication between GMU and UMD. Remote registration at UNH

6 Lessons Learned The SRB can easily handle textual metadata. Spatial data could be stored into extended Informix attributes, but querying was available only through the DAI  Limited SRB MCAT to Informix based systems

7 GRASP Architecture

8 GRASP uses a data grid as an abstract storage repository. Metadata in the grid is mined from the grid itself or from external sources and published into a browsable form.  Data grids may allow for platform independent metadata, but may not be optimal for access

9 GRASP Screenshot

10 Grid Holdings Registered GLCF holdings  Over 338,000 registered files  4.4Tb total size  Granular permissions on registered holdings No need to move all data into grid, registered pre-existing holdings in place.

11 Current data grid Designed using newer SRB software that allows for Federated grid model. Created using standard SRB software configuration (postgreSQL) Large data sites can maintain independent MCATs  Administratively independent  Ability to customize data grid to site security/data requirements while maintaining compatibility across federation. Smaller peers can register as clients of UMD

12 Data grid overview

13 Growing data grid Additional MCATs can be federated Additional SRB aware clients can be added Remote data supplying sites can contribute data sets from their resources Sites needing quick local access to data can replicate to local SRB setup.

14 Questions?


Download ppt "Data Grid: GRASP Mike Smorul. Grid Retrieval and Search Platform Based on concepts developed in the Earth Science Data Interface (ESDI) developed at the."

Similar presentations


Ads by Google