10 May 2001WP6 Testbed Meeting1 WP5 - Mass Storage Management Jean-Philippe Baud PDP/IT/CERN

10 May 2001WP6 Testbed Meeting2 Introduction A quick summary of project and month9 deliverables WP5 is different from other middleware WPs We will provide software to be distributed as part of testbed suite but this will need further work from testbed sites to interface to local mass storage system. Testbed sites should sign up to WP5 to develop their local interface.

10 May 2001WP6 Testbed Meeting3 Tasks A common API to heterogeneous mass storage systems Tape exchange including metadata Metadata publishing

10 May 2001WP6 Testbed Meeting4 Task 1 Common API Follow-on from InterStage discussion in 1999 A common API to MSS would allow user programs to be portable between centres. Task 1 takes this a step further and defines an API which can also be used by Grid middleware to interface to MSS Will have the side effect of making user programs portable as well.

10 May 2001WP6 Testbed Meeting5 Pre-ATF Mass Storage access was treated as local No need to handle grid proxies etc WP2 datamover, replication manager etc would handle that We would define an API like RFIO and testbed sites would implement locally

10 May 2001WP6 Testbed Meeting6 ATF Definition of a Storage Element (SE) includes direct access from Grid: –Access to disk and tape (ie all storage) –3 interfaces put/get open/read management (create, stage, reserve,....) Required a rethink, need software not just API

10 May 2001WP6 Testbed Meeting7 GridFTP Globus have reworked their i/o plans GridFTP forms the basis of future developments –tunable for network performance –parallel streams –file and stream interfaces RAL, and CERN have tested alpha code –but just for transfer

10 May 2001WP6 Testbed Meeting8 GridFTP server as SE Looks feasible given GASS experience to implement SE using GridFTP server Uses Globus infrastructure Handles GSI proxies Gives trivial access to local disk and any HSM with Unix filestore interface Plan to produce prototype for month9 for Unix filesystem and Castor

10 May 2001WP6 Testbed Meeting9 Task 2. If networks do not deliver, it will be necessary to move data by tape between CERN and Regional Centres Sometimes it will be easier to extract tapes from a robot and ship them than to copy the data onto an interchange tape. This task seeks a clean way of doing this and of transporting the associated metadata

10 May 2001WP6 Testbed Meeting10 Possibilities There is a AIIM standard C21.1 that claims to cover this requirement The WP will investigate this and implement if suitable Otherwise start from scratch Little progress on this so far

10 May 2001WP6 Testbed Meeting11 Task 3. Metadata Publishing WP5 envisages a requirement to provide static (slowly changing) and dynamic (on demand) metadata about an Storage Element and its contents Require somewhere to publish static information (info store and API to write to it, and method for registering as producer) Require support for acting as an active publisher (front- end to handle incoming queries, API for replying to them, and method for registering as producer)

10 May 2001WP6 Testbed Meeting12 Metadata Still require input from other WPs on what information they expect us to publish We will draft a minimal set to prove concept and prototype publishing code Needs Information Service

10 May 2001WP6 Testbed Meeting13 Deliverables WP5 will deliver software for:- data access metadata production

10 May 2001WP6 Testbed Meeting14 Data Access WP5 will provide an SE interface based on a GridFTP server. This will be software to be distributed as part of the testbed suite by the standard WP6 methods. This will provide GSI proxy handling for authentication and map onto local uid/gid as defined by authorisation system. The maintenance of this map file is an issue. For month 9 this will support Castor and Unix filesystems for access by the ReplicaManager, but it is not foreseen to distribute CASTOR as part of the testbed software.

10 May 2001WP6 Testbed Meeting15 Metadata WP5 will provide a utility to run which will write metadata to Information Service about –the Storage Element itself –the physical files In some cases this will run regularly (e.g. by cron) In others it will run on demand

10 May 2001WP6 Testbed Meeting16 Future Developments User API to allow direct user access to remote data –individual sites need not allow this SE on other HSMs Disk housekeeping as part of SE

10 May 2001WP6 Testbed Meeting17 Dissemination Mailing list Web page:- proj-datagrid-masstorage