Designing the Architecture for Grid File System (GFS) Arun swaran Jagatheesan San Diego Supercomputer Center Global Grid Forum 12 Brussels, Belgium

2 Global Grid ForumGFS Architecture draft IP & © The GGF takes no position regarding the validity or scope of any intellectual property or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; neither does it represent that it has made any effort to identify any such rights. Copies of claims of rights made available for publication and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the GGF Secretariat. The GGF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights which may cover technology that may be required to practice this recommendation. Please address the information to the GGF Executive Director. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the GGF or other organizations, except as needed for the purpose of developing Grid Recommendations in which case the procedures for copyrights defined in the GGF Document process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the GGF or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE GLOBAL GRID FORUM DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE." Intellectual Property Statement Copyright (C) Global Grid Forum (2004). All Rights Reserved.

3 Global Grid ForumGFS Architecture draft Outline Agenda Why is the GFS different What we must and must not do Issues Small talks by some selected folks Discussion (not to get answers right now)

4 Global Grid ForumGFS Architecture draft GFS Resource Providers GFS Resource Providers (GRP) providing content GRP /txt3.txt GRP

5 Global Grid ForumGFS Architecture draft GFS Administrative Domain GRP GFS Administrative Domain with one or more GFS Resource Providers /txt3.txt GRP Research Lab

6 Global Grid ForumGFS Architecture draft GFS Administrative domains /…/text1.txt /…//text2.txt GRP /txt3.txt GRP Storage-R-Us Resource Providers data + storage (50) Finance Department data + storage (40) Research Lab data + storage (10)

7 Global Grid ForumGFS Architecture draft GFS Service Provider /…/text1.txt /…//text2.txt GRP /txt3.txt GRP Storage-R-Us Resource Providers data + storage (50) Finance Department data + storage (40) Research Lab data + storage (10) /home/arun.sdsc/exp1 /home/arun.sdsc/exp1/text1.txt /home/arun.sdsc/exp1/text2.txt /home/arun.sdsc/exp1/text3.txt data + storage (100) Logical Namespace (Need not be same as physical view of resources )

8 Global Grid ForumGFS Architecture draft CIFS client NFS client GFS Client CIFS Server Interface & CIFS  GFS ? NFS Server Interface & NFS  GFS ? GFS Client Interface (includes VFDS – to be standardized) GFS Server NFS or CIFS Proprietary FS GFS Resource Interface (to be standardized) GFS Resource Interface (to be standardized) GFS Peer Interface (to be standardized) Organization A Data Center A Vendor Specific? Can provide all the capabilities of grid- based logical namespace

9 Global Grid ForumGFS Architecture draft Why is GFS different GFS Namespace NOT = Trad. FS namespace Different operations (replicate, meta-data-search, …) Users in the TFS might have different ACL in GFS Different organizations (multi-organizational) Will need a different client Yes, they are both about files

10 Global Grid ForumGFS Architecture draft GFS Services End-to-end GFS services Replicate, Manage ACL, ingest/register data, read, write.. User must not go to different services to get job done – GFS service provides all as a single interface Inter/Intra organizational Different resources, users can not be cross-registered Data management Managing inter/intra organizational data namespace

11 Global Grid ForumGFS Architecture draft Proposed Additions

12 Global Grid ForumGFS Architecture draft GFS Client Interface (includes VFDS – to be standardized) GFS Server code NFS or CIFS Proprietary FS GFS Resource Interface (to be standardized) GFS Resource Interface (to be standardized) Organization A Data Center A All inter/intra organizational data management in one interface… create, update, replicate, migrate, meta-data based search, plug-more-resources, plug-more- organizations, user-management,

13 Global Grid ForumGFS Architecture draft Other Experts Opinions

14 Global Grid ForumGFS Architecture draft Discussion

15 Global Grid ForumGFS Architecture draft Targeting GFS Operations Inter/Intra Organizational Resources Inter/Intra Organizational User management Logical Resource Namespace Meta-data management GFS System Standardized schema User-defined

16 Global Grid ForumGFS Architecture draft GFS Service Definition Expected specifications GFS Client GFS Resource Draft release soon Telecon weekly Participants welcome Reference Implementation work to start