GFS-WG: Informal Status Report Jane Xu, IBM Osamu Tatebe, AIST Arun Jagatheesan, SDSC (GFS-WG Chairs) https://forge.gridforum.org/projects/gfs-wg/ Global Grid Forum 12 OGSA Data Design Team FTF meeting Sep 2004, Brussels, Belgium

IP & © Intellectual Property Statement The GGF takes no position regarding the validity or scope of any intellectual property or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; neither does it represent that it has made any effort to identify any such rights. Copies of claims of rights made available for publication and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the GGF Secretariat. The GGF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights which may cover technology that may be required to practice this recommendation. Please address the information to the GGF Executive Director. Copyright (C) Global Grid Forum (2004). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the GGF or other organizations, except as needed for the purpose of developing Grid Recommendations in which case the procedures for copyrights defined in the GGF Document process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the GGF or its successors or assigns.   This document and the information contained herein is provided on an "AS IS" basis and THE GLOBAL GRID FORUM DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE." This has to be displayed on all GGF presenatations

Talk Outline GFS Participation What we were supposed to do What is done till now GFS VFDS Service Specification (emerging) GFS over all Architecture (draft) Summary Let’s talk Outline

GFS Participation AIST, Japan Hitachi Data Systems (HDS) Trying to add more participants for multi-implementer/vendor accepted standard AIST, Japan Hitachi Data Systems (HDS) IBM Almaden, IBM Storage Systems Group Network Appliance (NetApp) San Diego Supercomputer Center (SDSC) Storage Machines Other possible participation (HP, BMC, EMC)

Related Standards bodies IEEE SNIA DMTF

Charter of GFS-WG Two goals (two documents) Virtual Filesystem Directory Service Provides global namespace Manage namespace, access control, and metadata management Architecture for Grid File System Services Provides functionality of virtual file system in grid environment Facilitates federation and sharing of virtualized data Uses File System Directory Services and standard access protocols

Virtual Filesystem Directory Services Recommendation Document: “Virtual Filesystem Directory Service Specification” provides global namespace manages virtual file system directories in association with access permission, and other application-specific metadata https://forge.gridforum.org/projects/gfs-wg/document/Virtual_Filesystem_Directory_Service_Specification_-_GGF12/en/1 Will be submitted in GGF13 (March 2005)

Architecture of Grid File System Services Recommendation Document: “Architecture for Grid File System Services” provides a composition of services to realize Grid File System using File System Directory Services and other common services https://forge.gridforum.org/projects/gfs-wg/document/Architecture_Specification_for_Grid_File_System_Services/en/2 Will be submitted in GGF14 (June 2005)

Virtual Filesystem Directory Service

Virtual Filesystem Directory Service (1) /grid ggf jp data gfs file1 file3 file2 dir1 Web Service (aka Grid Service) Directory PortType Grid namespace (Virtual directory tree, Grid namespace for files, OGSA naming) Virtual directories Junctions (referrals) points to an object (logical or physical files, file systems, other Virtual Filesystem Directory Service instances) Translation point to a grid-wide pointer Attributes (and Status) Attributes: ACL, times (Status: open, lock) EPR2 EPR1 (WS-Addressing Endpoint Reference)

Virtual Filesystem Directory Service (2) Virtual Directory A VFDS entry that is represented as a filesystem directory It does not have any corresponding position in any physical filesystem /grid ggf jp data gfs file1 dir1 file1 file2 file3 dir1

Virtual Filesystem Directory Service (3) Junction A VFDS entry that interconnects an existing file or filesystem tree into the global namespace. Four basic types: physical junctions, logical junctions, referrals, and aliases. /grid ggf jp data gfs file1 dir1 file1 file2 file3 dir1

Virtual Filesystem Directory Service (4) [Physical junction] an EPR that points to the physical location of a filesystem tree or single file [Logical junction] an EPR to a secondary service for logical-to-physical resolution given a logical name [Referral] a junction that links to other VFDS instances facilitates such features as federation of independent domains of control, scalability of a single domain of control, availability of redundant service instances, etc [Alias] a junction that references another entry within the same service instance

Virtual Filesystem Directory Service (5) /grid ggf jp data gfs file1 file3 file2 dir1 Operations lookup, list Obtain entry or entries that includes target create, delete, move update Management of attributes and status

Architecture of Grid File System

Grid File System Services Provides virtual file system by integrating and federating file systems in a grid Service oriented architecture using several services Architecture document Provide a way of composition of services to realize Grid file system What kind of functionalities each service should have How these services relate with each other

Scenario (1): Architecture of Grid file system POSIX APIs File System Directory Services File System Services Client Library /grid ggf jp data gfs file1 file3 file2 file4 ReplicaSet Services File System Server file4 file4 Applications, Users, . . . file2 File Access Services (GridFTP server, SRM), . . . NFSv4, . . .

Scenario (2): GFS Resource Providers I start with by introducing the reality. We have some data and storage on some disk/ hierarchical resource. IT can have its own directories or physical names for the data. These are Grid Resource providers from the GFS perspective. Later these resource providers could be other sources like service registries or any thing dealing with a naming system or catalog. It is highly tempting to say that GFS Resource providers are nothing but SRM – at least looks for me to be so. But please refrain from doing that. We need to have the GSM also agree to this picture. GFS Resource Providers (GRP) providing content /txt3.txt GRP GRP

Scenario (2): GFS Administrative Domain GFS Administrative Domain with one or more GFS Resource Providers Research Lab Multiple such resource provides could be present in an organization or autonomous administrative domain as we called before. /txt3.txt GRP GRP

Scenario (2): GFS Administrative domains Research Lab data + storage (10) Storage-R-Us Resource Providers data + storage (50) Finance Department data + storage (40) There could be multiple administrative domains like this that makes up an virtual enterprise or what we call as the Grid GRP /txt3.txt GRP GRP GRP /…/text1.txt /…//text2.txt

Scenario (2): GFS Service Provider /home/arun.sdsc/exp1 /home/arun.sdsc/exp1/text1.txt /home/arun.sdsc/exp1/text2.txt /home/arun.sdsc/exp1/text3.txt data + storage (100) Logical Namespace (Need not be same as physical view of resources ) Research Lab data + storage (10) Storage-R-Us Resource Providers data + storage (50) Finance Department data + storage (40) The resources (data + storage) from all these autonomous administrative domains could be presented as an logical ensemble. The logical namespace created need not reflect the physical layout. Also the logical names do not have any location dependency GRP /txt3.txt GRP GRP GRP /…/text1.txt /…//text2.txt

Scenario (2) Vendor Specific? CIFS Server Interface & CIFS  GFS ? CIFS client NFS Server Interface & NFS  GFS ? NFS client Can provide all the capabilities of grid-based logical namespace GFS Client Interface (VFDS) GFS Client GFS Server GFS Resource Interface (to be standardized) Organization A NFS or CIFS GFS Peer Interface (to be standardized) GFS Resource Interface (to be standardized) Data Center A Proprietary FS

Grid File System Directory Service Specification (slide courtesy by Manuel Pereira at IBM Almaden Research Center)

Filesystem Directory Service Objects Object oriented service interface Foundational communication objects Reply Entry

Filesystem Directory Service Objects

Pathname Pathname Absolute pathname Relative pathname and an Object ID representing a virtual directory Object ID is a per session object identification token Unique per server instance

Federation of Services Resolving paths that span multiple namespace domains or service instances Service Referrals Delegated Resolution

Summary Grid File System WG (GFS-WG) Interest from vendors and implementers F2F discussions held Goals of GFS-WG Virtual Filesystem Directory Service Specification Architecture of Grid File System Two possible Architecture scenarios Thanks for all those who stayed back – it must be later there now. From the the GFS-WG members, to GSM-members have a nice stay and enjoy Hawaii  GGF GFS-WG https://forge.gridforum.org/projects/gfs-wg/