FP6−2004−Infrastructures−6-SSA CNGrid Middleware GOSv2 Yongjian Wang BUAA – Beijing, China Interoperability workshop of euchinagrid Beijing, June 2006

Yongjian Wang BUAA Bejiing, June 2006 Outline  Brief introduction to GOSv2  Overall architecture of GOSv2  Core Level Services  System/Application Level Services

Yongjian Wang BUAA Bejiing, June 2006 Brief introduction to GOSv2

Yongjian Wang BUAA Bejiing, June 2006 Outline  Brief introduction of GOSv2 Background Goals Research

Yongjian Wang BUAA Bejiing, June 2006 Backgrounds of GOSv2  Grid related research begins since 1999 in China  Part of the Grid Software program supported by the China Ministry of Science and Technology 863 program between 2002 and 2005

Yongjian Wang BUAA Bejiing, June 2006 Goals of GOSv2  Support multiple geographical distributed grid nodes such as super computing centers across China  Sharing mechanism and framework on computing, data, software and combined resources  Provide secured, uniformed and friendly interfaces accessing the scientific computing and information services

Yongjian Wang BUAA Bejiing, June 2006 Research  Focus on 4 key issues to satisfy common requirements: Naming mechanism Process or states maintain Virtual organization Programming model  Focus on implementing architecture, not protocols or services Use Computer System Approach, not middleware or network Use Service Oriented Architecture concept

Yongjian Wang BUAA Bejiing, June 2006 Overall Architecture of GOSv2

Yongjian Wang BUAA Bejiing, June 2006 Outline  Overall architecture of GOSv2 GOSv2 architecture  GOSv2 architecture EVP address spaces  Effective address space  Physical address space  Virtual address space Security mechanism

Yongjian Wang BUAA Bejiing, June 2006 GOSv2 Overall Architecture

Yongjian Wang BUAA Bejiing, June 2006 GOSv2 Architecture

Yongjian Wang BUAA Bejiing, June 2006 GOSv2 Architecture

Yongjian Wang BUAA Bejiing, June 2006 GOSv2 Architecture ( Runtime )

Yongjian Wang BUAA Bejiing, June 2006 EVP address space

Yongjian Wang BUAA Bejiing, June 2006 EVP address spaces  EVP provides three separate naming spaces effective address space Effective address space is used to logically categorized services Examples of effective address: eres://agora1:metaservice All addresses in this space with a prefix eres which is short for effective address physical address space Physical address space used to actually identify physical services Format of physical address is normal URL just as follows: virtual address space Virtual address space used to map effective address to physical address Virtual address used inside GOSv2 environment and starts with prefix vres://  Physical resource can enter or exit dynamically because effective and virtual address can hides the differences.

Yongjian Wang BUAA Bejiing, June 2006 EVP address spaces

Yongjian Wang BUAA Bejiing, June 2006 Security Mechanism in GOSv2

Yongjian Wang BUAA Bejiing, June 2006 Terms in use  User certificate X.509 certificate signed by CNGrid CA  User proxy certificate User proxy certificate is usually a session certificate with short live time. X.509 proxy certificate signed by user, delegate all or part of its owner’s authority Motivation of user proxy certificate is single login  SAML authorization token SAML Token contains attribute entries as description of authorization  GOSContext Java Object contains user proxy certificate and assert token

Yongjian Wang BUAA Bejiing, June 2006 X.509 Certificate format

Yongjian Wang BUAA Bejiing, June 2006

User Proxy Certificate Format  User proxy certificate consist of three different parts: CA’s X.509 certificate User’s X.509 certificate User’s private Key

Yongjian Wang BUAA Bejiing, June 2006 User Proxy Certificate

Yongjian Wang BUAA Bejiing, June 2006

SAML based authorization token

Yongjian Wang BUAA Bejiing, June 2006 Features of security mechanism  Transport layer SSL/TLS specification  Message layer WS-Security specification

Yongjian Wang BUAA Bejiing, June 2006 Axis handler chains mechanism  Axis handler chains adopt the chains of responsibility design pattern. Divide whole function such as security into a chain of small portions Every portion implements different sub-function Portions have no relationships among one another  Based on axis handler chains mechanism Add new function or remove old function are very easy Security mechanism doesn’t invade into concrete application Grip application can use or don’t use security mechanism just by modifying the configuration file.

Yongjian Wang BUAA Bejiing, June 2006 Security handlers in GOSv2 SignHandler Sign body of soap message and add ws-security soap header AddHandler Add GOSContext Object as soap attachment WSSecurityHandler Verify ws-security soap header GetAttachmentsHandler Get GOSContext Object from attachment of soap message VerifyCertsHandler Verify user certificate contained in GOSContext VerifyTokenHandler Verify token contained in GOSContext ACHandler Access control operation based on different policies

Yongjian Wang BUAA Bejiing, June 2006 Security Handler Chain

Yongjian Wang BUAA Bejiing, June 2006 Authentication & Authorization  Authentication Agora service – Provide resource management, user management and so on – Convert username and password to corresponding proxy and token  Authorization SAML Authorization Token – Subject Requester Agora Information Requester Role Information on Agora Server DN of requester – Action Operations of Requested Service

Yongjian Wang BUAA Bejiing, June 2006 Security mechanism

Yongjian Wang BUAA Bejiing, June 2006

Core Level Services of GOSv2

Yongjian Wang BUAA Bejiing, June 2006 Outline  Core Level Services Agora Service User Management Service Resource Management Service Security authentication and authorization Grip Service Grip Container Grip Struct Router Service Overlay network approach for resource management and locating Resource discovery in GOSv2

Yongjian Wang BUAA Bejiing, June 2006 Agora Service

Yongjian Wang BUAA Bejiing, June 2006 GOSv2 Architecture

Yongjian Wang BUAA Bejiing, June 2006 Functions of Agora Service  Role based grid user management Both external and internal user name Proxy certificates management  Service oriented resource management Mapping effective resource to virtual resource Currently using random resource selection algorithm  Token based authorization and access control management Multi-granularity SAML based and decoupled

Yongjian Wang BUAA Bejiing, June 2006 Architecture of Agora Service

Yongjian Wang BUAA Bejiing, June 2006 Grip Service

Yongjian Wang BUAA Bejiing, June 2006 GOSv2 Architecture

Yongjian Wang BUAA Bejiing, June 2006 Grip Service  Grip Service maintains state information for end user. Grip Container Exposed as Web Service Grip Struct is used to invoke different physical services on behalf of end user Used to access underlying physical service

Yongjian Wang BUAA Bejiing, June 2006 Grip Service

Yongjian Wang BUAA Bejiing, June 2006 Router Service

Yongjian Wang BUAA Bejiing, June 2006 GOSv2 Architecture

Yongjian Wang BUAA Bejiing, June 2006 Router Service  Router Service is used to convert virtual address to physical address. Maintain local virtual resource to physical resource mapping relationships Communicate with neighbor router to form global view of all the deployed router services and service locating can achieved in this way.

Yongjian Wang BUAA Bejiing, June 2006 Router Service  Different routers form an application-level virtual network to exchange V-P mapping information

Yongjian Wang BUAA Bejiing, June 2006 Router Scenario- Link

Yongjian Wang BUAA Bejiing, June 2006 Router Scenario- Neighbor Update

Yongjian Wang BUAA Bejiing, June 2006 Router Scenario- search

Yongjian Wang BUAA Bejiing, June 2006 How to discovery resource in GOSv2  Resource discovery in GOSv2 consist of the following steps: Find effective address of resource Convert effective address into virtual address Convert virtual address into physical address

Yongjian Wang BUAA Bejiing, June 2006 System/Application Level Services

Yongjian Wang BUAA Bejiing, June 2006 Outline  GFI (Grid File Infrastructure) Meta service Provide logically global user file space Data Service Distributed file storage File transferred using soap message  Grid Batch System Using Grip and GFI to support global file stagein/out Using simple batch driver to connect to local batch systems, such as OpenPBS, LSF etc.  Grid Batch Accounting System

Yongjian Wang BUAA Bejiing, June 2006 Meta Service

Yongjian Wang BUAA Bejiing, June 2006 GOSv2 Architecture

Yongjian Wang BUAA Bejiing, June 2006 Functions of meta service  Name mapping on grid file effective name  virtual name  physical name

Yongjian Wang BUAA Bejiing, June 2006 Functions of Meta Service (cont.)  Maintain global file information  Maintain file access permissions information  Cooperate with Authorization Authority in agora service for file access authorization  User quota management

Yongjian Wang BUAA Bejiing, June 2006 Meta Service - Operations CategoryService Operations meta data query related isFile 、 isDir 、 exist 、 info 、 getVirtualName 、 List 、 Search file directory related createNewFile 、 delete 、 mkdir 、 rmdir 、 move 、 rename 、 upload file access permissions related getACLInfo 、 Auth 、 Revoke

Yongjian Wang BUAA Bejiing, June 2006 Data Service

Yongjian Wang BUAA Bejiing, June 2006 GOSv2 Architecture

Yongjian Wang BUAA Bejiing, June 2006 Functions of Data Service  Map user identification to local file directory Different user correspond to different local file directory  Store user file in local file system  Transfer file (download/upload) by servlet  Form distributed, uniformed user file storage space

Yongjian Wang BUAA Bejiing, June 2006 Data Service - Operations CategoryService Operations file or directory operation related mkdir 、 rmdir 、 creatNewFile 、 delete file transfer related setUploadFileName 、 setDownloadFileName

Yongjian Wang BUAA Bejiing, June 2006 GFI Client Side APIs CategoryClient Side APIs Meta operation related isFile 、 isDir 、 exist 、 info 、 getVirtualName 、 List 、 Search File/directory operation related createNewFile 、 delete 、 mkdir 、 rmdir 、 move 、 rename File transfer related Upload 、 DownLoad File/directory access permission related Auth 、 Revoke

Yongjian Wang BUAA Bejiing, June 2006 Upload File Scenario

Yongjian Wang BUAA Bejiing, June 2006 Grid Batch System

Yongjian Wang BUAA Bejiing, June 2006 GOSv2 Architecture

Yongjian Wang BUAA Bejiing, June 2006 Batch System in GOSv2  Local batch system driver Hiding different local batch systems such as OpenPBS 、 LSF and so on  Grid batch service Local batch system service wrapper  Grid batch system client side APIs Interact with batch service by grip Hiding details such as service invocation, file stagein / stageout

Yongjian Wang BUAA Bejiing, June 2006 Architecture of Batch Service

Yongjian Wang BUAA Bejiing, June 2006 Batch Job Descriptor

Yongjian Wang BUAA Bejiing, June 2006 Batch Service Scenario

Yongjian Wang BUAA Bejiing, June 2006 Batch service in the future  GridSAM will act as batch service for different grid node Support JSDL specification Support plain ftp and GridFTP based stagein/stageout Maintain job states Extend GridSAM to support OpenPBS and LSF batch system Replace GridSAM security mechanism with CNGrid security mechanism

Yongjian Wang BUAA Bejiing, June 2006 Grid Batch Accounting System

Yongjian Wang BUAA Bejiing, June 2006 GOSv2 Architecture

Yongjian Wang BUAA Bejiing, June 2006 Grid Batch Accounting System

Yongjian Wang BUAA Bejiing, June 2006 Q&A