Data Management Expert Panel - WP2

WP2 Overview

Talk Outline u Introduction to EDG Work Package 2 u WP2 Services: Design and Interactions n Spitfire n Replication Services n Grid Simulation n Security u Status

Grid middleware architecture hourglass Current Grid architectural functional blocks: OS, Storage & Network services Basic Grid Services High Level Grid Services Grid Application Services (LCG) Common application layer CMSATLASCMSLHCb Specific application layer GLOBUS 2.2 EU DataGrid WP2

EU DataGrid WP2 Data Management Work Package Responsible for u Transparent data location and secure access u Wide-area replication u Data access optimization u Metadata access NOT responsible for (but partially relying on other WPs for) u Data storage u Proper Relational Database bindings u Remote I/O u Security infrastructure

WP2 Service Paradigms u Choice of technology: n Web Services (servers implemented in Java) s Tomcat, Oracle 9iAS n Interface definitions are exposed in WSDL n Client stubs for many languages (Java, C, C++) s Axis, gSOAP (auto-generated) n Persistent service data in Relational Databases s MySQL, Oracle u Modularity n Modular service design for pluggability and extensibility n No vendor specific lock-ins u Evolvable n Easy adaptation to OGSA foreseen, based on the same technology n Largely independent of underlying OS, RDBMS

Spitfire: Grid-enabling RDBMS u Capabilities: n Simple Grid enabled front-end to any type of local or remote RDBMS through secure web services n Sample generic RDBMS methods may easily be customized with little additional development n Web browser integration n GSI authentication n Hooks in place for local authorization u Status: current release version 2.1 n Used by EU DataGrid Earth Observation and Biomedical applications. n Not currently suitable for the retrieval of LARGE result sets

Storage Element Replication Services: Basic Functionality Replica Manager Replica Location Service Replica Metadata Catalog Storage Element Files have replicas stored at many Grid sites on Storage Elements. Each file has a unique Grid ID (GUID). Replica Location Service maps the GUID to the multiple physical locations of that file. Users may assign aliases to the GUIDs. These are kept in the Replica Metadata Catalog. The Replica Manager provides atomicity for file operations, assuring consistency of SE and catalog contents.

Storage Element Higher Level Replication Services Replica Manager Replica Location Service Replica Optimization Service Replica Metadata Catalog SE Monitor Network Monitor Storage Element The Replica Manager calls the Replica Optimization service to find the best replica based upon network and SE monitoring information.

Storage Element Interactions with other Grid components Replica Manager Replica Location Service Replica Optimization Service Replica Metadata Catalog SE Monitor Network Monitor Information Service Resource Broker User Interface or Worker Node Storage Element Virtual Organization Membership Service Applications and users will manage data only through the Replica Manager - either directly or via the Resource Broker. Management calls should never go directly to the SE.

Grid Simulation (OptorSim) u Standalone data-centric Grid simulation used to develop and evaluate replication strategies - Grid2003: Simulation e.g. of CMS spring 2002 testbed s of jobs, ~100 GB files (50 GB capacity SEs). - Access patterns based on measured CDF analysis jobs. - To add in measured background traffic on network links

Security: Infrastructure for Java- based Web Services u Trust Manager n Mutual client-server authentication using GSI (ie PKI X509 certificates) for all WP2 services n Supports everything transported over SSL u Authorization Manager n Supports coarse grained authorization: Mapping user DN -> role -> attribute n Fine grained authorization through policies, role and attribute maps n Web-based Admin interface for managing the authorization policies and tables u Status: n Fully implemented, authentication is enabled on the service level n Delegation implementation currently being developed n Authorization (using VOMS) currently being integrated with WP2 services.

WP2 Status u Current Status n All components are available now n Initial tests show that expected performance can be met n Need proper testing in a real user environment – EDG2; LCG1 n Good results from OptorSim. Work continuing. u Work-plan for next release n Full integration of the authorization module. n Replica Location Index. n See James talk.