Presentation is loading. Please wait.

Presentation is loading. Please wait.

Grid Interoperability Through Standards Dr. Alistair Dunlop Project Manager.

Similar presentations

Presentation on theme: "Grid Interoperability Through Standards Dr. Alistair Dunlop Project Manager."— Presentation transcript:

1 Grid Interoperability Through Standards Dr. Alistair Dunlop Project Manager

2 EU project: RIO31844-OMII-EUROPE The OMII-Europe Standardisation process Globus OMII-UK CROWN Components IN OUT Re-engineering Compliance testing and QA Benchmarking Component Integration Repository Evaluation Infrastructure Identifying new components Standards Implementation

3 EU project: RIO31844-OMII-EUROPE Session Outline Overall architecture and components in OMII-Europe (Morris Riedel) Standards and implementations from OMII-Europe –SAML (Valerio Venturi) –BES/JSDL (Stephen Crouch) –RUS and UR (Gilbert Netzer) –GLUE II (Sergio Andreozzi) –DAIS, DMI and ByteIO (Neil Chue Hong) Future services

4 EU project: RIO31844-OMII-EUROPE Future Work: Common implementations of specification and services for: –Data storage services Common data service layer building on DAIS, ByteIO, DMI and GSM –Service discovery –Grid monitoring and service mangement –Grid billing and pricing –Grid visualisation and steering –Grid Authorisation

5 Architecture and Security Open Standards used in real interoperability scenarios and use cases Morris Riedel, Forschungszentrum Juelich (FZJ), Jülich Supercomputing Centre (JSC), Germany Leader Infrastructure Integration (Interoperability) Activity Open Grid Forum 21, Seattle, 17th October 2007

6 EU project: RIO31844-OMII-EUROPE Outline Motivation: e-Infrastructure Islands in Europe OMII-Europe in Context Common Vision: Interoperability Highway Interoperability scenarios / use cases Infrastructure to test interoperability scenarios Interoperability scenario: WISDOM Conclusion

7 EU project: RIO31844-OMII-EUROPE DEISA Grid (Supercomputing/HPC community) –Non WS-based UNICORE 5: Proprietary jobs (AJO/UPL) –No Virtual Organization Membership Service (VOMS), Full X.509 –Suitable for massively parallel scientific jobs (MPI, …) EGEE Grid (mainly HEP community + others) –Non WS-based gLite: Proprietary jobs (JDL) –Proxy-based X.509 security, but proprietary VOMS support –Suitable for embarrassingly parallel scientific jobs Both Grids are currently not technical interoperable –Scientists cannot use one middleware to access both –Both Middlewares had so far less adoption of open standards e-Infrastructure Islands in Europe DEISA [2] EGEE [3]

8 EU project: RIO31844-OMII-EUROPE OMII – Europe in Context EC e-Infrastructures [4]

9 EU project: RIO31844-OMII-EUROPE Grid Middlewares Emerging Open Standards others End-users via clients & portals Interoperability highway based on open standards GOAL: Transparency of Grids for end-users Grid Resources others Common Vision: Interoperability Highway UNICORE [5]gLite [6]GT [7]CROWN [8]

10 EU project: RIO31844-OMII-EUROPE Infrastructure to test interoperability scenarios Next steps in RED/ORANGE SAML- based Attribute Authority (AA) VOMS gets central role & middleware independent Venturi et al. [9]

11 EU project: RIO31844-OMII-EUROPE OMII – Europe implements open standards… –OGSA-BES, OGSA-RUS, WS-DAIS (OGSA-DAI), SAML (VOMS)… E.g: OGSA - Basic Execution Services (OGSA-BES) –In real deployments is not a vanilla OGSA-BES interface available –Same exact client works not directly with gLite & UNICORE E.g. different security models: X.509 Proxies vs. full X.509 certificates Solution in OMII-Europe: Agree on a Common Security Profile Grid interoperability: use components together… –e.g. OGSA-BES job submission secured via VOMS AuthZ Interoperability scenarios / use cases: e-Infrastructure use cases requiring resources in more than one Grid! Interoperability Scenarios / Use cases

12 EU project: RIO31844-OMII-EUROPE The following interoperability scenarios have been discussed with members of the respective projects These scenarios demonstrate the required technical interoperability between e-Infrastructures in future that can be provided by OMII-Europe However, the negotiation for getting computational time on resources within these e-Infrastructures is still required by the respective projects Disclaimer

13 EU project: RIO31844-OMII-EUROPE WISDOM (Wide In Silicio Docking on Malaria) –WISDOM aims at developing new drugs for Malaria –WISDOM uses EGEE for large scale in silicio docking Comp. method for prediction of whether one molecule will bind to another –Using AutoDock and FlexX software provided via gLite in EGEE –Output is a list of best chemical compounds (potential drugs) That is not the final solution, only a potential list of drugs Refine best compound list via molecular dynamics(MD) –Fast MD computations use highly scalable AMBER in DEISA AMBER (Assisted Model Building with Energy Refinement), version 9 Goal: Accelerate drug discovery using EGEE + DEISA Interoperability Scenario: WISDOM (1) WISDOM [1]

14 EU project: RIO31844-OMII-EUROPE Interoperability Scenario: WISDOM (2)

15 EU project: RIO31844-OMII-EUROPE Conclusion Solutions for European e-Infrastructure Islands –Adopt components from OMII-Europe to gain interoperability Why OMII – Europe components… –Components going through quality assurance steps –Components adopt standards whereever possible –Components tested in real interoperability scenarios (e.g. WISDOM) Not only interesting for European e-Infrastructures –Beneficial for all Grids that adopt Globus, gLite, and UNICORE E.g. National German Grid Initiative (D-Grid) Continuing work in the open standards working groups! –Interoperability highway… realize the true global Grid

16 EU project: RIO31844-OMII-EUROPE References [1] WISDOM Project, [2] DEISA Project, [3] EGEE Project, [4] European e-Infrastructures, [5] European UNICORE Grid middleware, [6] European gLite Grid middleware, [7] US Globus Toolkit, [8] CROWN, [9] Using SAML-based VOMS for Authorization within Web Services-based UNICORE Grids, Venturi et al., UNICORE Summit EuroPar 2007 [10] Improving Quantum Computing Simulations,

17 EU project: RIO31844-OMII-EUROPE Acknowledgements Open Middleware Infrastructure Institute for Europe –OMII – Europe project under EC grant RIO31844-OMII-EUROPE, duration May April 2008 Jülich Supercomputing Centre (JSC) of Forschungszentrum Jülich (FZJ) in the HELMHOLTZ association Forschungszentrum Jülich in der Helmholtz-Gesellschaft

18 EU project: RIO31844-OMII-EUROPE e-Science 2007 International Grid Interoperability & Interoperation Workshop in conjunction with e-Science 2007, Bangalore, India

19 EU project: RIO31844-OMII-EUROPE IQCS (Improving Quantum Computater Simulation) –One project of DECI (DEISA Extreme Computing Initiative) Evaluation period/test runs of IQCS code in EGEE –IQCS scientists use currently their own desktop machines or some spare resources within institutes to develop/evaluate/test their code –Thats slow and the management doing it personally is cumbersome This can be significantly improved using EGEE clusters! Final IQCS code production runs within DEISA –Leverage massive amount of CPUs/memory available on HPC resources Goal: Accelerate code development with EGEE + DEISA Optional: Interoperability Scenario: DECI IQCS DEISA DECI IQCS [10]

20 EU project: RIO31844-OMII-EUROPE Optional: Interoperability appraoch in OMII-Europe e-Infrastructure Interoperability by using more than one technology! Venturi et al. [9]

21 EU project: RIO31844-OMII-EUROPE Questions for JRA3 – Task 2 Morris Riedel JRA 3 Team

22 EU project: RIO31844-OMII-EUROPE Attribute Exchange Profile Problem statement –Interoperable, pluggable, replaceable attribute services –Easier aggregation of attributes coming from difference sources OGSA Architecture envision the use of authorization policies coming from different stakeholders Solution –Standard interface for requesting attribute assertions

23 EU project: RIO31844-OMII-EUROPE Attribute Exchange Profile Problem statement –Attributes assertions understood across middleware boundaries Solution –Agreed format for attribute assertions

24 EU project: RIO31844-OMII-EUROPE Status of the specification The specification is being worked on inside the OGF OGSA Authorization Working Group –Basically profiles existing specification (OASIS, WS-I) for use with Grid Services Bases are solid –OASIS SAML V2.0 set of specifications SAML V2.0 Deployment Profiles for X.509 Subjects Profile presented on Monday Those interested should join the OGS Authorization WG

25 EU project: RIO31844-OMII-EUROPE Specification Implementers VOMS –gLite and OMII-Europe developed, used in EGEE, OSG, Naregi –Implementing the service UNICORE –Implementing the client part of the profile in OMII- Europe Part of the 6.2 release planned for early 2008 –Implementing the service in the chemomentum project GridShib

26 EU project: RIO31844-OMII-EUROPE Implementation status OMII-Europe is re-engineering the Virtual Organization Membership Service (VOMS) to meet the specification VOMS is developed also in EGEE VOMS is an Attribute Authority focused on VO management –It allows users to register to a VO and being assigned attributes such as groups and roles –The attributes are used by Grid Services to drive authorization decisions VOMS is already used in EGEE, OSG and Naregi –Plugin for the globus Authorization framework OMII-Europe is also adding VOMS support to UNICORE –UNICORE is implementing the client part of the profile

27 EU project: RIO31844-OMII-EUROPE Implementation Status Alpha version of the VOMS SAML Service available since May 2007 for integration testing purposes –Demonstrated at OGF 20 in Manchester in conjunction with UNICORE OGSA-BES Beta version available early November –Then into the QA process of OMII-Europe, that comprises standard compliance testing VOMS SAML Fed back to gLite early 2008 OMII-Europe support for VOMS –Available from version 6.2 early 2008

28 EU project: RIO31844-OMII-EUROPE Implementation Plans Undergoing integration with BES services re-engineered by OMII-Europe –UNICORE OGSA BES already done, demonstrated at OGF 20 –gLite CREAM BES nearly done –May be demonstrated at SC07 Planned integration with other OMII-Europe re-engineered components –RUS services: SGAS, gLite DGAS, UNICORE OGSA RUS –OGSA DAI Integration with BES services is a proof of concept of the first addressed issue –Availability of attributes across middleware boundaries

29 EU project: RIO31844-OMII-EUROPE Further plans Shibboleth 2 on its way out –GridShib will implement the same interface We plan to test integration scenarios –Shibboleth attributes concern Home Institution Affiliation –VOMS attributes concern Virtual Organization This would be proof of concept of the second issue addressed –Easy aggregation of attributes coming from different sources

30 Job Submission: OGSA-BES & JSDL Moreno Marzolla Stephen Crouch SOTON

31 EU project: RIO31844-OMII-EUROPE Contents The problem of incompatible job submission systems The solution: interoperability through standards –OGSA-BES –JSDL Standards status and limitations Implementations of standards within OMII-Europe –Computation endpoints –Meta-scheduler Future interactions with other OMII-Europe activities

32 EU project: RIO31844-OMII-EUROPE Setting the Scene: Grid Job Submission User problem: –User (e.g. researcher) has a problem they need to solve computationally –They do not have local access to appropriate resources in terms of computation and/or applications A non-local provider: –Has the required computation/application resources –Wishes to allow them to be shared by appropriate users Grid job submission provides the technological glue to allow users to utilise the providers computational/application resources User Job Comp/App Resources Job Submission

33 EU project: RIO31844-OMII-EUROPE The Problem Different job submission systems often incompatible in terms of –Interface –Job description –Security (more general problem) Overhead in terms of –Use – learning curve –Infrastructure support – multiple client installations –Application development – knowledge of multiple APIs Interoperation solutions expensive in terms of development & maintenance Client App Comp/App Resources Job Submission A Client (A) Client (B) Job Submission B Comp/App Resources … …… Desc A Desc B

34 EU project: RIO31844-OMII-EUROPE Solution: Interoperability through Standards Two key standards for two key elements: –The Basic Execution Service web service interface (OGSA-BES) 1 instance of job container within OGSA-EMS (Execution Mngt Service) Handles basic job lifecycle management Defines simple (but extendable) job state model –Pending, running, cancelled, failed or finished –The Job Submission Description Language (JSDL) Specify job executable, data staging and resource requirements Client App Comp/App Resources Job Sub. A Job Sub. B Comp/App Resources BES Client JSDL BES I/F JSDL …… …

35 EU project: RIO31844-OMII-EUROPE Summary of OGSA-BES BES-Management Port-type StopAcceptingNewActivitiesRequest that the BES stop accepting new activities StartAcceptingNewActivitiesRequest that the BES start accepting new activities BES-Factory Port-type CreateActivityRequest the creation of a new activity GetActivityStatusesRequest the status of a set of activities TerminateActivitiesRequest that a set of activities be terminated GetActivityDocumentsRequest the JSDL documents for a set of activities GetFactoryAttributesDocumentRequest XML document containing BES properties BES-Activity Port-type (optional) GetStatusRequest the status of an activity TerminateRequest that an activity be terminated GetDocumentRequest the JSDL document for an activity GetActivityAttributesDocumentRequest XML document containing activity properties

36 EU project: RIO31844-OMII-EUROPE JSDL Example I cat job cat job description no annotation an example project /bin/cat dir1/file1.txt stdout.txt stderr.txt

37 EU project: RIO31844-OMII-EUROPE JSDL Example II dir1/file1.txt overwrite stdout.txt overwrite true output/stdout.txt

38 EU project: RIO31844-OMII-EUROPE Standards Status & Limitations Both JSDL & OGSA-BES have reached version 1.0 (recommendation) Future support being considered –Parametric jobs – being considered as JSDL extension –Collections of jobs with dependencies Neither addresses security –Fundamental requirement for interoperability in real deployments –Mutually compatible security model (JRA3/Security, JRA1/VOMS)

39 EU project: RIO31844-OMII-EUROPE Implementations: Endpoint Support for BES & JSDL BES 1.0/JSDL 1.0 support developed in OMII-Europe for –UNICORE 6.1 (Jan/Feb 2008): BES/JSDL support in vendor middleware –gLite 3.1: CREAM-BES component based on CREAM-CE Available through OMII-Europe repository in Dec 2007 –Globus 4.2: Some BES/JSDL support exists within vendor middleware Work progressing – target release date TBD Support for BES within other job submission implementations: –OMII-UK, Beihang, Microsoft, University of Virginia, Platform Computing

40 EU project: RIO31844-OMII-EUROPE Implementations: Meta-scheduling support for BES & JSDL BES/JSDL 1.0 support in CROWN –Progressing towards support for BES 1.0 CROWN Meta-scheduler hosted within Globus 4.2: –Accept BES submissions, delegate to BES endpoints based on policy –Release targeted March 2007, available from OMII Europe repository Res. Job Sub. A BES I/F Res. Job Sub. B BES I/F Client App BES Client JSDL Meta Scheduler BES I/F JSDL

41 EU project: RIO31844-OMII-EUROPE Future Interactions with OMII-Europe Activities Security (JRA3) –Support common security profile Compliance testing (SA2) w.r.t. OGSA-BES/JSDL –Test endpoints –Meta-scheduler plugs in to compliance test Benchmarking (JRA4) Training (NA3)

42 Accounting in OMII-Europe Gilbert Netzer (KTH)

43 EU project: RIO31844-OMII-EUROPE Outline Accounting Standards –Overview –Usage Record Format –Resource Usage Service OMII-Europe Accounting Activity –Goals –Used Standards –Schedule –OMII-Europe Accounting Scenarios Questions

44 EU project: RIO31844-OMII-EUROPE Benefits of Standardization Many (batch) systems are alike –All batch systems collect similar metrics –However units, data format and semantics differ Large systems are heterogeneous –Standards help in building these systems –Off-the-shelf components can be plugged together Standards give guidance for implementations –Define parts of the requirements –Can help defining the scope of an application Promote sharing of knowledge

45 EU project: RIO31844-OMII-EUROPE OGF Standards for Accounting Usage Record Format (URF, GFD-R-P. 98) –Recommends a document format for usage information –Does not address transport of information –Proposed Recommendation status –Interoperability workshop scheduled for OGF22 Resource Usage Service (RUS, GWD) –Specifies means to transport usage information –Uses the URF as data format –Working Draft stage –Meetings were held at OGF19, 20 and 21

46 EU project: RIO31844-OMII-EUROPE Accounting Standards Usage Scenario

47 EU project: RIO31844-OMII-EUROPE OMII-Europe Goals Make existing accounting infrastructure interoperable Use OGF standards as vehicle: –Usage Record Format (UR) –Resource Usage Service (RUS) Address the three middlewares of the project: –Distributed Grid Accounting System (DGAS) –SweGird Accounting System (SGAS) –UNICORE-RUS First step, focus on the existing basics only Augment existing systems to allow easy transition

48 EU project: RIO31844-OMII-EUROPE OMII-Europe Schedule Alpha versions demonstrated at OGF20 (Manchester) –DGAS client uploads/downloads from SGAS server –UNICORE demonstrated using RUS for monitoring Beta versions November-December 2007 –Will have both server and client components for all three middlewares –Will be available for download from repository Final versions for general public release –Will be ready in March 2008 Software will be released via OMII-Europe Repository –We also try to feed SW back to the MW distributions

49 EU project: RIO31844-OMII-EUROPE Alpha Demonstration - OGF20 OMII-Europe added value (RUS) UNICOREgLite DGASGlobus SGAS End User Applications Services Resources LLViewDGAS ClientsSGAS Clients UNICORE RUS LUTS UNICORE UR Generator DGAS Sensor UNICORE Alpha at German eScience 2007

50 EU project: RIO31844-OMII-EUROPE OMII-Europe Accounting Scenario OMII-Europe added value (RUS) UNICOREgLite DGASGlobus SGAS End User Applications Services Resources LLViewDGAS ClientsSGAS Clients UNICORE RUS HLR RUS Service LUTS UNICORE UR Generator DGAS SensorJARM Legacy Interaction

51 EU project: RIO31844-OMII-EUROPE Standards in OMII-Europe Accounting Phase I (Current Project) –Uses RUS draft version 17 Is available Has passed public comment phase –Uses late URF draft This is used by RUS draft 17 Different namespace than GWD-R.P. 98 Minor differences (e.g. new GlobalUserName) –RUS is work in progress! Phase II (Next Project, ) –Upgrade to final RUS&URF recommendation

52 EU project: RIO31844-OMII-EUROPE Usage Record Format XML Document Format for Usage Information Approach –Defines many elements for different metrics –Most elements optional, implementation picks –Extension framework for further properties Limitations –Only computational jobs considered in version 1 –One record per job, no summarization –VO membership information was not considered –Limited extension framework, only string content

53 EU project: RIO31844-OMII-EUROPE Anatomy of a UsageRecord PBS scottmo 2870 completed 1234 BottomFeeder 4 mscfops Colony PT1S T17:34:50Z T18:37:38Z 2 batch 1435 NWChem nwchem_linux Unique ID & Timestamp Reference to Job Reference to User Usage Information Extensions

54 EU project: RIO31844-OMII-EUROPE Anatomy of the Resource Usage Service Defines a web-service interface (draft 17): –Store (insert) Usage Records –Extract information using XPath queries Whole UsageRecords wrapped with audit information Only the ID assigned by the RUS (not RecordId!) –Modify stored Usage Records Replace whole UR with new UR Modify UR using XUpdate expression Atomically increment numeric property of a UR –Delete stored Usage Records by ID or XPath query –Get local constraints (mandatory elements)

55 EU project: RIO31844-OMII-EUROPE Limitations of RUS Extract always returns results in one message –Problems with large queries (memory, timeouts) –Planned to support enumerations No summarization possible –Cannot return e.g. total jobs this month –Planned for Advanced RUS specification Not aware of VOs in standard way –Can use UR extensions, standardization in URF v. 2 No federation support –Need to always query whole set, no diffs.

56 EU project: RIO31844-OMII-EUROPE OMII-Europe's Contribution in Accounting Usage Record Working Group –Plan to supply UR-data to interop WS at OGF22 Resource Usage Service Working Group –2 of 3 co-chairs come from OMII-Europe –Will provide three independent implementations –Will provide implementation experience

57 EU project: RIO31844-OMII-EUROPE References OMII-Europe Website: UR-WG GridForge RUS-WG GridForge

58 GLUE 2: Common Information Model for Grid resources Sergio Andreozzi, INFN 17 Oct 2007, Seattle, WA, USA

59 EU project: RIO31844-OMII-EUROPE Problem Statement How do we describe resources shared in Grid systems in order to enable: –Resource awareness –Resource discoverability –Resource requirements expression –Resource basic monitoring

60 EU project: RIO31844-OMII-EUROPE Use Case 1 I want to run my job on an execution environment characterized by: –OS Linux, Distribution X, version Y –CPU Archicture IA64 –Available software packages: S1, S2

61 EU project: RIO31844-OMII-EUROPE Use Case 2 I want to know –how many job slots are used by members of the VO A –what is the global available storage space for the users of VO B

62 EU project: RIO31844-OMII-EUROPE Focus on Computing Resources OGSA-BES and JSDL are already considered by OMII-Europe They lack a common description of Grid resources suitable for discovery, monitoring and scheduling Many descriptions exist –e.g.: GLUE Schema, NorduGrid Schema Barrier to interoperable Grid systems

63 EU project: RIO31844-OMII-EUROPE How Are We Contributing Co-leading the OGF GLUE WG which purpose is to define the next-generation information model for Grid resoruces as a community standard OGF GLUE WG Goals: –define a use case document collecting use cases from different Grid projects/infrastructures –define a conceptual model defining the abstract schema GLUE 2.0 satisfying the collected use cases. –develop reference implementations Timeline: go public comments by OGF 22 (Jan 2008)

64 EU project: RIO31844-OMII-EUROPE Relationship to other OGF WGs JSDL GLUE Reference Model SAGA GSM OGSA Res.Mgt. Used to express requirements in Common service descripton for discovery API Used to describe exposed resources Should fit into the picture BES Used to describe exposed resources

65 EU project: RIO31844-OMII-EUROPE How Are We Contributing Rendering of the GLUE conceptual model as XML Schema and inclusion into the OMII-Europe repository –Other renderings like LDIF are expected implementation of information providers –for BES implementations developed in the context of OMII- Europe –leveraging DMTF standards Inclusion of this capability into the OMII-Europe Repository

66 EU project: RIO31844-OMII-EUROPE Activity Status OGF Documents –Specification and Use Cases drafts available – Rendering as XML Schema –First version to be issued in Nov 2007 based on OGF21 draft Information providers –Prototype framework based on OpenPegasus to be released in Nov 2007 Production Quality full implementation by Apr 2008

67 EU project: RIO31844-OMII-EUROPE Adoption and Related Efforts The output of this activity will be adopted by the OMII-Europe middleware providers The software will be publicly accessible from the OMII-Europe repository and will be advertised for larger adoption Other projects will make their own implementation of the GLUE spec (e.g. ARC/NorduGrid)

68 Data Specs: DAIS, OGSA ByteIO, OGSA-DMI Neil Chue Hong OMII-UK

69 EU project: RIO31844-OMII-EUROPE Overview Data specifications and OMII-Europe –Database Access and Integration Service (DAIS) –OGSA ByteIO (ByteIO) –OGSA Data Movement Interface (OGSA-DMI) What is the problem What is the status of the specification What is the status of the implementations

70 EU project: RIO31844-OMII-EUROPE Uniform access to structured data Each database vendor typically supports different methods of access and different data models –difficult to federate DAIS provides a uniform interface for data access that allows –location independence –heterogeneity –composability It also presents interfaces that allow virtualisation of database resources, and result sets

71 EU project: RIO31844-OMII-EUROPE DAIS Specifications WS-DAI WS-DAIRWS-DAIX WS-DAIO Sets general pattern for DAIS realisations Currently defined Draft at GGF14 RDF In discussion

72 EU project: RIO31844-OMII-EUROPE Data Service Composition GDS Client GDS Client 1 Operation GDS Client 2 DB Operation DB 4 Operation DB GDS 3 Operation DB GDS Client 5 Operation DB GDS

73 EU project: RIO31844-OMII-EUROPE Complex Scenario

74 EU project: RIO31844-OMII-EUROPE Interact with multiple resources in a workflow

75 EU project: RIO31844-OMII-EUROPE DAIS The OGSA-DAI project has been heavily involved in standards process –implementation of DAIS proposed recommendation in progress Work in OMII-Europe has concentrated on –interoperability across Globus, UNICORE and gLite –integration with VOMS security Other implementations of DAIS from –Ohio State University –EGEE (DAIS interface to AMGA) –IBM

76 EU project: RIO31844-OMII-EUROPE DMI Scenario

77 EU project: RIO31844-OMII-EUROPE User moves data from A B Does not care how it is done Does not want to: –Deal with different protocols –Deal with different security models –Learn new interface for different data types Handle structured & un-structured data –Agnostic about the data semantics 77

78 EU project: RIO31844-OMII-EUROPE DEPR OGSA-DMI in a nutshell 78 Data Transfer Instance Port Type User B A GridFTP file Services capable of placing data at the sink DEPR Existing Data Transfer protocol Existing data transfer protocols OGSA-DMI protocol http file GridFTP Services capable of retrieving data from the source Source Location Sink Location The Problem! Client Transfer A B EPR Data Transfer Factory Port Type Service

79 EU project: RIO31844-OMII-EUROPE ByteIO: POSIX like A concise, standard way of interacting with grid data resources as POSIX-like files –interfaces defined for random access and streamable With other specs, allows a file and directory-like view of a Grid and many other simplified views of data –shell, file explorer, database result sets, pipes ByteIO Spec out of public comment with four implementations –UNICORE implementation by FZJ –OGSA-DAI implementation by EPCC –Clean room implementation by FLE –Genesis II implementation by UVa

80 EU project: RIO31844-OMII-EUROPE Staging files in Job Submission HPC Profile enables interoperable job submission systems OGSA-DMI provides standard way of staging files in and out OGSA ByteIO allows part of a large remote file to be staged in without transferring the whole file

81 EU project: RIO31844-OMII-EUROPE Summary OMII-Europe is a 24 Month EU funded project with 16 partners to establish grid infrastructure interoperability through implementing a set of agreed open standards on all middleware platforms OMII-Europe is implementing a common Job submission system, Accounting service, Database service, Virtual Organisation service, Information Service and Security model for major middleware platforms. This will allow identically specified jobs to be run, managed and migrated to different middleware platforms thus enabling the Global Grid Initial versions of BES, VOMS and security service have already enabled UNICORE and EGEE resources to be used by the same job A complete set of fully interoperable services will be available in spring 2008 with early versions of some services available already Users can try interoperability on the OMII-Europe evaluation infrastructure, or obtain services for installation on their own resources from the OMII-Europe repository We anticipate OMII-Europe services to be integrated into standard middleware distributions as well as deployed on large scale e-infrastructures such as EGEE and DEISA OMII-Europe will request continuing funding in the September EU call to support the existing services and provide further services in the areas of data and grid management

82 EU project: RIO31844-OMII-EUROPE Further Information Q & A

Download ppt "Grid Interoperability Through Standards Dr. Alistair Dunlop Project Manager."

Similar presentations

Ads by Google