Vision of UNICORE – UniGrids and beyond Daniel Mallmann 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006

2 Objectives Develop Unicore/GS Based on Unicore and the Open Grid Services Architecture Compliant with the Web Services Resource Framework Develop generic software components Visualisation and steering of simulations Device monitoring and control Access to distributed data and databases Validate UniGrids concepts and demonstrate the application of Unicore/GS in Industrial and scientific domains

3 Duration, Funding 6 th Framework Programme of the European Commission 2 nd Call of the IST Workprogramme ♦Objective: GRID-based systems and solving complex problems Instrument:STReP Duration:24 months Start:July 2004 Effort:297 PM (12 developer + manager) Costs:3.0 M€ Grant:1.9 M€

4 European Context inteliGRID Semantic Grid based virtual organisations Provenance Trust and provenance for Grids DataminingGrid Datamining tools & services K-WF Grid Knowledge based workflow & collaboration GRIDCOORD Building the ERA in Grid research OntoGrid Knowledge Services for the semantic Grid HPC4U Fault tolerance, dependability for Grid SIMDAT Grid-based generic enabling application technologies to facilitate solution of industrial problems NextGRID EU-driven Grid services architecture for businesS and industry Akogrimo Mobile Grid architecture and services for dynamic virtual organisations CoreGRID European-wide virtual laboratory for longer term Grid research-creating the foundation for next generation Grids Specific support actionIntegrated projectNetwork of excellenceSpecific targeted research project Courtesy: Max Lemke, European Commission UniGrids Extended OGSA Implementation based on UNICORE UniGrids Extended OGSA Implementation based on UNICORE

5 European Context UniGrids

6 Partner Research Centre Jülich (Project manager) Consorzio Interuniversitario per il Calcolo Automatico dell’Italia Nord Orientale Fujitsu Laboratories of Europe University of Warsaw Intel GmbH University of Manchester T-Systems SfR

7 Technology Foundations Develop Unicore Services Job and file management User authorisation Dynamic resource discovery and management Extend security to include explicit trusted delegation Network Job Supervisor Unicore Gateway Unicore Client Target System A Target System B Target System C Resource Database Resource Broker User Database

8 Hosting Environment Build on Jetty Server and Axis SOAP engine WS Resource Framework ♦ResourceProperties ♦Lifetime ♦BasicProfile ♦ServiceGroup WS Notification ♦NotificationProducer ♦NotificationConsumer ♦SubscriptionManager WS Addressing ♦Address URL ♦Resource Disambiguator Technology Foundations

9 Web Service Unicore Gateway HTTP and HTTPS connector Support for other protocols Dispatches incoming SOAP messages based on ♦WS-Addressing headers or ♦URL information from the HTTP transport layer Web Service Unicore User Database Supports different storage formats – default XML file Interface for secure, remote administration Single UUDB can be used by multiple services

10 Usite B Vsite B2Vsite B1 Usite A Vsite A1 Technology Foundations Multisite Job Gateway Internet Target System Interface Network Job Supervisor Target System Interface Network Job Supervisor Client Gateway

11 Secondary Network Job Supervisor Primary Network Job Supervisor SSL Client = User certificate= NJS certificate Job Sub Job Multisite Job Consigner: entity that consigns a job or sub-job ♦Expressed by use in SSL connection Endorser: entity that authorises the tasks to be performed ♦Expressed by signing of serialized AJO direct acyclic graph Technology Foundations

12 Usite A Vsite A1 Explicit Trust Delegation Gateway Internet Target System Interface Network Job Supervisor Client Portal Technology Foundations

13 SSL Network Job Supervisor SSL Portal WS- Client (Browser) Job User: name = User certificate= Portal certificate Explicit Trust Delegation User: new role besides consignor and endorser ♦Entity (user) on whose behalf tasks will be performed Trusted Agents (Portal) ♦Added to the UUDB explicitly ♦Allowed to endorse AJO on behalf of users Technology Foundations

14 Interoperability UniGrids Atomic Services Job Management Storage Management File Transfer Target System UnicoreGlobusCGSP Grid Client Grid Portal Target System Service Job Management Service Storage Management Service File Transfer Service Target System Factory UniGrids Atomic Services Standardisation OGSA-BES Basic Execution Services OGSA-DMI Data Movement Interface

15 Globus Toolkit 4 Hosting Environment Globus Security Target System Factory TSF Target System Service TSS Job Mgmt Service JMS Storage Mgmt Service SMS File Transfer Service FTS GridFTP/GASSGRAM/TSI UniGrids Atomic Service Interfaces UniGrids Atomic Services for GTK4 Available as Open Source (BSD license) at SourceForge (gpe4gtk.sourceforge.net) Interoperability

16 Security Client to server ♦Security provider concept allows to plug in security model on client side GTK4 server to Unicore/GS server ♦Extract original user certificate from proxy certificate chain, pass to UGS (reduced security model) Unicore/GS server to GTK4 server ♦UGS could use proxy certificate from ETD-enabled MyProxy server Generic, standards-based VOMS preferable GSIETD GTK4Unicore/GS Interoperability

17 Expert Client Application Client Portal Client GB Clients and GridBeans GPE Clients work on top of Unicore/GS, GTK4 or ChinaGrid ♦Only requirement: UniGrids Atomic Services Interfaces GridBeans are portable without modifications ♦Client API hides underlying infrastructure details Interoperability

18 Grid Ontology Cooperation with OntoGrid Encoded in OWL, now published on the UniGrids web site ( Permits reasoning and thus can fill in gaps in manually entered descriptions Interoperability

19 Grid Economics Economic Service Infrastructure Full WSRF-Based Resource Broker ♦Use of extensible Economic terms for resource selection ♦Resource consumption feedback from Job Execution Engines Resource Accounting System ♦Publication through WS Resource Properties Payment System ♦Based on MCS Payment protocol ♦Uses broker as agent for user and/or their VO

20 Higher-level Services Visualization and application steering VISualization Interface Toolkit (VISIT) ♦Light-weight library that supports bi-directional data exchange between visualizations and parallel applications VISIT GridBean ♦Manage participants of visualization sessions via VISIT / GS family Framework works with all VISIT - enabled applications Unique Unicore/GS feature for interactive scenarios

21 Higher-level Services Unicore/GS Streaming Framework (UGSF) Components ♦Web service ♦Streaming server ♦Client library Architecture ♦Streaming Service is a stand-alone application that performs streaming ♦Client uses dedicated protocol to connect to the streaming service and obtains real data stream ♦Web service uses RMI to communicate with streaming service and setup streams

22 Higher-level Services File transfer service (including data filter) Exploiting GridFTP Client – Server transfers Server – Server transfers ♦Including SRB Client GridBean available

23 Higher-level Services Access to distributed data, access to databases Interface to OGSA-DAI resources ♦WSRF Data services deployed in Unicore/GS Container ♦OGSADAI GridBean ♦OGSADAI Client

24 Applications Deploy test beds Prove UniGrids concepts in industrial and scientific domains Oil reservoir modelling Bio-molecular, genomic and proteomic Nuclear risk management Re-engineer industrial applications using Unicore/GS WSRF approach

25 Standardisation Influence standardisation process for selected Grid standards JSDL WS-RF OGSA-BES Promote European research results Leverage partner participation in standards organisations SAGA-RG WS-N OGSA-RSS

26 Beyond UniGrids Grid projects exploiting UniGrids results D-Grid ( DEISA ( OMII-Europe (omii-europe.com) A-WARE ( Chemomentum ( EGEE-II ( UNICORE 6 Release Schedule Alpha release ≡ UniGrids results:July 2006 Beta release: mid 2007 Final release: end 2007

27 A-WARE Objectives Simplify usage of Grid ♦Workflow management ♦User focused workflow orchestration ♦Repository to store workflows Fill an existing gap ♦Between middleware and portals

28 Chemomentum Objectives Provide an integrated Grid solution for workflow-centric, complex applications; focus on the end users Provide Grid-enabled applications, data services and knowledge management solutions ♦Risk assessment ♦Toxicity prediction ♦Drug design Test-drive developed services in the context of the European regulatory initiative “Registration and Evaluation of Chemicals” (REACH) ♦Aimed at reducing animal testing, by developing in silico, Grid-based tools

29 OMII Europe Objectives Provide key software components for building e-infrastructures Common interoperable services: ♦Database Access ♦Virtual Organisation Management ♦Portal Infrastructure integration ♦Initial EGEE/UNICORE/Globus interoperability ♦Interoperable security framework ♦Accounting ♦Job Submission ♦Job Monitoring

30 Summary UniGrids built Unicore/GS Based on Unicore and the Open Grid Services Architecture Compliant with the Web Services Resource Framework UniGrids achieved interoperability UniGrids developed generic software components UniGrids results are the base of UNICORE 6 UniGrids results are exploited by several Grid projects