1. EGEE is a project funded by the European Union under contract IST-2003-508833 NA3 Strategy for Future Grids Malcolm Atkinson Director of the National e-Science Centre Induction Course, Edinburgh, 27 April 2004 www.eu-egee.org

2. NA3 Induction Course Development, Edinburgh, 27 April 2004 - 2 Contents What is the Web Service Resource Framework  Why has it emerged?  What has it to do with Grids?  What are the parts of WSRF?  What is the status of WSRF? Standards process Implementations Globus Alliance Plans WSRF in Perspective  What is important?

3. NA3 Induction Course Development, Edinburgh, 27 April 2004 - 3 Reminder: what are our goals? Address the challenge  build inter-enterprise systems NOT just connecting systems WORK together to build infrastructure  That persistently and adaptively supports multiple Virtual Organisations  VOs that span organisational structures  Distributed implementation and operation Pioneering new ways of working  John Taylor’s vision We pioneer & transfer results to industry Easy tasks can use any technology Only some of our goals align with industry

4. NA3 Induction Course Development, Edinburgh, 27 April 2004 - 4 Why OGSI adopted web services Expectation: WS would meet several Grid needs  E.g. Standard interface definition language Foundation for better engineering  E.g. Standard invocation mechanism Foundation for interoperability But other channels used for performance  Good commercial tooling (eventually) Reliability and performance Service-Oriented Architecture  Has valuable scalability and durability properties E.g. ICENI using Jini

5. NA3 Induction Course Development, Edinburgh, 27 April 2004 - 5 Web Services components and framework Not a silver bullet or a complete solution!  Most of the engineering effort What you do when you get a message Not how you address, package and deliver it  Most of the standardisation effort Agreeing how to factor large systems and the semantics of services Agreeing conventions for information in messages  Confusing & Rival standards proposals  Limited quality public implementations Don’t give up – engage and help fix it?  Is this the role of EGEE?  Is there just one answer? Incremental adoption of WS-I, WS-Security, WS-Addressing Incremental adoption of WSRF as it emerges We all agree that it is a good strategy to use web services. The issue are: Which ones to adopt when? and What conventions organise our systems?

6. NA3 Induction Course Development, Edinburgh, 27 April 2004 - 6 OGSI & GT3 investment Three forms of investment Architectural effort  OGSA: Use cases, Design Patterns, …  Factoring & describing a complex engineering domain Standardisation effort  OGSI, DAIS, WS-Agreement, etc.  WSDL 2.0, WSDM, WS-Security, etc. Implementation effort  Combined OGSI & Grid component work This investment carries forward into WSRF

7. NA3 Induction Course Development, Edinburgh, 27 April 2004 - 7 Grid Web Several (some partial) implementations Issues: technical, political & commercial Successes: a number of operational grids Started far apart in apps & tech OGSI GT2 GT1 HTTP WSDL, WS-* WSDL 2, WSDM Have been converging ? Combining Grid and Web Services – First try Technology intercept is not easy People accepted OGSI

8. NA3 Induction Course Development, Edinburgh, 27 April 2004 - 8 Not the only possible technical solution Combining Grid and Web Services: Second try Grid Web WSRF Started far apart in apps & tech OGSI GT2 GT1 HTTP WSDL, WS-* WSDL 2, WSDM Have been converging Support from major WS vendors especially service management suppliers e.g., CA, HP, IBM, Fujitsu, BEA, SAP, … Technology intercept is still not easy

9. NA3 Induction Course Development, Edinburgh, 27 April 2004 - 9 Core Ideas in WSRF Preserves OGSI functionality  Lifetime, properties, notification, error types, … Separates service from resource  Service is static and stateless  Resource is dynamic and stateful Builds on WS-Addressing Is WS-I compliant  But note that WS-I alone doesn’t make the problems go away, still need to worry about how to manage lifetime, naming, state, …

10. NA3 Induction Course Development, Edinburgh, 27 April 2004 - 10 “Components” of WSRF WS-AddressingMarch 04 www-106.ibm.com/developerworks/library/specification/ws-add/ WSRF White paper on modelling stateful resources www.globus.org/wsrf/ WS-ResourceLifetimeMarch 04 www.globus.org/wsrf/ WS-ResourcePropertiesMarch 04 www.globus.org/wsrf/ WS-BaseFaultsMarch 04 www.globus.org/wsrf/ WS-RenewableReferences March 04 www.globus.org/wsrf/ WS-ServiceGroupMarch 04 www.globus.org/wsrf/ WS-Notification WS-BaseNotificationMarch 04 …/specification/ws-notification/ WS-TopicsMarch 04 …/specification/ws-topics/ WS-BrokeredNotification March 04 …/specification/ws-pubsub/

11. NA3 Induction Course Development, Edinburgh, 27 April 2004 - 11 From OGSI to WSRF: Refactoring and Evolution OGSIWSRF Grid Service ReferenceWS-Addressing Endpoint Reference Grid Service HandleWS-Addressing Endpoint Reference HandleResolver portTypeWS-RenewableReferences Service data defn & accessWS-ResourceProperties GridService lifetime mgmtWS-ResourceLifetime Notification portTypesWS-Notification Factory portTypeTreated as a pattern ServiceGroup portTypesWS-ServiceGroup Base fault typeWS-BaseFaults Identity & naming is being done by OGSA

12. NA3 Induction Course Development, Edinburgh, 27 April 2004 - 12 GW GT3.2 Improved robustness, scalability, performance, usability 3.2 March 4.0  Q2 4.0 Q3 4.2 Q2 ‘05 Numerous new WSRF-based services GT4.2 GT4.0 WSRF; some new functionality; further usability, performance enhancements 2004 2005 Not waiting for finalisation of WSRF specs. Use as submitted GT & WSRF Timeline OASISGGF10interopTC 1 techPre

13. NA3 Induction Course Development, Edinburgh, 27 April 2004 - 13 Components in GT 3.0 GSI WS-Security Security WS Core Resource Management Data Management RFT (OGSI) RLS WU GridFTP JAVA WS Core (OGSI) OGSI C Bindings Information Services MDS2 WS-Index (OGSI) Pre-WS GRAM WS GRAM (OGSI)

14. NA3 Induction Course Development, Edinburgh, 27 April 2004 - 14 Components in GT 3.2 GSI WS-Security CAS (OGSI) SimpleCA Security Data Management RFT (OGSI) RLS OGSA-DAI WU GridFTP XIO Information Services MDS2 WS-Index (OGSI) Resource Management Pre-WS GRAM WS GRAM (OGSI) WS Core JAVA WS Core (OGSI) OGSI C Bindings OGSI Python Bindings (contributed) pyGlobus (contributed)

15. NA3 Induction Course Development, Edinburgh, 27 April 2004 - 15 Planned Components in GT 4.0 GSI WS-Security CAS (WSRF) SimpleCA Security Authz Framework Data Management RFT (WSRF) RLS OGSA-DAI New GridFTP XIO WS Core JAVA WS Core (WSRF) C WS Core (WSRF) Information Services MDS2 WS-Index (WSRF) Resource Management Pre-WS GRAM WS-GRAM (WSRF) CSF (contribution) pyGlobus (contributed)

16. NA3 Induction Course Development, Edinburgh, 27 April 2004 - 16 Importance of collaboration: VDT A highly successful collaborative effort  VDT Working Group  VDS (Chimera/Pegasus) team Provides the “V” in VDT  Condor Team  Globus Alliance  NMI Build and Test team  EDG/LCG/EGEE Middleware, testing, patches, feedback …  PPDG Hardening and testing  Pacman Provides easy installation capability Currently Pacman 2, moving to Pacman 3 soon Used by many projects Systematic testing Rich integration of components EGEE is part of this – exploit test bed contribute components Thanks to Miron Livny Collaboration is a two way street – or should be!

17. NA3 Induction Course Development, Edinburgh, 27 April 2004 - 17 VDT Timeline Highlights Fall 2001: VDT started by GriPhyN and iVDGL  Supported US-CMS testbed in 2002 March 2002: VDT support system inaugurated Early 2003: Adopted by European Data Grid & LHC Computing Grid April 2003: VDT Testers group started Fall 2003: Supporting Grid3 Fall 2003: Adopted by Particle Physics Data Grid Nov 2003: Nightly test infrastructure deployed Thanks to Miron Livny

18. NA3 Induction Course Development, Edinburgh, 27 April 2004 - 18 VDT Growth VDT 1.0 Globus 2.0b Condor 6.3.1 VDT 1.1.3, 1.1.4 & 1.1.5 pre-SC 2002 VDT 1.1.7 Switch to Globus 2.2 VDT 1.1.11 Grid2003 VDT 1.1.8 First real use by LCG Thanks to Miron Livny

19. NA3 Induction Course Development, Edinburgh, 27 April 2004 - 19 Tools in the VDT 1.1.13 Condor Group  Condor/Condor-G  DAGMan  Fault Tolerant Shell  ClassAds Globus Alliance  Job submission (GRAM)  Information service (MDS)  Data transfer (GridFTP)  Replica Location (RLS) EDG & LCG  Make Gridmap  Certificate Revocation List Updater  Glue Schema/Info prov. ISI & UC  Chimera & Pegasus NCSA  MyProxy  GSI OpenSSH  UberFTP LBL  PyGlobus  Netlogger Caltech  MonaLisa VDT  VDT System Profiler  Configuration software Others  KX509 (U. Mich.) Thanks to Miron Livny

20. NA3 Induction Course Development, Edinburgh, 27 April 2004 - 20 Relative Importance What envelopes you put your messages in  How they are delivered  Infrastructure to organise a common technical platform – the foundations of communication

21. NA3 Induction Course Development, Edinburgh, 27 April 2004 - 21 Relative Importance What envelopes you put your messages in  How they are delivered  Infrastructure to organise a common technical platform – the foundations of communication What information you send in your messages  Their patterns of Use - sequences that mean something  Their Contents  The Grammar and Vocabulary of Communication  Agreed Interpretations

22. NA3 Induction Course Development, Edinburgh, 27 April 2004 - 22 Relative Importance What envelopes you put your messages in  How they are delivered  Infrastructure to organise a common technical platform – the foundations of communication What information you send in your messages  Their patterns of Use - sequences that mean something  Their Contents  The Grammar and Vocabulary of Communication  Agreed Interpretations What you do when you get a message  The Application Code you Execute  The Middleware Services Security, Privacy, Authorisation, Accounting, Registries, Brokers, …  Integration Services Multi-site Hierarchical Scheduling, Data Access & Integration, …  Portals, Workflow Systems, Virtual Data, Semantic Grids  Tools to support Application Developers, Users & Operations Incremental deployment tools, diagnostic aids, performance monitoring, … Technical Experts

23. NA3 Induction Course Development, Edinburgh, 27 April 2004 - 23 Relative Importance What envelopes you put your messages in  How they are delivered  Infrastructure to organise a common technical platform – the foundations of communication What information you send in your messages  Their patterns of Use - sequences that mean something  Their Contents  The Grammar and Vocabulary of Communication  Agreed Interpretations What you do when you get a message  The Application Code you Execute  The Middleware Services Security, Privacy, Authorisation, Accounting, Registries, Brokers, …  Integration Services Multi-site Hierarchical Scheduling, Data Access & Integration, …  Portals, Workflow Systems, Virtual Data, Semantic Grids  Tools to support Application Developers, Users & Operations Creative Actions and Judgements of Researchers, Designers & Clinicians  Data, Models & Analyses  In Silico Experiments, Design, Diagnosis & Planning  Creating the Scientific Record Domain Experts

24. NA3 Induction Course Development, Edinburgh, 27 April 2004 - 24 Conclusions - Strategy If you are making long-term plans  Plan to use WSRF If you develop or research middleware  Engage with groups developing WSRF  If you need those or related functions E.g. to notify or handle state without incremental resource loss If you run distributed Grid operations  Plan to use WSRF  But only when components using it are robust  Incremental transition is possible – even necessary

25. NA3 Induction Course Development, Edinburgh, 27 April 2004 - 25 Conclusions - Tactics Value your team’s skills & momentum Change only if necessary If you’re an applications researcher  Stay with what you have working WS-I +GSI; GT2, VDT, LCG2, GT3, … If you’re a computing researcher  If your platform serves your investigation stay on it WS-I +GSI; GT2, VDT, LCG2, GT3, … If you’re doing middleware R&D  Hard choices & frustrating times – keep going  Those who understand the new order will reap advantage  Therefore engage with WSRF WSRF principles and design patterns  Are a useful guide to building distributed infrastructure  Adopt them, even while WSRF details are being agreed To ease transition

26. NA3 Induction Course Development, Edinburgh, 27 April 2004 - 26 Conclusions - Final WSRF  Good enough for recurrent platform requirements  Has significant commercial and technical momentum  Improves engagement with industry  Only sensible flag to rally behind Must collaborate internationally  Scale of challenge & international virtual organisations Discourage localised alternatives  Avoid effort fragmentation and unnecessary arguments Coping well with transitions … Is a primary Darwinian selector!