1. Experiences of the Grid… Gavin McCance University of Glasgow NeSC Meeting, 24 October 2001

2. 2/38NeSC 24 October 2001Gavin McCance, University of Glasgow Background Experimental Particle Physics background Analysing the structure of matter …Fortran (19)77 ! Working in Grid-like areas since January this year

3. 3/38NeSC 24 October 2001Gavin McCance, University of Glasgow GridPP 20+ institutes…

4. 4/38NeSC 24 October 2001Gavin McCance, University of Glasgow …GridPP £17M 3-year project Working in collaboration with EU DataGrid project Middleware production Integration of middleware technologies into HEP experiments Validation of Grid Software

5. 5/38NeSC 24 October 2001Gavin McCance, University of Glasgow …GridPP Initial GridPP testbed underway A personal snapshot of activities on the grid… Middleware activities were involved in Some examples Technologies were using Issues with integration of Grid with particle physics experiments

6. 6/38NeSC 24 October 2001Gavin McCance, University of Glasgow Middleware What is middleware…??? Application programs – local gridopen() Data access specifics – HPSS, Castor Job submission specifics – PBS, LSF Specific security procedures Grid middleware Layered APIs. Transparent security. Transparent data access. Intelligent use of distributed resources.

7. 7/38NeSC 24 October 2001Gavin McCance, University of Glasgow Middleware Activities GridPP ~mirrors EU DataGrid: Workload Management What jobs go where? Data Management (*) Wheres the (best) data? Information Services (*) Whats the state of everything?

8. 8/38NeSC 24 October 2001Gavin McCance, University of Glasgow …Middleware Activities Fabric Management Interfaces to underlying systems Mass Storage Management How to get the data to/from the fabric e.g. Implementing file-save() APIs for different mass storage systems Security Crops up everywhere … transparent to applications

9. 9/38NeSC 24 October 2001Gavin McCance, University of Glasgow Data Management Data Replication Transparent and Secure Data Access Meta Data Storage Query Optimisation

10. 10/38NeSC 24 October 2001Gavin McCance, University of Glasgow Example problem: Data Replication Problems if data exist only in one place Multiple accesses to the same data overload network! Petabytes! Funding constraints! e.g. CERN cant store all of the data required Make Replica! But need to keep track of all the files and their various replica! Need replica catalogue!

11. 11/38NeSC 24 October 2001Gavin McCance, University of Glasgow …Catalogues Examples solutions: Have a globally unique Logical File Name (LFN) mapping to multiple physical instances of the file (PFNs). Replica selection required Choose the best / nearest / fastest Cost modelling… how time expensive to transfer files X from A to B File-1 Paris Glasgow Chicago LFN

12. 12/38NeSC 24 October 2001Gavin McCance, University of Glasgow …Data Replication Grid Data Mirroring Package C, C++, JAVA, command-line APIs Replication issues: File transfer… Synchronisation / consistency models Basic middleware doesnt enforce any policy Scalable architectures

13. 13/38NeSC 24 October 2001Gavin McCance, University of Glasgow …GDMP File transfer uses GridFTP Existing IETF-approved (?RFC?) ftp additions + the standard grid security (GSI) Registers new files in replica catalogue E.g. interfaced to the existing Globus Replica Catalogue Basic replica manager functionality to maintain consistency of replica sets

14. 14/38NeSC 24 October 2001Gavin McCance, University of Glasgow …Implementation issues Structure not imposed by the middleware software itself… But … must think about scalable implementations E.g. a RC may exist on each storage element responsible for its own files CERN Root RC CERN RCUK RCINFN RC Queries will propagate down until replica information is found…

15. 15/38NeSC 24 October 2001Gavin McCance, University of Glasgow …Longer term problems Query / Replica Optimisation Grid can make / delete replica Eg. Many people in Glasgow & Edinburgh access the ATLAS Higgs dataset A1… Autonomously make new replica in / near Scotland based on historical information Grid might re-cluster data A2 A3 A1 B1 B3 B2 GlasgowParis A2 A3 A1 B1 B3 B2 GlasgowParis

16. 16/38NeSC 24 October 2001Gavin McCance, University of Glasgow …longer term real Grid... …simulated Grid provides testing arena for more adventurous ideas! MONARC simulation tool

17. 17/38NeSC 24 October 2001Gavin McCance, University of Glasgow …Integration of middleware Many iterations of requirements and use-cases with end-users… meetings… Middleware solutions must be scalable and useable by a variety of end users HEP, Biological, Earth sciences, Astro Always looking for common elements E.g. replica / meta-data catalogues… data transport… security…

18. 18/38NeSC 24 October 2001Gavin McCance, University of Glasgow …examples of common interfaces: generic meta-data catalogue tools SQL Database Service: Problem: many relational databases, diverse security, diverse wire protocols …Solution: Build on existing wire protocols: XML transported over HTTP(S) Grid standard security framework (GSI)

19. 19/38NeSC 24 October 2001Gavin McCance, University of Glasgow..examples Leverage open-source technology JAVA servlet based (Apache Tomcat engine) JDBC drivers Utilises Oracles XSQL servlet (open source) Security over HTTPS with Grid-standard GSI mechanism

20. 20/38NeSC 24 October 2001Gavin McCance, University of Glasgow …examples Allows any HTTP compliant system e.g. Web- browsers / standard C++ HTTP libraries to access any relational database… = SQL Database Service (Spitfire) Oracle PostgreSQL + PKI Security + Standard communication protocols (XML over HTTPS)

21. 21/38NeSC 24 October 2001Gavin McCance, University of Glasgow Global Grid Forum Global Grid Forum meetings GGF1: Amsterdam meeting in April 2001 Helps define aspects common to all Grid-like projects. E.g. architectures, grid protocols As example… Grid Monitoring Architecture (GMA)

22. 22/38NeSC 24 October 2001Gavin McCance, University of Glasgow Information Services - GMA One Implementation of the GMA Globus MDS, currently based on (Open)LDAP Hierarchical directory like structure Very fast for information retrieval if you already know the query designed into structure. Bad for complex or ranged queries

23. 23/38NeSC 24 October 2001Gavin McCance, University of Glasgow..complementary implementation Producer Producer API Register, re-register, publish Relational Database Consumer Querying API Query Producer Servlet Registry Servlet Schema Servlet subscribe stream Implementation of GMA Relational queries in SQL format

24. 24/38NeSC 24 October 2001Gavin McCance, University of Glasgow …relational GMA Information is transferred in generic SQL format… Producers of information register themselves… Consumers construct (possibly complex) SQL query and are streamed query results directly from Producers.

25. 25/38NeSC 24 October 2001Gavin McCance, University of Glasgow …implementation Again, uses JAVA servlets Tomcat servlet engine Again, communication with servlet is over standard HTTP. All the internal parts communicate via HTTP and XML modular design, easily replaceable…

26. 26/38NeSC 24 October 2001Gavin McCance, University of Glasgow Useful Tools… JAVA… nicely platform independent UML Universal(?) Modelling Language Architecture and APIs should be defined in this…! CASE tools Together Control Centre

27. 27/38NeSC 24 October 2001Gavin McCance, University of Glasgow …useful tools Globus toolkit Both the original and its java implementation (CoG) My experience of CoG so far is generally good…! Easy GSI authentication, Globus file transfer, Globus job submission, MDS interface

28. 28/38NeSC 24 October 2001Gavin McCance, University of Glasgow Testbeds For GridPP, primary testbeds are the HEP experiment ones CERN LHC (EU DataGrid WP8) US experiments, e.g. Fermilab, SLAC First software release now!! Integration team show-and-tell at CERN end of this month…

29. 29/38NeSC 24 October 2001Gavin McCance, University of Glasgow...testbed work Grid software packaged for release to experiments! Primarily packaged using RPM For end of October release, supported platforms are: Linux (and Solaris on a best effort basis)

30. 30/38NeSC 24 October 2001Gavin McCance, University of Glasgow..Globus installation Generally found the Globus software installation OK! Successfully deployed on a number of batch systems in UK Experience fed back into eScience Centres Difficulties were setting up and recognising each countries Certificate Authorities (CAs) Tricky legal implications to resolve!

31. 31/38NeSC 24 October 2001Gavin McCance, University of Glasgow Testbed work so far… UK Certificate Authority set-up… Many institutes already on testbed Grid Status and Network monitoring demonstrator available soon Networking status information provided by GridPP and DataGrid networking groups!

32. 32/38NeSC 24 October 2001Gavin McCance, University of Glasgow …testbed work so far Successful tests within ATLAS (and others) of some middleware products E.g. Large file transfers between UK, Italy, US and CERN Further tests planned with new release!

33. 33/38NeSC 24 October 2001Gavin McCance, University of Glasgow …experimental integration Work to do… Taking the kit and trying to integrate it into the experiments software frameworks ATLAS/LHCb software framework (GAUDI) Grid middleware GANGA framework Make Grid Services transparently available to ATLAS and LHCb programs

34. 34/38NeSC 24 October 2001Gavin McCance, University of Glasgow Grid validation Preliminary tests of basic middleware has been successful Now we have opportunity to see how it performs and scales with real datasets and real experimental users

35. 35/38NeSC 24 October 2001Gavin McCance, University of Glasgow Overall experience Middleware development is fun! Several good products have already been delivered Re-using industry standard components and protocols where they exist LDAP, SQL, HTTP(S), XML, SOAP PKI security Open Source…!

36. 36/38NeSC 24 October 2001Gavin McCance, University of Glasgow …overall Middleware being built using a variety of languages… JAVA, C++, C, Python APIs should be available for all JAVA, C++, C and command line… web access(?)

37. 37/38NeSC 24 October 2001Gavin McCance, University of Glasgow …overall Coordination very important Forums for discussion: Vital to ensure middleware is useful to a wide range of applications Prevent divergent technology

38. 38/38NeSC 24 October 2001Gavin McCance, University of Glasgow …finally Preliminary grid software architectures have been defined Basic middleware has been delivered Large scale validation underway NOW!! A excellent base to build on! Plenty still to do!