1. Gavin McCance University of Glasgow GridPP2 Workshop, UCL
WP2: Data Management
Gavin McCance
University of Glasgow
GridPP2 Workshop, UCL

2. EDG Summary Report WP2: Data Management

3. The team Based at Glasgow University
William Bell
David Cameron
Tony Doyle
Gavin McCance
Paul Millar
Caitriana Nicholson
We formed close collaboration with WP2 of European DataGrid
Became an integral part of the WP2 team
All our products were delivered jointly with EDG

4. What we set out to do? Initial work-plan was in two key areas:
Grid Optimisation
Simulating the Grid from the data-centric viewpoint
Applying the results from the simulations to develop algorithms for optimisation of replicas
Development of optimisation software for replica management
Metadata management
Development of secure metadata services for the experiments

5. What we actually did… Grid Optimisation
Developed a data-centric Grid simulation [OptorSim] (fully documented final release available now)
Compared of a variety of replica optimisation techniques and algorithms (many publications)
Worked on developing and refining these algorithms
Exploitation of some of the early work, we developed the Replica Optimisation Service (ROS) component of the Replica Management System
provides file access cost estimation to the user and the WP1 resource broker.

6. What we actually did… Metadata Management
Early production of the Spitfire browser service allowing remote access to RDBMS via a web browser
Early adoption of GSI security into Spitfire to allow secure use of the service using Grid certificates
Initial adoption of web services into EDG with the creation of the web services API version of Spitfire, allowing programmatic access to remote RDMBS
The web services technology built for Spitfire was subsequently factored out and used in all other WP2 services
The security built into Spitfire was subsequently factored out and led to the creation of the WP2 Java security modules

7. What we actually did… Replica Management Services
As focus of EDG changed to “stability, stability, stability” and production, we changed with it
Helped factor-out the web services and security frameworks from Spitfire
Moved into the core replica management services
Replica Manager
Replica Location Service
Replica Metadata Catalogue
Replica Optimisation Service
Provided continued support for these products to EDG and LCG
Led the thorough performance testing of our products for WP2’s final EU deliverable

8. What we actually did… WP2 Integration and Testing WP2 Testbed support
Significant integration expertise developed
Paul Millar is the WP2 integration team member
Glasgow ran the WP2 CVS repository
We created the testing framework for WP2 software
WP2 Testbed support
Provided support for the WP2 mini-testbed
Machines managed at ScotGrid and at CERN
WP2 Management
Gavin McCance is WP2 deputy
Major contributions to WP2 deliverables

9. What we actually did… Standards Publications Optimisation Metadata
Contribution of GGF Data Area documents
Publications
Work was fully reported: 15 or so papers published…
Optimisation
Metadata
Replication
GGF
Deployment 6 3 2 1

10. EDG Effort Delivered
Substantial UK effort delivered to EDG : ~150% of contracted

11. Exploitation
Exploitation of the technology to deliver the next solutions to the experiments and to enable their own solutions
Stateful web services
Security
Service oriented
Focus on manageability and robustness – easily deployable
On the metadata side, emphasis upon Grid enabling the experiments own metadata solutions

12. WP2: What we learned Software development process…
sometimes learned the hard way…!
Much feedback from our users on the software
Very useful for the design process in the next projects
The amount of effort required to make production quality services out of our software
Take all this forward into GridPP2!