1. Testing Grid Software on the Grid Steven Newhouse Deputy Director

2. © Open Middleware Infrastructure Institute Provider of open source grid software Based at the University of Southampton Utilise existing software and standards Production focused software development Integrate, test & document ‘a product’ To deliver high quality middleware to the UK e-Science community that enables scientists and engineers to make use of globally distributed computing resources

3. © How will OMII deliver? By providing a software repository of grid components from e-science projects By re-engineering software, hardening it and providing support for components sourced from the community By a managed programme to contract the development of “missing” software components necessary in grid middleware By providing an integrated grid middleware release of the sourced software components

4. © So the problem… Integrate complex service based software Validate functionality on different platforms Demonstrate interoperability Cross platform Different Version In the future… different systems

5. © Activity Within OMII Bugs Regression Tests Testing Functional Specifications Sources Development Teams Priority List Evaluation Risk and Prioritisation Fix Bugs Requests for Enhancements Use Cases Build Test Cases Repository Helpdesk Open Source Distribution User Community Quality Review Issues

6. © Manual & Non-functional Testing Exploring the install environment Different platforms Different JDKs Different configurations Using the documentation Do we tell people to do what we need to do? Reading and maintaining documentation is costly Tutorials & Training

7. © Automatic & Functional Testing Requirements  Specification  Test suite We use a lot of JUnit testing Validate Integration Need to do whole system testing A common problem across all infrastructures

8. © NMI Build & Test Framework

9. © What is it? A Condor job running in a Condor pool A framework (perl scripts) that creates dependent jobs using DAGman Application specific ‘plugins’ Bash scripts Sensible default behaviour Hawkeye modules to provide platform details Pre-installed software modules

10. © Obtain Sources (scp, cvs, gridftp) pre_all post_all platform_pre platform_post remote_pre remote_post remote_task Tasks executed on the submitting machine Tasks executed on the remote execution machine Tasks repeated for each platform platform_pre platform_post remote_pre remote_post remote_task platform_pre platform_post remote_pre remote_post remote_task x86_fc4 x86_rh3 suse_90

11. © Build Actions (1/2) Manual CVS checkout & patch source Access restricted outside of OMII pre_all: Remove non-build directories remote_pre: Build & install Ant, PostgreSQL & Perl modules Configure OMII databases

12. © Build Actions (2/2) remote_task: Install database, software stack & managed programme remote_post Stop server Package up the install tree platform_post Upload binary Clean up build location post_all Create source bundle & upload

13. © Test Actions – for each build remote_pre Deploy & configure server remote_task Install client remote_post Use client to run against server platform_post Analyse test results post_all Clean up

14. © Issues No inter-machine testing Need to express: server on A & client on B Multiple clients for stress testing Need a client to communicate with a specific server Located on port X on machine A Linking into other build resources Submit at Southampton and run in Madison (& reverse) Enabled through firewalls using CondorC Test build environment Remove ‘overhead’ from using Condor More expressive error reporting Remote/portal access for the managed programme

15. © Roadmap Basic WS Hosting Environment Services WS Infrastructure Application Exemplars Client Environment WSSecITWSS4J Job Resource Allocation DataAccount GridSAM OGSA-DAI WS-I WS-Notification WS-Eventing WS-Reliable Messaging WS-Reliability Grimoires BPEL Java CauchyMANGO JythonMatlabPerl

16. © Collaboration is essential Build & Test workshop at OMII last February OMII NMI @ Madison (Condor Team) GT EGEE OGSA-DAI OMII-China European Activity ETICS OMII-EU proposed

17. © Claim your Travel Mug Download the OMII client from http://www.omii.ac.uk http://www.omii.ac.uk Install the software and open an account on our server Enter your name in the popup window Go to the OMII booth and claim your Travel Mug! Full instructions on the website.