1. EGEE is a project funded by the European Union under contract IST-2003-508833 Tools survey status, first experiences with the prototype Diana Bosio EGEE JRA1 testing team JRA1 All hands meeting @ RAL, 28-30 June 2004 www.eu-egee.org

2. JRA1 all hands meeting, June 28 2004 - 2 Contents Testing tool selection of candidates Shortlist of candidates for framework Testing manual deployment First experience with the prototype

3. JRA1 all hands meeting, June 28 2004 - 3 Tool survey Lots of tools, few requirements http://egee-jra1-testing.web.cern.ch/egee-jra1- testing/tools.htmlhttp://egee-jra1-testing.web.cern.ch/egee-jra1- testing/tools.html Concentrate on the testing framework Few “Strong” Requirement:  E.g. test must run with and without the framework  http://egee-jra1-testing.web.cern.ch/egee-jra1-testing/evaluation- requirements.html Survey and evaluation described in EDMS document https://edms.cern.ch/document/473309/1 https://edms.cern.ch/document/473309/1

4. JRA1 all hands meeting, June 28 2004 - 4 Tool survey issues Problems:  hard to test as prototype is not very stable and testing testbed is still in the installation phase  very hard to integrate in glite as glite acts like a shell Not clear to me how to do job storms or cron jobs Not clear what APIs are available, Perl only?  Very little feedback on the document, it is very hard for me to decide for everyone without knowing the different opinions present in the project 3 candidates for framework tool: Qmtest, LCG TSTG suite, xUnit framework

5. JRA1 all hands meeting, June 28 2004 - 5 Candidate 1: QMtest Advantages:  LCG-SPI has already experience with it, but they support only an old version.  QMtest claims to store result in XML format, in reality output file is some binary format, so not clear how to produce text XML In any case we need to write a tool to produce standard test reports  Support is prompt and helpful and solves problems very fast Disadvantages:  Has undergone some major rewriting and does not seem to be very stable at the moment

6. JRA1 all hands meeting, June 28 2004 - 6 Candidate 2: LCG-TSTG test suite Advantages  Quite complete, a lot of work has been done on it  Used in a Grid environment Disadvantages  At the moment it is tailored for LCG, not clear how we could adapt it for EGEE and the glite shell  Standard output is html, we need to do some work to produce XML output  No external support available Slide 1

7. JRA1 all hands meeting, June 28 2004 - 7 Candidate 3: x-Unit framework Advantage  uniform interface across the languages and for the whole project, integration team is doing work on standardization of the output for unit testing Disadvantages  We will have to write extension classes to parse the output and produce result pages in XML  Test will only run as part of the x-Unit framework, so violates the requirement that all test should be run independently from the framework

8. JRA1 all hands meeting, June 28 2004 - 8 Deployment testing The idea is to install and run the configuration scripts for all glite RPMs and tarballs to detect conflicts/incompatibilities, incorrect dependencies, etc… At the moment RPMs installation only  Basic bash script that downloads and installs rpm and produce a log file  At the moment I am waiting for clear installation instructions that work In the future will install all RPMs produced by the build system on a regular basis, all on one freshly installed machine

9. JRA1 all hands meeting, June 28 2004 - 9 First experience with the prototype RGMA  The rpm does install, but the service does not start correctly.  JRA1-UK is working on it LRC  Installs and configure fine. New prototype CE  The rpm does install, but the Condor service with modified files gives segmentation fault

10. JRA1 all hands meeting, June 28 2004 - 10 Conclusions Still finalising the decision on testing framework  Need to look at the three candidates in the glite environment and understand what benefits there are to using a framework Hard to decide which framework when in initial phase of development and the software is changing rapidly (and does not install correctly) Testing of manual installation and configuration from RPMs is starting