1. Leanne Guy EGEE JRA1 Test Team Manager
EGEE JRA1 Testing
Leanne Guy
EGEE JRA1 Test Team Manager

2. Overview Middleware testing activity Test team structure
Mandate, deliverables
Test team structure
Scope of the middleware testing activity
Testing process and framework

3. Testing Activity Mandate
Test all middleware components that form part of the integrated middleware system to ensure a production quality release that fulfills the requirements of the applications.
Assess that all software requirements have been correctly and completely implemented and are traceable to system requirements,
Test the integrated software for scalability, platform independence and stress resilience

4. Testbed Testbed is distributed across 3 sites Requirements on sites
Minimum to test all basic grid functionality
Ensure that assumptions valid at only one site have not been made
All sites run a binary compatible version of Red Hat Enterprise Linux
CERN: SLC3
NIKHEF: CentOS 3.2
RAL: Scientific Linux
More sites can join the testing activity at a later stage !
Requirements on sites
1 sysadmin dedicated at least 50% to the activity
Deploy candidate releases (~weekly)
Diagnose and report installation problems – provide fixes
At least 15 machines dedicated to the testing and validation activity
Support for the primary platform – RHES 3.0
If support for secondary platform not possible we will need to look to other sites to participate

5. Test team structure (II)
CERN:
Main testing site, responsible for coordination and management of the distributed team
Responsible for producing test suites
Leanne Guy: Test team manager
Diana Bosio, David Collados, Mario Reale : Testers
NIKHEF
Davide Salomoni: sys admin, Jeff Templon, David Groep
RAL
Steve Traylen: sys admin

6. Testing scope (I) Unit testing Build validation Deployment testing
Done by developers in each activity
Coverage tools for unit testing integrated into the build system
Build validation
Tests all rpms against requirements in the developers guide
License present, rpms relocatable, files in correct place
Being expanded now to test tar files as well
Automated test suite that runs nightly on the nightly and integration builds
Deployment testing
Tests the correct installation and configuration of gLite components
Tool independent – Test the manual installation of gLite components
Tests done independently at the 3 sites
Based on the installation guides provided by integration team
All inconsistencies with the installation guide reported as bugs
Services should all start and a basic test should pass
i.e. Jobs submission, file transfer, query on the information system

7. Build validation

8. Testing scope (II) Documentation testing Functional testing
Installation and user guides will be produced by integration team
Test team verifies all documentation for release
Functional testing
Fully integrated system validated against the functional requirements as specified in user requirements documents.
Driven by application uses cases
Requirements come from:
The NA4 requirements database
Security requirements document
HEPCAL documents
Overlap with NA4 application testing team
Bugs in requirements are tracked in savannah

9. Testing scope (III) Regression testing Security testing
Essential to ensure that bugs fixed in one release do not reappear in a later release
Security testing
Secure access to data especially biomedical requirements!
JRA3 security team help with security test planning
Scalability, Stress resilience, Performance testing
Important to find the limits of the system

10. Testing process Deploy and test Integration builds gLite Testsuites
Automatic installation at all sites using quattor or kickstart
gLite component installation via deployment modules
Configuration using post installation configuration scripts
gLite Testsuites
Build validation run on all rpms nightly builds
Functional tests run on distributed testbed
Bug reporting - savannah

11. Testing framework Test suites written using the xUnit framework
PyUnit for python APIs and CLIs
JUnit test framework for Java
CPPUnit for C/C++
QMTest framework
Provided and supported by the tools group and used in LCG
Deployed now on the testing tested
Used to manage the execution of all testsuites.
NA4 test team developing a framework for job submission tests