1. 4 th DataGRID Project Conference, Paris, 5 March 2002 Testbed Software Test Plan I. Mandjavidze on behalf of L. Bobelin – CS SI; F.Etienne, E. Fede – CPPM; A. Formica, I. Mandjavidze – CEA/Saclay

2. 4 th DataGRID Project Conference, Paris, 5 March 2002 Testbed Software Test Plan Irakli.Mandjavidze@cea.fr 2 Outline Motivation and Goals Proposed Test Strategy –Organization –Scope –Tests of Functionality Pre-requisites –Documentation –Deployment Discussion

3. 4 th DataGRID Project Conference, Paris, 5 March 2002 Testbed Software Test Plan Irakli.Mandjavidze@cea.fr 3 Motivation and Goals DataGRID middleware is rapidly evolving –Complex software –Rich in functionality Stringent schedule of software release plan –Minor releases every two months –Next major release in October, 2002 Need for continuous test activity –Check for bug-fixes –Validate added new capabilities –Identify eventual « unexpected features » A well-defined systematic approach is necessary

4. 4 th DataGRID Project Conference, Paris, 5 March 2002 Testbed Software Test Plan Irakli.Mandjavidze@cea.fr 4 Proposed Test Strategy: Organization Form an independent Test Group within the WP6 –Volunteers from several test (non-production) sites –Close contact with the Integration Team and WPs For each release –Participate in last stage of integration as an observer New features and « particularities », initial experience –Deploy and validate at each test site –Examine multi-site operation integrating test sites –Document test results, distribute them within the collaboration Act after integration and before validation by applications –Give a « green light » to the Validation Team Thorough examination and feedback to middleware WPs

5. 4 th DataGRID Project Conference, Paris, 5 March 2002 Testbed Software Test Plan Irakli.Mandjavidze@cea.fr 5 Test Strategy: Scope Installation, configuration –Appreciate required effort, evaluate code portability Functionality and performance –Check conformance with the documentation –Derive performance figures, identify eventual bottlenecks Security –Trace for security holes in deployed external packages –Check for non-authenticated and unauthorized accesses, … Scalability, robustness, fault tolerance, error detection –Operate in distributed multi-user multi-site GRID environment –Operate at high loads, with malfunctioning services, … See draft document on « Testbed Software Test Plan »

6. 4 th DataGRID Project Conference, Paris, 5 March 2002 Testbed Software Test Plan Irakli.Mandjavidze@cea.fr 6 Test Strategy: Tests of Functionality Follow layered structure of the DataGRID architecture –Identify interdependency of services Do not test upper level service if low-level service is faulty –Determine validation tests for every service Work is well advanced, see the « Test Plan » document –Develop test suites Split responsibilities among the test sites –Depending on expressed interests –Some overlapping is not excluded, might even be necessary Expect basic tests for services delivered within releases –Correctness of installation and configuration –Basic functionality of services and connection with low-levels Automate test suites when possible

7. 4 th DataGRID Project Conference, Paris, 5 March 2002 Testbed Software Test Plan Irakli.Mandjavidze@cea.fr 7 Pre-requisites: Documentation Administration Guide –Download, installation, configuration, maintenance and tests –Detailed description of system-level commands Users Guide –User environment set-up and test –Detailed description of user-level commands Developers Guide –Detailed description of available APIs Per-release updates Good understanding of functionality and services provided

8. 4 th DataGRID Project Conference, Paris, 5 March 2002 Testbed Software Test Plan Irakli.Mandjavidze@cea.fr 8 Pre-requisites: Deployment Installation and configuration procedure –Currently extremely tedious and time-consuming –Difficult to keep pace with releases May quickly discourage Test Group members Suggestion for major releases –Automate the procedure as much as possible e.g. self-deployment based on site description Suggestion for minor releases –Self-extracting patches i.e. upgrade only concerned parts of middleware Concentrate on functional tests rather than on deployment

9. 4 th DataGRID Project Conference, Paris, 5 March 2002 Testbed Software Test Plan Irakli.Mandjavidze@cea.fr 9 Discussion Some open questions –Is the exposed strategy acceptable? –Does the Test Group interfere already during integration? e.g. from a convenient moment validating Testbed in situ –Would the Integration Team be interested in some test suites? –How long (or short) is an acceptable testing period? Before providing feedback to WPs, first get one from them –Finalized « Test Plan » document as a basis for discussions –Agreement with the Integration Team and WP representatives Form the Test Group, share responsibilities and … Volunteers?