1. T-StoRM: a StoRM testing framework
Elisabetta Roncheri, INFN CNAF
as member of the StoRM team
Munich, 28 March 2012

2. E. Ronchieri, EGI UF, 26-30 March 2012, Munich, Germany
Overview
Problem Statement
T-StoRM solution
Tests details
T-StoRM usage
Future work
Conclusions
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

3. E. Ronchieri, EGI UF, 26-30 March 2012, Munich, Germany
PROBLEM STATEMENT
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

4. E. Ronchieri, EGI UF, 26-30 March 2012, Munich, Germany
What we considered
StoRM is an implementation of the SRM interface
It is characterized by
Being usable on any posix file systems, such as GPFS and Lustre;
Supporting several transfer protocols, such as gsiftp, file, https and http;
Supporting access to tape by using GEMSS;
Publishing information by using the GLUE standard;
Supporting VOMS and GSI for authentication and authorization.
Its deployment can be both easy and extremely complex according to site requirements.
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

5. Stand Alone StoRM deployment
FrontEnd (FE)
Exposes the Web service interface
Manages user authentication
Sends the request to the BackEnd
DataBase (DB)
Stores SRM request and status
Stores file and space information
BackEnd (BE)
Binds with the underlying file systems
Enforces authorization policy on files
Manages SRM file and space metadata
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

6. Common StoRM deployment
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

7. Complex StoRM deployment
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

8. E. Ronchieri, EGI UF, 26-30 March 2012, Munich, Germany
What we looked for
StoRM is a multi-service software subject to intense testing, validation and verification activities.
It needed something to
Optimize the deployment of a new StoRM software version to be certified
Simplify the reproduction of environments where error events raised to react as quick as possible
Provide users that experience criticalities with suitable support in reasonable time
Delegate validation and verification of remote sites to their administrators
Reduce time and effort to support users
Improve its software development life cycle
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

9. E. Ronchieri, EGI UF, 26-30 March 2012, Munich, Germany
T-storm solution
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

10. What we thought: T-StoRM
T-StoRM is a StoRM testing framework that orchestrates tests by using a proper deployment and a test engine fed with a pre-built configuration file.
It is designed to simplify the integration of new test suites.
It verifies the StoRM implementation of the SRM interface by using all the SRM clients.
It tests SRM calls one by one, providing atomic test that can be arbitrarily arranged to build more complex tests.
It produces a report log for all the executed tests.
It will be included in EMI 2.
This development is ongoing in collaboration with IGI.
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

11. E. Ronchieri, EGI UF, 26-30 March 2012, Munich, Germany
T-StoRM deployment
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

12. E. Ronchieri, EGI UF, 26-30 March 2012, Munich, Germany
Tests details
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

13. E. Ronchieri, EGI UF, 26-30 March 2012, Munich, Germany
Testing Levels
Levels
Descriptions
Area
Atomic testing
verifies any SRM operation
Any SRM
Functional testing
verifies any SRM functional specification
Sanity testing
verifies the correctness of the StoRM installation and configuration
StoRM Specific
Regression testing
verifies the correctness of each bugfix in StoRM
Integration testing
verifies the behavirour between different SRM implementations
Performance testing
determines how StoRM performs in terms of responsiveness and stability under a particular workload
Conformance testing
determines whether the SRM implementation agrees with the SRM specification and the SRM memorandum of understanding
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

14. E. Ronchieri, EGI UF, 26-30 March 2012, Munich, Germany
Atomic Testing
This level verifies any SRM operation
This level contains a set of tests that are also used in functional, regression, and integration levels.
SRM Operations
SRM Clients
ping
ClientSRM
srmping
arcsrmping
ptp
ptg rm
srmrm
lcg-del
rmdir
srmrmdir
mkdir
srmmkdir
...
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

15. E. Ronchieri, EGI UF, 26-30 March 2012, Munich, Germany
Functional Testing
This level verifies any SRM functional specification.
Each test reproduces a certain use case in order to verify the SRM functional specification.
Each use case can be implemented by using different SRM clients, such as dCache client, lcg-utils, and StoRM client.
Some examples are:
Move a file from the local node to the storage element by using the gsiftp protocol
Move a file between the local node and the storage element by using the gsiftp protocol
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

16. E. Ronchieri, EGI UF, 26-30 March 2012, Munich, Germany
Sanity Testing
This level verifies the correctness of the StoRM installation and configuration.
It contains tests that are specific to StoRM.
Each test of this level needs to be executed before those of the other levels.
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

17. E. Ronchieri, EGI UF, 26-30 March 2012, Munich, Germany
Regression Testing
This level verifies the correctness of each StoRM bugfix.
It contains tests that are specific to StoRM.
Each test verifies bugfixes for the current X.Y.Z-W and previous versions.
If the StoRM version under verification is lower than X.Y.Z-W, the test should show the issue.
Some tests can be only executed in a given version range.
The test is executed on top of a test environment similar to the one that showed the bug.
These tests cover atomic, functional, and sanity levels.
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

18. E. Ronchieri, EGI UF, 26-30 March 2012, Munich, Germany
Integration Testing
A new testing level has added to satisfy a request coming from EMI JRA1
This level verifies the transfer functionality between different SRM implementations.
The designed and implemented tests verify the files transfer between the EMI storage elements by using the gsiftp protocols.
Even if it is out of the T-StoRM original scope, it well fits in its architecture.
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

19. Integration Testing: Use Case 1
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

20. Integration Testing: Use Case 2
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

21. E. Ronchieri, EGI UF, 26-30 March 2012, Munich, Germany
Performance Testing
This level determines how StoRM performs in terms of responsiveness and stability under a particular workload.
The tests are not specific to StoRM.
Load and stress tests are currently supported.
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

22. E. Ronchieri, EGI UF, 26-30 March 2012, Munich, Germany
Load and Stress Tests
Load tests are executed to understand the behaviour of the system under a specific load. Metrics to be gathered are for example:
Response time of the service by using an SRM client on a user interface
Service resource usage such as CPU and memory.
Stress tests are executed to understand the upper limits within the StoRM system, and its robusteness in terms of load.
They are also useful in production deployment to determine if the system satisfies expected bust load.
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

23. E. Ronchieri, EGI UF, 26-30 March 2012, Munich, Germany
Conformance Testing
This level determines if the SRM implementation agrees with
the SRM specification (
the SRM Memorandum of Understanding.
The more significant tests will be developed.
One of the examples is:
Verify the srmPING behaviour:
having authorizationID as Input
returning as Ouput:
versionInfo of the SRM specification that is implemented
extraInfo of the SRM implementation such as the StoRM version
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

24. E. Ronchieri, EGI UF, 26-30 March 2012, Munich, Germany
T-storm usage
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

25. E. Ronchieri, EGI UF, 26-30 March 2012, Munich, Germany
The Current Usage
Who:
the StoRM team
the IGI-RTC (Release, Testing, Certification) group
Why:
To certify the StoRM version
To verify the correctness of a StoRM installation and configuration in the StoRM testbed
To verify new functionalities and fixed bugs
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

26. The Test Production Procedure
The production of a new T-StoRM test follows the 5 steps described below:
the test is identified with the pair (id, name):
id is a unique identifier of 6 chairs;
it is defined and described in the Test Plan document where test id is included
it is developed in the correct test level
it is included in the test suite
finally, it is released in T-StoRM
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

27. E. Ronchieri, EGI UF, 26-30 March 2012, Munich, Germany
The Near Future Usage
Who:
Site-administrators
SRM certifiers
Other software certifiers
Why
To validate a StoRM installation and configuration
To certify SRM software products
To certify other software products containing tests suitable for other software
To monitor SRM instances
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

28. E. Ronchieri, EGI UF, 26-30 March 2012, Munich, Germany
Future work
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

29. E. Ronchieri, EGI UF, 26-30 March 2012, Munich, Germany
Future Work
Web-based Reporting
To simplify the relation among the test plan, the tests and the report logs
T-StoRM integration with a Continuos Integration Framework
To automate tests execution
To immediate tests results reporting
Taking advantage from virtual technologies
To automate deployment and configuration of StoRM
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

30. E. Ronchieri, EGI UF, 26-30 March 2012, Munich, Germany
conclusions
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany

31. E. Ronchieri, EGI UF, 26-30 March 2012, Munich, Germany
Conclusions
T-StoRM is making faster and simpler the StoRM certification process.
It has shown to be able to cover several test levels even not considered at the design time.
It is designed to be easily extendible in order to be used for the certification of other software.
02/08/2018
E. Ronchieri, EGI UF, March 2012, Munich, Germany