1. Validation services developed in the ESS
ESTP training course
Item 7(2)
Luxembourg, Nov 2017
Eurostat, Unit B1

2. Validation in the ESS – IT building blocks
VRM
STRUVAL
CONVAL

3. Structural Validation Service (STRUVAL)
First step of validation
Performs structural validation of statistical data files based on structural information following the SDMX information model
Checks that syntax and coding follow the definition given in an SDMX DSD, the completeness of the records and highlights basic consistency errors (e.g. if the same key is repeated twice)
Already used in Eurostat and available to Member States

4. Structural Validation Service (STRUVAL)
In order to use STRUVAL you need:
An SDMX DSD or an SDMX dataflow
An SDMX-ML file
Potential future improvements: ability to read a wider variety of data formats

5. Content Validation Service (CONVAL)
Performed after structural validation
Includes the complete range of validation operations on data prepared for statistical production
basic logical and content checks
intra- and inter-file data plausibility
consistency checks
cross-domain source based checks
A first version of CONVAL used in Eurostat and in a limited fashion as a shared service for Member States

6. Content Validation Service (CONVAL)
In order to use CONVAL, you need
A description of the validation rules
Additional data files
(if a validation of level 2 or above is required)
CONVAL will receive as input validation rules written in the standard validation language VTL (feature to be implemented at the beginning of 2018). It is likely a subset of VTL will be covered first, to be expanded in the future.
VTL

7. EDIT vs CONVAL What is the relationship between EDIT and CONVAL?
CONVAL is a CSPA service: at its core, CONVAL is a specification of a set of functionalities and of a service interface. There can be several implementations of CONVAL.
EDIT is a first implementation of CONVAL. This however may or may not change in the future…

8. Validation Rule Manager
Allowing searching, browsing and managing validation rules
Second prototype made in 2016/2017
To be developed over the course of 2018

9. Validation Rule Manager – 2nd prototype

10. Validation Rule Manager – 2nd prototype

11. Validation services and CSPA
Both CONVAL and STRUVAL have been built according to the CSPA guidelines
The services are described according to the standard CSPA templates, which will be made available in the new CSPA service catalogue
First (but not only) Eurostat CSPA services

12. Availability timetable
Validation services can be made available to Member States under different scenarios…

13. Scenario 2
Member States use common ESS services in their process for validation (as shared or replicated services)
Shared service
Replicated services

14. Scenario 3
Member States use common ESS processes for validation

15. Availability timetable
Scenario 2
Scenario 3
STRUVAL
Available for both replicated and shared
Available
CONVAL
Available as shared in a limited fashion
Available as replicated starting in Q3/2018
VRM
N/A
Available end-2018 / Beginning 2019

16. Thank you for your attention!
Any questions?