1. Joint meeting of the ESS.VIP.BUS ICT Project
Methodological Study
Joint meeting of the ESS.VIP.BUS ICT Project
and the ESS.VIP IT TF
Luxembourg, 3 March 2014
Daniel Rase / Unit G4

2. Methodological Study (2013)
Task 1: Documentation and analysis of production processes
Task 2: Logical data model and data exchange format
Task 3: Metadata Exchange format
Task 4: Data validation issues
Task 5: Data quality issues
Task 6: Calculation of EU aggregates and indicators
Task 7: Statistical confidentiality
Task 8: Treatment of data releases and embargos
Task 9: Legal and contractual aspects
The methodological study is carried out by Unit G6
Nine different tasks were foreseen.

3. Census Hub versus ICT Census Hub ICT statistics Large amount of data
Relatively small amount of data
Stability: 10 years
Yearly survey + 40 % change in the variables collected
Data already validated
Data to be validated
No calculation – no aggregation
Calculation of ratios and EU aggregates
Countries not available accepted
Countries not available to be avoided
Data not confidential
Confidential data
This table shows the main differences between the Census Hub context and ICT statistics.

4. Task 1: Business Process Management Approach
Modelling of the current 'AS IS' state including the description of the current capabilities
Modelling of the 'TO BE' state including the description of the capabilities to kept and the new ones that were identified during the 'TO BE' analysis
The second strand was the business perspective. It was decided to step back and to model the business processes to really understand the workflow. It is important to mention that the modeling is complementing the ‘process documentation’ that is passive metadata (i.e. EPMS).

5. Task 1: BPM
AS IS
TO BE
On the basis of the AS IS a specific workshop was carried out to discuss with all the stakeholders involved in the process, the future architecture of the ICT production environment.

6. Task 1: To-be-state Users European Statistics - Dissemination
Interoperability standards & common metadata
Shared Services
Storage
Production
Dissemination …
European Statistics – Production Processes
NSI
ESTAT
European Statistical System
STAT
Other Statistical Actors
ESS Secure Bus
European Statistics - Dissemination
Primary Collection
Administrative Sources - Surveys – Official Sources – Private Sources
Users
Other Partners
OECD
IMF
As the ICT project became part of the ESS VIP programme, we certainly need to take the 'TO BE' state of the entire programme into consideration.

7. Task 1: Mapping with GSBPM
A series of workshops was organized with the help of the team from DG DIGIT. Their experience was really indispensable to carry out this pilot exercise that was financed by Dir. B. (an internal group was created , chaired by the EA, in order to reflect on a possible introduction of BPM – Business Process Management- in ESTAT; a MoU was signed between DIR B and DIGIT. In this MoU, days were dedicated to the modelling of the ICT processes).
From, the beginning the modeling exercise was carried out with the GSBPM in mind. It was demonstrated that the GSBPM is applicable to the modeling of processes from the ESTAT perspective even if it was mainly designed to be used by the MS.

8. Task 1 : BPM (1)
The project team decided, for the first task, to step back a little bit and to model the production processes of the ICT surveys.
DG DIGIT BPM Competence Centre was contacted and they were ready to co-operate with us.
At the same time, a group was created at ESTAT level, chaired by JM Museux. The mandate of the group is to reflect on the possible implementation of BPM at ESTAT. ICT was considered as a good pilot in this context.
A MoU was signed between DIR B and DIGIT. In this MoU, days were dedicated to the modelling of the ICT processes.
This first diagram made during a first workshop is a value added chain diagram that provides the overall description of the process.
Let's zoom in

9. Task 1 : BPM (2)
In this part of the diagram we can identify the sequential activities and those that can run in parallel.
The sub process marked with a sign where further detailed using 'Collaboration diagrams'
Let's drill down on the validation sub process

10. Task 1 : BPM (3)
The collaboration diagram shows the process in more detail. It is called 'collaboration diagram' because the swim lanes represent the stakeholders of the process (in this case: Member States and Eurostat)
Please note that the Member States are shown as black boxes exchanging messages with ESTAT.
Let's zoom in

11. Task 1 : BPM (4)
Here we see that there is one more level of description. The red arrow shows a link to a FAD diagram (Function Allocation Diagram)

12. Task 1 : BPM (5)
This FAD diagram is very important during the 'AS IS' analysis.
It identifies the inputs, outputs, actors and roles but the most important are the capabilities.
Let's have a look to the capabilities identified in the validation sub process

13. Task 1 : BPM (6)
The capabilities identified will need to be preserved in the 'TO BE' state, new capabilities could also be identified at this stage.
The capabilities make the link between the processes and the functionalities that will be requested from the software applications or tools. Certain capabilities could already be provided by existing application (or in more than one application), or will require the development of new application(s).
One can see that this approach of 'capabilities' is independent from any software implementation and can be efficiently used to rationalise the pool of applications available at ESTAT

14. Task 1: BPM (7) Decoupling capabilities from their implementations
The capabilities are decoupled from their implementation. The expression of the required capabilities makes the link between the business and the IT department. The IT specialists can implement them in the way they feel the most appropriate. The business side will sign a ‘service level agreement’ with the IT department that will deliver the solution. A decision could be taken by IT to replace an application or a service but this should remain transparent to the business side.

15. Phase I: BPM (8) AS IS TO BE TO BE TO BE Different levels of 'TO BE'No
Survey
Phase I: BPM (8)
Different levels of 'TO BE'
AS IS
TO BE
TO BE
TO BE
On the basis of the AS IS, two TO BE workshops were organized until now. The first one was attended by the ICT team and other ESTAT stakeholders concerned (EA, quality, methodology, etc.).
The outcome of this first workshop was interesting but the reflections remained at the level of the current process. It was more an improvement workshop than a real TO BE exercise.
It was therefore decided to organize a second workshop in Brussels with the main user (DG CNECT). It was organized by B1 (W. Kloek) and facilitated by a colleague from the CBS. This workshop and the way it was facilitated drove the participants to a more creative and innovative discussion really projecting them into a long term perspective.
The output from these two workshops will serve as a basis for the organisation of one or two more TO BE workshop(s) with to come out with the desired pool of capabilities.
There are still discussion about the separation of the TO BE state in different stages. The ‘ultimate’ TO BE could be ‘No survey’ and exclusive use of Big Data
Short term
Medium term
Long term

16. Task 2 and 3 : Logical data model and data exchange format - Metadata Exchange format
Close co-operation with ESS VIP.CRC IMS
By the end of 2013 early guidelines on the repartition of standards for aggregate and micro data were provided by the IMS project
Early guidelines available upon request

17. Task 4 : Validation
Contacts were taken with the project manager of ESS VIP.BUS on Validation
A leaflet is available on the Eurostat Website

18. Task 4 : Validation Architecture
Coherent validation policy in the different statistical domains, in cooperation with Member States
Including a coherent distribution of validation responsibilities in the production chain
Standard Data Validation Communication:
Standard syntax, common validation reports, structured error messages, standard communication of validation rules, …
Solutions for complex validation actions to ensure:
Coherence between data files
Coherence between Member States
Integrity of ESS data

19. Task 4 : Validation Architecture
Package 1: IMPLEMENTATION
Implementation of the methodological developments in the statistical domains / Working Groups
Maintenance and refinement of standards developed
User requirements for further developments
Evaluation, monitoring and reporting
Package 2: MICRO-DATA
Extension of the functional specifications to apply to micro data validation
Integrated solutions for micro data validation
Vertical integration of the micro data validation within the ESS production processes

20. Task 4 : Validation Architecture
Package 3: SOLUTIONS
Adaptation of existing validation tools to the functional specifications issued in the previous related activities
Deployment of validation solutions to Member States:
web services architecture
Supporting validation at national level before transmission to Eurostat
Distribution of validation rules in agreed languages
Package 4: COORDINATION
Coherence of validation approaches within ESS
Implementation of the Validation syntax in the ESS
Links with other projects & related activities
Good practices identification
ESSnet definition and specifications

21. Task 4 : Data Validation

22. Task 5 : Quality
ICT was selected to by the pilot for the new quality assessment process in close cooperation with unit B1
A comparison was made between the current quality reports and the ESS standards for quality reports. The objective being to make them compliant with ESQRS.

23. Task 6 : Calculations of ratios (EU)
EU aggregate value is: Num / Den
Num: ValueBE + ValueBG + … + ValueUK
Den: ValueBE + ValueBG + … + ValueUK
If one value is missing in Num, corresponding value must be omitted in Den, and vice versa
We only publish ratios, percentages and EU aggregates.
We saw earlier that, in the Hub configuration, one or more Member States could be unavailable at a certain point in time.

24. Task 6 : Calculations of ratios (EU)
Countries are counted
Their population is added
Result is kept if both:
Count of countries > 55%
Sum of population > 60%
What we do today for the calculation of EU aggregates is the following.
Directorate B is investigating to what extent the question of unavailable countries could be dealt with with technical solution like redundant databases or cache mechanisms. The report will also be available before the end of 2013.
Of course, for each potential solution, the element of cost will need to be carefully taken into consideration.

25. Task 7 : Confidentiality
In the current environment, confidentality is ensured by eDAMIS and then the data is stored in the confidential environment
The issue of confidentiality will be tackled in the context of the ESS VIP.CRC ESDEN project, the priority being SIMSTAT
As described earlier, dependencies were identified in the project data collection template with ESS VIP.BUS ESDEN

26. Task 8: Data releases and embargos Task 9: Legal aspects
These two tasks were postponed

27. Thank you for your attention!
Contact:
Unit G4, Eurostat