1. Item 2.2 of the agenda IT Working Group meeting 2016
ESDEN
Item 2.2 of the agenda
IT Working Group meeting 2016

2. IT Working Group meeting 26-05-2016
Agenda
What is ESDEN?
Benefits of ESDEN for the ESS
Current project status
Approach and planning
Results achieved
Next steps
IT Working Group meeting

3. IT Working Group meeting 26-05-2016
What is ESDEN?
IT Working Group meeting

4. IT Working Group meeting 26-05-2016
ESS Vision 2020
Where do we want to be?
How do we get there?
Five key areas for delivering our vision:
Identifying user needs and cooperation with stakeholders
Quality of European statistics
New data sources
Efficient and robust statistical processes
Dissemination and communication on European statistics
ESS = European Statistical System
The ESS Vision 2020 is a common strategic response of the European Statistical System (Eurostat, EU Member States and EFTA countries) to the challenges that official statistics are facing.
It was adopted by the ESS Committee in May 2014.
More information on the ESS Vision 2020:
The ESDEN project contributes to two of the five key areas defined in the ESS Vision 2020.
IT Working Group meeting

5. IT Working Group meeting 26-05-2016
ESS Vision 2020
Key area 3: new data sources
We will exploit the potential of new data sources
Find suitable technology to use new data sources
A digital transformation is taking place across the globe: the data revolution is already presenting official statistics with profound strategic questions.
The ever increasing availability of data is a trend that is of strategic relevance for official statistics. We want to harness new data sources to produce meaningful statistics.
The possibility of using big data in the production of official statistics will be fully explored.
IT Working Group meeting

6. IT Working Group meeting 26-05-2016
ESS Vision 2020
Key area 4: efficient and robust processes
Collaboration based on standards and common technological and statistical elements
Sharing IT services and infrastructure
We will use common methods and tools.
We will explore other areas and use all the opportunities, in particular when dealing with new technology-driven areas like big data, open data and visualization techniques where we expect concrete opportunities to emerge.
We will benefit from exchange of (micro)data, while fully respecting statistical confidentiality.
CONCLUSION: all the elements of the ESS Vision 2020 explained so far call for a modernisation of the data exchange infrastructure.
IT Working Group meeting

7. IT Working Group meeting 26-05-2016
What is ESDEN?
ESDEN is a project of
the ESS Vision 2020
Objective:
"Modernisation of the data exchange services across the ESS, increasing the data exchange capacity and upgrading the security to allow a more efficient and secure data exchange in the ESS"
IT Working Group meeting

8. Benefits of ESDEN for the ESS
IT Working Group meeting

9. IT Working Group meeting 26-05-2016
Objectives of ESDEN
Enable the usage of new data sources
Increase security, sensitive data
Upgrade capacity, high volumes
Creation of efficient and robust statistical processes
Modernise interfaces, standard transmission protocols
Foster automation
IT Working Group meeting

10. IT Working Group meeting 26-05-2016
Benefits for the ESS
New user interface for EDAMIS, easier to use
Increased security
Better support for encryption
Leverage of private, secure european networks
Better traceability
Capacity to transfer higher volumes faster
Use of standard protocols
Easier to plug to national systems
Allow automation
IT Working Group meeting

11. Capacity to transfer higher volumes
The change we need to adapt to is not a simple progression, it is a revolution.
IT Working Group meeting

12. Increase security Secure the content Secure the transport
Secure the storage
CONCLUSION: provide different options, with different characteristics, adapted to the needs of each transmission.
Content
Transport
Storage

13. IT Working Group meeting 26-05-2016
Secure the content
Better support encryption
Provide encryption for information systems that do not have it
Non-encrypted file
Encrypted file
IT Working Group meeting

14. Secure the transport Alternatives for physical networks NSI Internet
EC-provided networks

15. IT Working Group meeting 26-05-2016
Project status
IT Working Group meeting

16. IT Working Group meeting 26-05-2016
Approach and planning
Phase 1 ( )
Secure network infrastructure for the pilot on microdata exchange for project SIMSTAT
Phase 2 ( )
Connect rest of ESS members to the secure network infrastructure
Extend the data exchange services to other statistical domains
Phase 3 (2018)
Integrate with new generations of secure networks
IT Working Group meeting

17. Governance of the project
Phase 1
SIMSTAT steering committee
SIMSTAT IT task force
For the next phases
Steering Group created
Member States: AT, IT, HR, PL, SI, FI, DK, HU
IT Working Group meeting

18. IT Working Group meeting 26-05-2016
Phase 1 - Activities
Selection a secure network for the pilot (CCN)
Connection of the countries and Eurostat to the selected network
Connection of 20 countries in less than 3 months
Development of the software components to allow the data exchange
Deployment of the software components in the Member States participating in the SIMSTAT pilot
IT Working Group meeting

19. Phase 1 – technical details
More than 40 Gigabytes exchanged over the network during the SIMSTAT pilot exchange
All files correctly delivered without major incidents
Very positive feedback on the network infrastructure received from all participant countries
IT Working Group meeting

20. Phase 1 – technical details

21. Phase 1 – Lessons learned
Identifying correspondents in all organisations is key, communication is vital, reactivity is essential
National provider of intra-EU statistics
National authority hosting the CCN gateway
National network provider
Eurostat
DG TAXUD
The network is resilient, volumes were easily absorbed
Synchronised Service Desks were essential

22. Phase 1 – Survey VIN Benefits Barriers and risks
Capability to securely exchange confidential micro-data
Opportunity to extend this benefit to other statistical domains beyond SIMSTAT
Barriers and risks
Complexity of involving many stakeholders
Dependency on third-party at the national level
Availability of resources for the deployment and the integration with the existing national solutions
Need to adhere to national security protocols

23. After phase 1 - opportunities
Extension to other statistical domains
European networks converging
CCN migrating to TESTA
Proof-of-concept to evaluate TESTA services
Opportunities beyond data exchange
Enable transport for remote access for researchers
Enable secure services for the ESS
IT Working Group meeting

24. IT Working Group meeting 26-05-2016
Phase 2 – Deliverables
Deliverable
Actor
Deployment actions
Connection to the secure network
Member States
Similar to first phase for countries still not connected
New version of the data exchange infrastructure
Eurostat
Incorporate the results of phase 1 to the Single Entry Point infrastructure, used for the exchange in other domains
New version of the communication component
Eurostat / Member States
Eurostat to create a new version of the software, Member States to install it
IT Working Group meeting

25. Phase 2 – Timeline Action When
Enhancement of the ESDEN software developed by Eurostat to further automate the data exchange 
2016 Q1
Launch of connectivity grants to help the remaining countries to connect to the secure network
2016 Q3
Connection of the remaining countries to the secure network
2016 Q4
Launch of ESSNet for the collaboration of the countries
Modernisation of the Single Entry Point based on the results of the first phase of the project
2017 Q4

26. Involvement of ESS Members
Grants
To help ESS Members in the effort to connect to secure European networks
ESSNet
To allow ESS Members to contribute to the definition of the requirements of ESDEN
To collaborate in the testing of the software delivered by the project
Grants
ESSNet
- 2 work packages foreseen, 3 workshops to be organised, report to be produced.
- Eurostat working currently on the technical problems derived from the end of support of Java on the browsers (WebForms and secure upload).
- The project would like to incorporate other requirements from ESS Members.

27. The future – phase 3 At the end of phase 2:
Assessment of the services of the new generation of the secure network
Consultation with the ESSC on the implementation of phase 3
If approved
Upgrade the network connection to the new generation of EC secure network
Kick-off production operations: go-live

28. IT Working Group meeting 26-05-2016

29. IT Working Group meeting 26-05-2016
Additional slides
IT Working Group meeting

30. IT Working Group meeting 26-05-2016
Phase 1 - Deliverables
Deliverable
Status
Main actor
Physical connection to the CCN network
Delivered
ESTAT/SIMSTAT IT Task force
Creation of the backbone of the communication infrastructure for the SIMSTAT project
ESTAT
Installation and configuration of the communication component
SIMSTAT IT Task force
IT support during the pilot micro data exchange
Ongoing
IT Working Group meeting

31. Phase 1 – Deployment actions MS
Provision of hardware infrastructure to host the communication platform
Creation of communication link between NSIs and the National authorities
Deployment of the software provided by Eurostat
IT support for issues on the local infrastructure
IT Working Group meeting

32. IT Working Group meeting 26-05-2016
Phase 2 – Objectives
Connect rest of ESS members to the secure network infrastructure
Modernisation of the existing data exchange infrastructure used for other statistical domains based on the results of the first phase of the project. Main target areas:
Standard communication protocols
Modernisation of user interfaces, replacement of technologies in phase out
Enhancement of security mechanisms
IT Working Group meeting

33. Phase 2 – Objectives
Modernisation of the existing data exchange infrastructure (EDAMIS) used for other statistical domains
Standard communication protocols
Modernisation of user interfaces, replacement of technologies in phase out
Enhancement of security mechanisms
The data revolution is already presenting official statistics with profound strategic questions
Global digital transformation
Ever increasing availability of data
We want to harness new data sources to produce meaningful statistics