1. The Vision Infrastructure Project “Enhanced Dissemination Chain”
Point 17 of the agenda
The Vision Infrastructure Project
“Enhanced Dissemination Chain”
Eurostat Unit B6
Eurostat Unit D4
Dissemination Working Group Meeting November 2011

2. VIP “Enhanced Dissemination Chain”
Vision Infrastructure Projects
aim to improve the vertical and horizontal integration of the European Statistical System; this ESS integration is one of the central goals of the ESS vision.
Commission Communication COM(2009)404 on the production method of EU statistics: a vision for the next decade
As new technologies become available, there is a clear drive to maximise their use and to gear statistical methods toward them. New forms of communicating with users and producers such as web 2.0, as well as other new developments in information technology, are likely to result in profound changes in communication channels and
data handling and storage.
VIPs deal with improvements which are the precondition for achieving more vertical and horizontal integration of the European Statistical System :
- horizontal integration : depart from the approach where statistics are handled domain by domain
- vertical integration : joint structures / tools / processes + collaborative networks
The improvements enable more integrated dissemination of European data and indicators.
VIPS fall under a 2009 Commission Communication on the production method of EU statistics
The Communication offers a vision for reforming the production method of European statistics., departing from the classical stovepipe model
Particular attention goes to the usage of the internet and web based technology in a broad sense, as these are nowodays the main vehicle for statistical dissemination.
Technology allows achieving dissemination targets such as full access to data, to all users at the same time & under same conditions, free of charge
Dissemination Working Group Meeting November 2011

3. VIP “Enhanced Dissemination Chain”
Enhanced Dissemination Chain - From 2011 to 2013
Short term
Eurostat’s New Dissemination Chain : improvement, simplification & better integration
Disseminated data : quality improvement
EU policy indicators : introduction of a new production line
Involvement of ESS (Member States) in the New Dissemination Chain
Medium term
Further development of Eurobase towards a more integrated data warehouse for Eurostat and the ESS
Better integration of Eurostat & national dissemination
Short term – definition of short term
1. simplification and better integration, diminish the risk of disruption of
a) the daily data updates and
b) the online data access,
PLUS
increase the performance of dissemination
2. data quality of the European data which is disseminated, in particular through the use of checking algorithms on all published data (Eurobase functionalities)
3. producing and disseminating policy-oriented indicators includes the development of
(a) a new production line for indicator tables, to facilitate flexible combination of data from several domains and
(b) new ways of presenting indicators and dashboards on the Eurostat website,
would be used for the EU2020 indicators, the Sustainable Development Indicators, macroecomic imbalance scorecard...
4. Involvement in the dissemination chain software components & tools to be shared – will detail in a few minutes minute
Medium Term – definition of medium term
1. Data warehouse
Will detail more about this in the next slide
2. better integration within the dissemination of Member States.
This covers further work to facilitate NSIs putting European data and metadata on their websites
Dissemination Working Group Meeting November 2011

4. VIP “Enhanced Dissemination Chain”
Actions
Deliverables
Development towards a Eurostat / ESS
data warehouse 1
Strategy (to be worked out with NSIs)
Simplification and integration of the Eurostat Dissemination Chain
More functionality for producing and disseminating Eurostat policy indicators 2 3
COLOR CODING
ORANGE = NSIs involved / impacted / can use deliverables
BLUE = EUROSTAT action or deliverable
What will happen in the 2011 to 2013 period ? 4 actions
2 with deliverables relevant to the NSI’s
MEDIUM TERM
Warehouse strategy
Context :
Classical approach so far : statistics for specific domains are produced independently from each other (stovepipe);
For the future : establish a model (the so-called data warehouse approach) for an integrated approach accross domains -> topic to be further explored
SHORT TERM OBJECTIVES
Visualisation & SDMX-compliant web services : Involvement of ESS (Member States) in the New Dissemination Chain
2 with deliverables focussing more on Eurostat’s internal dissemination chain
- Improvement of the quality of the data disseminated
More integration of Eurostat data and metadata in NSI dissemination 4
Embeddable visualisation tool
SDMX-compliant web services
Dissemination Working Group Meeting November 2011

5. VIP “Enhanced Dissemination Chain”1
Eurostat / ESS Data Warehouse
2nd half of 2012
ESS Business Process
Incl. governance
& IT
Communication
on impact
Strategy,
development &
implementation
Strategy paper
4 steps,
blue underline = Eurostat milestone
Organe underline = milstone involving NSIs
Dissemination Working Group Meeting November 2011

6. VIP “Enhanced Dissemination Chain”
Embeddable visualisation tool for NSIs
Simple visualisation tool, targetting a broad audience
Widget(s) based
To be embedded into the website(s) of NSI’s
In the language(s) of the NSI website(s)
Consuming Eurostat data
Development by Eurostat, in close collaboration with NSIs
Dissemination Working Group Meeting November 2011

7. VIP “Enhanced Dissemination Chain”
Embeddable visualisation tool for NSIs
...building on experience in the open data community
Dissemination Working Group Meeting November 2011

8. VIP “Enhanced Dissemination Chain”
Embeddable visualisation tool for NSIs
...building on experience in the open data community
Dissemination Working Group Meeting November 2011

9. VIP “Enhanced Dissemination Chain”4b
Access to Eurostat data & metadata
via SDMX-compliant web services
Automated access, SDMX (2.0) compliant :
data request using SDMX Queries
download facility for the results and pre-generated data set(s) in SDMX-ML format
Availability : 2012 Q1
Possibility to consume services exposed by eServices application
The dataset web service exposes three web methods. The services exposed by Dataset web service are
Get available datasets
Get Data Structure Definition
Execute SDMX Query
The main purpose of eService application is to give professional users of Eurostat’s web site a download access to Eurostat’s data. The application provides a SDMX compliant web service to query this data and an RSS feeds service to receive notifications on data changes and updates. It also provides file download interface (centralised point of access) for these users to have an automated download of ESTAT data
Dissemination Working Group Meeting November 2011

10. VIP “Enhanced Dissemination Chain”
Actions
Deliverables 1
Development towards a Eurostat / ESS
data warehouse
Strategy (to be worked out with NSIs)
Simplification and integration of the Eurostat Dissemination Chain
More functionality for producing and disseminating Eurostat policy indicators 2 3
What will happen in the 2011 to 2013 period ? 4 actions
Explain more about the Eurostat specific actions 4
More integration of Eurostat data and metadata in NSI dissemination
Embeddable visualisation tool
SDMX-compliant web services
Dissemination Working Group Meeting November 2011

11. VIP “Enhanced Dissemination Chain”
2010 : Eurostat’s Dissemination Chain was renovated
Eurobase
Comext /
Dissemination
Data Explorer
TGM
Etc
This is an extremely simplified representation of Eurostat's dissemination chain.
Dissemination Chain, this is covering
- the reference database Eurobase,
- Comext dissemination and
- the Eurostat website, including its data access and visualisation tools.
Eurobase - Main Reference database containing data for about 5000 non comext datasets
Comext - Reference Database for external trade and Prodcom data ( datasets)
Comext Dissemination - Database containing data from both Eurobase ( datasets) and Comext databases (180 datasets)
New Query Generator (NQG) - The tool used to customise a dataset’s default view
Data Explorer - The main application used to access datasets
TGM – Table Graphs & Maps
Dissemination Working Group Meeting November 2011

12. VIP “Enhanced Dissemination Chain”
Actions
Deliverables 1
Development towards a Eurostat / ESS
data warehouse
Strategy (to be worked out with NSIs)
Checking algorithms in Eurobase
Simplification and integration of the Eurostat Dissemination Chain
More functionality for producing and disseminating Eurostat policy indicators 2 3
Service Impact Manager
Possible direct connection between
Data Explorer & Eurobase
New production line for
indicator tables
Flexible dashboards for
policy indicators
What will happen in the 2011 to 2013 period ? 4 actions
Explain more about the Eurostat specific actions 4
More integration of Eurostat data and metadata in NSI dissemination
Embeddable visualisation tool
SDMX-compliant web services
Dissemination Working Group Meeting November 2011

13. VIP “Enhanced Dissemination Chain”
Timing overview for the coming months
ESS Business Process incl. governance & IT
2012 S2
Communication 1
2013
Development
& roll out
Analysis
Functional & technical
specs 4a
Preparing WS
availability
Consumption by NSI’s
Potential evolutions 4b
2011
2012
2013
Mid Oct
Mid Jan
Mid April
Dissemination Working Group Meeting November 2011

14. VIP “Enhanced Dissemination Chain”
Governance & work approach
Eurostat Steering Group
With NSIs
Regular communications
Task forces of around 5 NSIs
willing to work actively on the different VIP-EDC work areas over period
From 2012 onwards, workshops for NSIs, Eurostat or Commission users 4b 4a 1
Project will require some work by volunteer countries,
it is important that NSIs only join this project if they are able to make this short-term investment, the scale of the commitment is small: it would include sharing of their existing tools, and testing of shared tools and services.
NSIs which are prepared to make this investment could get corresponding short-term benefits as well, in terms of access to the latest tools and services being developed at Eurostat
Dissemination Working Group Meeting November 2011

15. VIP “Enhanced Dissemination Chain”
Eurostat contacts
Sabine GAGEL
Project Manager - Statistics
Unit B6 - Reference databases and metadata
Tel:
Marc VANDERPERREN
Project Manager - IT
Unit D4 - Dissemination
Tel:
Dissemination Working Group Meeting November 2011