The Vision Infrastructure Project “Enhanced Dissemination Chain” Point 17 of the agenda The Vision Infrastructure Project “Enhanced Dissemination Chain” Eurostat Unit B6 Eurostat Unit D4 Dissemination Working Group Meeting - 10-11 November 2011

VIP “Enhanced Dissemination Chain” Vision Infrastructure Projects aim to improve the vertical and horizontal integration of the European Statistical System; this ESS integration is one of the central goals of the ESS vision. Commission Communication COM(2009)404 on the production method of EU statistics: a vision for the next decade As new technologies become available, there is a clear drive to maximise their use and to gear statistical methods toward them. New forms of communicating with users and producers such as web 2.0, as well as other new developments in information technology, are likely to result in profound changes in communication channels and data handling and storage. VIPs deal with improvements which are the precondition for achieving more vertical and horizontal integration of the European Statistical System : - horizontal integration : depart from the approach where statistics are handled domain by domain - vertical integration : joint structures / tools / processes + collaborative networks The improvements enable more integrated dissemination of European data and indicators. VIPS fall under a 2009 Commission Communication on the production method of EU statistics The Communication offers a vision for reforming the production method of European statistics., departing from the classical stovepipe model Particular attention goes to the usage of the internet and web based technology in a broad sense, as these are nowodays the main vehicle for statistical dissemination. Technology allows achieving dissemination targets such as full access to data, to all users at the same time & under same conditions, free of charge Dissemination Working Group Meeting - 10-11 November 2011 http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2009:0404:FIN:EN:PDF

VIP “Enhanced Dissemination Chain” Enhanced Dissemination Chain - From 2011 to 2013 Short term Eurostat’s New Dissemination Chain : improvement, simplification & better integration Disseminated data : quality improvement EU policy indicators : introduction of a new production line Involvement of ESS (Member States) in the New Dissemination Chain Medium term Further development of Eurobase towards a more integrated data warehouse for Eurostat and the ESS Better integration of Eurostat & national dissemination Short term – definition of short term 1. simplification and better integration, diminish the risk of disruption of a) the daily data updates and b) the online data access, PLUS increase the performance of dissemination 2. data quality of the European data which is disseminated, in particular through the use of checking algorithms on all published data (Eurobase functionalities) 3. producing and disseminating policy-oriented indicators includes the development of (a) a new production line for indicator tables, to facilitate flexible combination of data from several domains and (b) new ways of presenting indicators and dashboards on the Eurostat website, would be used for the EU2020 indicators, the Sustainable Development Indicators, macroecomic imbalance scorecard... 4. Involvement in the dissemination chain software components & tools to be shared – will detail in a few minutes minute Medium Term – definition of medium term 1. Data warehouse Will detail more about this in the next slide 2. better integration within the dissemination of Member States. This covers further work to facilitate NSIs putting European data and metadata on their websites Dissemination Working Group Meeting - 10-11 November 2011

VIP “Enhanced Dissemination Chain” Actions Deliverables Development towards a Eurostat / ESS data warehouse 1 Strategy (to be worked out with NSIs) Simplification and integration of the Eurostat Dissemination Chain More functionality for producing and disseminating Eurostat policy indicators 2 3 COLOR CODING ORANGE = NSIs involved / impacted / can use deliverables BLUE = EUROSTAT action or deliverable What will happen in the 2011 to 2013 period ? 4 actions 2 with deliverables relevant to the NSI’s MEDIUM TERM Warehouse strategy Context : Classical approach so far : statistics for specific domains are produced independently from each other (stovepipe); For the future : establish a model (the so-called data warehouse approach) for an integrated approach accross domains -> topic to be further explored SHORT TERM OBJECTIVES Visualisation & SDMX-compliant web services : Involvement of ESS (Member States) in the New Dissemination Chain 2 with deliverables focussing more on Eurostat’s internal dissemination chain - Improvement of the quality of the data disseminated More integration of Eurostat data and metadata in NSI dissemination 4 Embeddable visualisation tool SDMX-compliant web services Dissemination Working Group Meeting - 10-11 November 2011

VIP “Enhanced Dissemination Chain” 1 Eurostat / ESS Data Warehouse 2nd half of 2012 ESS Business Process Incl. governance & IT Communication on impact Strategy, development & implementation Strategy paper 4 steps, blue underline = Eurostat milestone Organe underline = milstone involving NSIs Dissemination Working Group Meeting - 10-11 November 2011

VIP “Enhanced Dissemination Chain” Embeddable visualisation tool for NSIs Simple visualisation tool, targetting a broad audience Widget(s) based To be embedded into the website(s) of NSI’s In the language(s) of the NSI website(s) Consuming Eurostat data Development by Eurostat, in close collaboration with NSIs Dissemination Working Group Meeting - 10-11 November 2011

VIP “Enhanced Dissemination Chain” Embeddable visualisation tool for NSIs ...building on experience in the open data community Dissemination Working Group Meeting - 10-11 November 2011 http://www.guardian.co.uk/world/interactive/2011/mar/14/new-europe-statistics-interactive

VIP “Enhanced Dissemination Chain” Embeddable visualisation tool for NSIs ...building on experience in the open data community Dissemination Working Group Meeting - 10-11 November 2011 http://www.idescat.cat/api/indicadors/giny/?lang=en

VIP “Enhanced Dissemination Chain” 4b Access to Eurostat data & metadata via SDMX-compliant web services Automated access, SDMX (2.0) compliant : data request using SDMX Queries download facility for the results and pre-generated data set(s) in SDMX-ML format Availability : 2012 Q1 Possibility to consume services exposed by eServices application The dataset web service exposes three web methods. The services exposed by Dataset web service are Get available datasets Get Data Structure Definition Execute SDMX Query The main purpose of eService application is to give professional users of Eurostat’s web site a download access to Eurostat’s data. The application provides a SDMX compliant web service to query this data and an RSS feeds service to receive notifications on data changes and updates. It also provides file download interface (centralised point of access) for these users to have an automated download of ESTAT data Dissemination Working Group Meeting - 10-11 November 2011

VIP “Enhanced Dissemination Chain” Actions Deliverables 1 Development towards a Eurostat / ESS data warehouse Strategy (to be worked out with NSIs) Simplification and integration of the Eurostat Dissemination Chain More functionality for producing and disseminating Eurostat policy indicators 2 3 What will happen in the 2011 to 2013 period ? 4 actions Explain more about the Eurostat specific actions 4 More integration of Eurostat data and metadata in NSI dissemination Embeddable visualisation tool SDMX-compliant web services Dissemination Working Group Meeting - 10-11 November 2011

VIP “Enhanced Dissemination Chain” 2010 : Eurostat’s Dissemination Chain was renovated Eurobase Comext / Dissemination Data Explorer TGM Etc This is an extremely simplified representation of Eurostat's dissemination chain. Dissemination Chain, this is covering - the reference database Eurobase, - Comext dissemination and - the Eurostat website, including its data access and visualisation tools. Eurobase - Main Reference database containing data for about 5000 non comext datasets Comext - Reference Database for external trade and Prodcom data (+- 1100 datasets) Comext Dissemination - Database containing data from both Eurobase (+-5000 datasets) and Comext databases (180 datasets) New Query Generator (NQG) - The tool used to customise a dataset’s default view Data Explorer - The main application used to access datasets TGM – Table Graphs & Maps Dissemination Working Group Meeting - 10-11 November 2011

VIP “Enhanced Dissemination Chain” Actions Deliverables 1 Development towards a Eurostat / ESS data warehouse Strategy (to be worked out with NSIs) Checking algorithms in Eurobase Simplification and integration of the Eurostat Dissemination Chain More functionality for producing and disseminating Eurostat policy indicators 2 3 Service Impact Manager Possible direct connection between Data Explorer & Eurobase New production line for indicator tables Flexible dashboards for policy indicators What will happen in the 2011 to 2013 period ? 4 actions Explain more about the Eurostat specific actions 4 More integration of Eurostat data and metadata in NSI dissemination Embeddable visualisation tool SDMX-compliant web services Dissemination Working Group Meeting - 10-11 November 2011

VIP “Enhanced Dissemination Chain” Timing overview for the coming months ESS Business Process incl. governance & IT 2012 S2 Communication 1 2013 Development & roll out Analysis Functional & technical specs 4a Preparing WS availability Consumption by NSI’s Potential evolutions 4b 2011 2012 2013 Mid Oct Mid Jan Mid April Dissemination Working Group Meeting - 10-11 November 2011

VIP “Enhanced Dissemination Chain” Governance & work approach Eurostat Steering Group With NSIs Regular communications Task forces of around 5 NSIs willing to work actively on the different VIP-EDC work areas .. .. over period 2012 - 2013 From 2012 onwards, workshops for NSIs, Eurostat or Commission users 4b 4a 1 Project will require some work by volunteer countries, it is important that NSIs only join this project if they are able to make this short-term investment, the scale of the commitment is small: it would include sharing of their existing tools, and testing of shared tools and services. NSIs which are prepared to make this investment could get corresponding short-term benefits as well, in terms of access to the latest tools and services being developed at Eurostat Dissemination Working Group Meeting - 10-11 November 2011

VIP “Enhanced Dissemination Chain” Eurostat contacts Sabine GAGEL Project Manager - Statistics Unit B6 - Reference databases and metadata Tel: 00352 4301 36734 Email: sabine.gagel@ec.europa.eu Marc VANDERPERREN Project Manager - IT Unit D4 - Dissemination Tel: 00352 4301 30697 Email: marc.vanderperren@ec.europa.eu Dissemination Working Group Meeting - 10-11 November 2011