23-25/5/2011 Modernisation of ESS infrastructure: The ESS instruments - a review E. di Meglio – P. Jacques – J.M. Museux

23-25 May 2011MSIS Conference The Vision Integration and standardisation of ESS processes : data warehouses Incorporation of ICT development (SOA, distributed architecture, Web 3.0) New methods for producing statistics (reuse, data matching and linking, mixed modes approach, visualisation and analysis of big datasets, …) New generation of indicators (Stiglitz …)

23-25 May 2011MSIS Conference Legal framework : from stove pipe legislation to cross cutting framework legislation Methodological developments /standardisation –Best practices – guidelines – rules –Common tools Modernisation of IT infrastructure (SOA, web services, distributed and cloud computing, use of standardised IT solutions) Standardisation and industrialisation –Common description and semantic –Common data exchange standard –Common metadata –Common tools ESS actions required BACK

23-25 May 2011MSIS Conference Reference ESS Enterprise architecture BACK International standards

23-25 May 2011MSIS Conference Role of Research Framework programme in Official Statistics –FP5 (35 seminal research projects) – FP7 (new indicators, data infrastructure, ) and upcoming FP8 Preparing and addressing future challenge –New data sources (i.e. Social networking, Internet of Things, Data Streams, mobile info/GI,..) –Data analysis / data mining / visualisation of big datasets (Data Deluge) –Web 3.0, semantics –New modes of data collection and related estimation issues –Understanding non response –Set up e-infrastructures –Critical events forecasting Collaboration with academics (statistical watch - think tank, adviser for innovation, training of Official statisticians) Research BACK

23-25 May 2011MSIS Conference ESSnets A collaborative network of several ESS organisations aimed at providing results beneficial to the whole ESS 26 ESSnet projects since ESSnet review on going – ESSnet coordination - ESSnet planning ESSnet characterisation according to their contribution to innovation process BACK

23-25 May 2011MSIS Conference ESSnets typology 1.ESSnet research consolidation and incorporation of research knowledge and testing of prototype tools Ex. SDC I - ISAD 2.ESSnet pole of expertise specialisation in process steps – cover wider range of activities (best practices identification, guidelines, tool development, dissemination of knowledge-know how and training Ex SDC II, DI, SAE, Mixed mode approach 3.ESSnet infrastructure facilitation to incorporation of international standards, recommendations, cases studies, taylorisation, improvement of standards, setting up of common infrastructure Ex. EGR, SDMX I & II, MEETS Datawarehouse, Geostat, CORA & CORE, Decentralised access to EU micro data BACK

23-25 May 2011MSIS Conference ESSnets typology 1.ESSnet standardisation contribute to knowledge and process standardisation (concept, tools, guidelines, ….) Ex SDC harmonisation, MEETS Memobust, Preparing Standardisation 2.ESSnet strategy : support to strategic action like Sponsorship on Standardisation 3.ESSnet piloting domain dependent, testing in common, practical implementation BACK

23-25 May 2011MSIS Conference ESSnet – contribution to a reference architecture CORE DI ESSnet SDMX DW Consistency Memobust SDC Harm Stand Prep Decentralised Access EGR

23-25 May 2011MSIS Conference Contribution to ESS business process BACK Administrative Data SDC SAE DI Profiling Net SILC ICT Linking Culture PHGeostat

23-25 May 2011MSIS Conference Common tool development Tool developed in common for implementing ESS guidelines Need for specific governance and development strategy –Open source –Modular – web service – portability (JAVA) –ESS Steering Group – User group Best practice –Demetra+ : Tool for implementing ESS seasonal adjustment guidelines Demetra+ user group – prototype development Java toolkit (API, GUI, Data cruncher to be run in batch, library of core functions, structured objects) Re-used for Ecotrim, NAPS developments –Argus+ : sustainability of T-Argus – proof of concept BACK

23-25 May 2011MSIS Conference Eurostat modernisation - VIPs Strategic project for Eurostat and ESS modernisation Joint venture between business area units (relevance) and horizontal units (methodology, IT) 6 projects (still on going) launched in BACK 1.VIP data validation 2.VIP data matching 3.VIP new dissemination 4.VIP National account production 5.VIP Secure infrastructure for confidential data 6.VIP on PILLAR approach for household data collection ESTAT process / infrastructure ESS design

23-25 May 2011MSIS Conference Eurostat modernisation - VIPs Common characteristics –Cross domains development –Mainly ESTAT focus - IT or process oriented –Prepare for vertical (MS- Eurostat) integration Need to maintain global coherence –Planning – yearly work programme –Spin off of ESS(net) project –ESS relevance - coordination

23-25 May 2011MSIS Conference VIP, ESSnet – elements of coherence NAPS Data Matching Pillar CORE DI SICON ESSnet SDMX VIP New Diss DW Consistency Memobust SDC Harm Stand Prep Data Validation Decentralised Access DW Common tools Demetra+ Argus+ EGR

23-25 May 2011MSIS Conference Strategic level – sponsorship Sponsorship on standardisation Sponsorship group : 6 general directors or deputies of NSOs, co chaired by Eurostat and a NSO) to build mechanism and steer competence on key challenge for the ESS over a relatively short period (2 years) Main objectives –to investigate ways to organise and strengthen the sharing of methods and tools and the creation of common infrastructures –to contribute to the establishment of an ESS Enterprise Architecture for standardisation (key principles, …) –to establish a ESS work programme towards a more integrated way of producing European statistics building on existing instrument BACK

23-25 May 2011MSIS Conference VIP, sponsorship and ESSnet – global picture Data Matching Pillar CORE DI Quality Standardisation Communication ESSnet SDMX Sponsorship VIP DW Consistency Memobust SDC Harm Stand Prep NAPS New Diss Data Validation Decentralised Access DW Common tools Demetra+ Argus+ SICON EGR

23-25 May 2011MSIS Conference Conclusions Many initiative on going – bottom up approach initially Hard to maintain coherence between different layers, scope Big hope with the Sponsorship on standardisation Need for an Enterprise Architecture at ESS level BACK