Presentation is loading. Please wait.

Presentation is loading. Please wait.

Eurostat Unit B5 – Statistical Information Technologies SDMX Basics Training – October 2011 The European Census Hub Changing the Business Model Adam Wroński,

Published byDwain Robbins Modified over 8 years ago

Similar presentations

Presentation on theme: "Eurostat Unit B5 – Statistical Information Technologies SDMX Basics Training – October 2011 The European Census Hub Changing the Business Model Adam Wroński,"— Presentation transcript:

1 Eurostat Unit B5 – Statistical Information Technologies SDMX Basics Training – October 2011 The European Census Hub Changing the Business Model Adam Wroński, Marco Pellegrino, Nadezhda Vlahova, Bengt-Åke Lindblad Eurostat, “Quality, Methodology and Information Systems” Directorate

2 2 Eurostat Unit B5 – Statistical Information Technologies SDMX Basics Training – October 2011 The European Census Hub: key issues Dissemination of the data from the 2011 population and housing censuses in the European Union Massive amount of data produced and controlled by Member States Comparable data, structured according to agreed “hypercubes”: harmonised concepts and definitions (Census Regulation) Easy access to detailed census data (advanced functionalities) Easy to use: tabulation of data from different sources High accessibility of data and metadata

3 3 Eurostat Unit B5 – Statistical Information Technologies SDMX Basics Training – October 2011 EU Census: Implementing measures Regulation (EC) 763/2008 on population and housing censuses authorises the European Commission to adopt implementing measures on: Regulation (EC) 763/2008 –technical specifications of the topics and their breakdown (Regulation (EC) 1201/2009)Regulation (EC) 1201/2009 –programme of the statistical data and metadata to be transmitted to Eurostat (Regulation (EU) 519/2010)Regulation (EU) 519/2010 –quality reporting and technical format of data transmission (Regulation (EU) 1151/2010)Regulation (EU) 1151/2010

4 4 Eurostat Unit B5 – Statistical Information Technologies SDMX Basics Training – October 2011 Article 6 of Regulation (EU) 1151/2010 –Member States shall transmit the required data conforming to the data structure definitions and related technical specifications provided by the Commission (Eurostat) –The technical format to be used for the transmission of data and metadata for the reference year 2011 shall be the Statistical Data and Metadata eXchange (SDMX) format –Member States shall store until 1 January 2025 the required data and metadata for any later transmission requested by the Commission (Eurostat) Technical format for data transmission

5 5 Eurostat Unit B5 – Statistical Information Technologies SDMX Basics Training – October 2011 A system of 60 Data Structure Definitions 60 DSDs built on SDMX 2.0 (standard concepts and codes) and in use for 31 countries of the European Statistical System SDMX data message (cross-sectional format) SDMX query message Tests on-going: started with dummy data and sample hypercube, now to continue with real hypercubes Infrastructure to be maintained (and data stored) at least until 2025

6 6 Eurostat Unit B5 – Statistical Information Technologies SDMX Basics Training – October 2011 Population topics Sex SEX AgeAGE Legal marital statusLMS Country/place of birth POB Country of citizenship COC Place of usual residence - one year prior to the censusROY (Size of the) LocalityLOC Household statusHST Type of private householdTPH Size of private householdSPH Family statusFST Type of family nucleusTFN Size of family nucleusSFN Topics required for all geographical levels down to Local Administrative Units (LAU = municipalities) Housing topics Occupancy status of conventional dwellingsOCS Number of occupantsNOC Useful floor space and/or Number of roomsUFS/NOR Density standardDFS/DRM Dwellings by type of buildingTOB Dwellings by period of constructionPOC Type of living quartersTLQ Which data for which geographical area?

7 7 Eurostat Unit B5 – Statistical Information Technologies SDMX Basics Training – October 2011 Topics required for aggregated geographical levels NUTS 2, NUTS 1 and nation Population topics Year of arrival in the country YAE / YAT Educational attainmentEDU Location of place of workLPW Current activity status CAS OccupationOCC Industry IND Status in employmentSIE Tenure status of households TSH Housing topics Housing arrangements HAR Type of ownership (of dwellings) OWS Water supply system WSS Toilet facilities TOI Bathing facilities BAT Type of heating TOH Which data for which geographical area?

8 8 Eurostat Unit B5 – Statistical Information Technologies SDMX Basics Training – October 2011 Example: DSD for Table 6 (Marital Status) IDCONCEPTCODELIST TIMETime period or rangeCL_TIME GEOGeographical areaCL_GEO SEXSexCL_SEX FSTFamily statusCL_FST LMSLegal marital statusCL_LMS CASCurrent activity statusCL_CAS POBCountry/place of birthCL_POB COCCountry of citizenshipCL_COC AGEAgeCL_AGE FREQFrequencyCL_FREQ IDATTACHMENT LEVEL CODELIST OBS_STATUSObservationCL_OBS_STATUS OBS_LEVELObservationCL_OBS_LEVEL OBS_NOTEObservation HC_NOTESeries IDNAME OBS_VALUEObservation value Dimensions Measures Attributes

9 9 Eurostat Unit B5 – Statistical Information Technologies SDMX Basics Training – October 2011 The Hub concept  The Hub is based on the concept of data sharing, where a group of partners agree on providing access to their data according to standard processes, formats and technologies.  The SDMX Hub approach offers several advantages: –decoupling of NSIs' systems from the central hub via standard formats and techniques for the exchange; –NSIs free to provide more information than what is contained in the agreed hypercubes without additional effort; –Limited investment, re-usability (with the advantage of using recognized international standards). –SDMX-RI is furnished by Eurostat.

10 10 Eurostat Unit B5 – Statistical Information Technologies SDMX Basics Training – October 2011 IT Architectures for Data Exchange

11 11 Eurostat Unit B5 – Statistical Information Technologies SDMX Basics Training – October 2011 Data Repository (Warehousing) Architecture NSI Eurostat Pull Requestor eDAMIS Data Input SDMX Registry Intermediate storage Verification / Conversion To SDMX Received data in SDMX-ML Loader register Warehouse storage Eurobase query Dissemination XSL for SDMX-ML PULLPULL PUSHPUSH

12 12 Eurostat Unit B5 – Statistical Information Technologies SDMX Basics Training – October 2011 The SDMX Hub architecture Pull mode and data sharing Hub SDMX-RI infrastructure

13 13 Eurostat Unit B5 – Statistical Information Technologies SDMX Basics Training – October 2011

14 14 Eurostat Unit B5 – Statistical Information Technologies SDMX Basics Training – October 2011 Through the SDMX Hub, a data user can… Browse the Hub to define a dataset of interest, navigating via structural metadata: - Select hypercube or search by topic (filters) - Select data (level of detail, breakdowns) - Select layout (axes) View a table Save a query Export a file (CSV, Excel, SDMX-ML)

15 15 Eurostat Unit B5 – Statistical Information Technologies SDMX Basics Training – October 2011 How the Hub works Eurostat Census Hub National Statistical Institute

16 16 Eurostat Unit B5 – Statistical Information Technologies SDMX Basics Training – October 2011 http://193.109.72.201:8080/CensusHub2/

17 17 Eurostat Unit B5 – Statistical Information Technologies SDMX Basics Training – October 2011 Where are we - ESS? Countries integrated within the Hub (19) (end 2011) Czech Republic, Germany, Ireland, Italy, Luxembourg, Poland, Portugal, Spain, Norway, Sweden, Slovenia, Bulgaria, Belgium, Latvia, Malta, Kypros, Hungary (11/2012), Finland(11/2012), France* Expressing their interest or working on it, but not yet in the Hub (6) Estonia, Greece, Lithuania, Netherlands, Switzerland, United Kingdom Asking for more information (6) Austria, Denmark, Iceland, Liechtenstein, Romania, Slovakia 1-2 June 2010: 1 st Census IT technical WG1 st Census IT technical WG 14-15 February 2011: 2 nd Census IT technical WG2 nd Census IT technical WG 7-8 December 2011: 3 rd Census IT technical WG3 rd Census IT technical WG SDMX training, technical meetings

18 18 Eurostat Unit B5 – Statistical Information Technologies SDMX Basics Training – October 2011 How to know more CIRCA - > Eurostat - > X-DIS Census Hub -> Census Hub public documents CIRCA - > Eurostat - > X-DIS Census Hub -> Census Hub public documents Read the experiences from the MSs that have already implemented the infrastructure http://circa.europa.eu/Public/irc/dsis/x-dis-xensus-hub/home http://circa.europa.eu/Public/irc/dsis/x-dis-xensus-hub/home In case of need, contact Eurostat unit B5 for technical advice –SDMX training available –Technical bilateral meeting can be arranged upon request

19 19 Eurostat Unit B5 – Statistical Information Technologies SDMX Basics Training – October 2011 How to know more (2) The Implementation Kit SDMX Structure files –ConceptsConcepts –Codelists (excluded GEO)Codelists –Keyfamilies (one for each hypercube)Keyfamilies –Partial Geo Codelist SDMX data message example (cross-sectional), SDMX Query message example and Dummy hypercube in CSV format SDMX data message example (cross-sectional), SDMX Query message example and Dummy hypercube in CSV format MIG XML schema (XSD file) Explanatory notes Census Hub WS Implementation Guideline Census Hub WS Security Guideline Census 2011 Regulation Census 2011 Explanatory notes

20 20 Eurostat Unit B5 – Statistical Information Technologies SDMX Basics Training – October 2011 Census Hub environment

21 21 Eurostat Unit B5 – Statistical Information Technologies SDMX Basics Training – October 2011 Lesson learnt and benefits in participating Statistical needs first. Then, technological aspects. Capacity-building is a must –Participating organisations are gaining a good in-house experience in SDMX and its implementation A system of distributed databases is harmonised through the use of SDMX standards and content guidelines SDMX-RI can be reused for sharing data in other domains –Limited cost for installations, development costs can be reduced –Step forward towards generic solutions for statistical domains

22 22 Eurostat Unit B5 – Statistical Information Technologies SDMX Basics Training – October 2011 For more information: Marco.Pellegrino@ec.europa.eu Nadezhda.Vlahova@ec.europa.eu Bengt-Ake.Lindblad@ec.europa.eu Unit B5, Section «Standardisation and Advanced IT for Statistics» Census Hub on CIRCA: http://circa.europa.eu/Public/irc/dsis/x-dis-xensus-hub/library

Download ppt "Eurostat Unit B5 – Statistical Information Technologies SDMX Basics Training – October 2011 The European Census Hub Changing the Business Model Adam Wroński,"

Similar presentations

UNIVERSITY OF JYVÄSKYLÄ INTERNATIONAL COOPERATION.

UNIVERSITY OF JYVÄSKYLÄ INTERNATIONAL COOPERATION.
Knowledge Management LXV International Council Meeting Qawra, Malta 16 th - 23 rd of March 2014.

Knowledge Management LXV International Council Meeting Qawra, Malta 16 th - 23 rd of March 2014.
16 out of 27 member states Known as euro zones 2 nd largest traded currency after the dollar The name euro was officially adopted on 16 December 1995.

16 out of 27 member states Known as euro zones 2 nd largest traded currency after the dollar The name euro was officially adopted on 16 December 1995.
Agenda Item 3.3 SDMX reference architecture for NSIs Francesco Rizzo 24 th Meeting of the STNE Working Group “Statistics, Telematic Network & EDI” 16-17.

Agenda Item 3.3 SDMX reference architecture for NSIs Francesco Rizzo 24 th Meeting of the STNE Working Group “Statistics, Telematic Network & EDI”
European Commission TACIS programme. European Business Registration Survey 2005.

European Commission TACIS programme. European Business Registration Survey 2005.
ELSA Shop(ping) LXIV International Council Meeting Opatija, Croatia October 28 th - November 3 rd 2013.

ELSA Shop(ping) LXIV International Council Meeting Opatija, Croatia October 28 th - November 3 rd 2013.
1 SDMX Reference Infrastructure SDMX Global Conference 2 – 4 May 2011, Washington DC Adam Wroński, Marco Pellegrino, Bengt-Åke Lindblad, Nadezhda Vlahova.

1 SDMX Reference Infrastructure SDMX Global Conference 2 – 4 May 2011, Washington DC Adam Wroński, Marco Pellegrino, Bengt-Åke Lindblad, Nadezhda Vlahova.
Eurostat Unit B3 – IT and standards for data and metadata exchange SDMX Basics Training – 2012 IT architectures for data exchange SDMX-RI and the Hub approach.

Eurostat Unit B3 – IT and standards for data and metadata exchange SDMX Basics Training – 2012 IT architectures for data exchange SDMX-RI and the Hub approach.
Eurostat – Directorate B: Corporate statistical and IT services SDMX Basics Training – 2013 SDMX basics Marco Pellegrino Eurostat, Directorate B.

Eurostat – Directorate B: Corporate statistical and IT services SDMX Basics Training – 2013 SDMX basics Marco Pellegrino Eurostat, Directorate B.
European Business Register Congress of the Notaries of Europe, Brussels, 28 June 2011.

European Business Register Congress of the Notaries of Europe, Brussels, 28 June 2011.
THE EUROPEAN UNION. HISTORY 28 European states after the second world war in 1951 head office: Brussels 24 different languages Austria joined 1995.

THE EUROPEAN UNION. HISTORY 28 European states after the second world war in 1951 head office: Brussels 24 different languages Austria joined 1995.
13-Jul-07 Implementation of SDMX for data and metadata exchange Balance of Payments Working Group 2-3 April 2012 Daniel Suranyi Eurostat B5 Management.

13-Jul-07 Implementation of SDMX for data and metadata exchange Balance of Payments Working Group 2-3 April 2012 Daniel Suranyi Eurostat B5 Management.
1 Meeting on the Management of Statistical Information Systems (MSIS 2010) SDMX architecture for data sharing and interoperability Francesco Rizzo, ISTAT,

1 Meeting on the Management of Statistical Information Systems (MSIS 2010) SDMX architecture for data sharing and interoperability Francesco Rizzo, ISTAT,
Slide 1 Eurostat Unit B3 – Statistical Information Technologies CoRD Meeting – 4 June 2007 Agenda Item 8 Preliminary ideas for a 2011 census hub Giuseppe.

Slide 1 Eurostat Unit B3 – Statistical Information Technologies CoRD Meeting – 4 June 2007 Agenda Item 8 Preliminary ideas for a 2011 census hub Giuseppe.
Institutional Visit LXV International Council Meeting Qawra, Malta 16 th - 23 rd of March 2014.

Institutional Visit LXV International Council Meeting Qawra, Malta 16 th - 23 rd of March 2014.
Natural gas, and oil sectors in Europe Vaidotas Levickis Fort Worth, Texas 2015.

Natural gas, and oil sectors in Europe Vaidotas Levickis Fort Worth, Texas 2015.
Eurostat 6. SDMX: A non-technical overview of the SDMX architecture and IT tools 1 Raynald Palmieri Eurostat Unit B5: “Central data and metadata services”

Eurostat 6. SDMX: A non-technical overview of the SDMX architecture and IT tools 1 Raynald Palmieri Eurostat Unit B5: “Central data and metadata services”
2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015.

2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, October 2015.
© Enterprise Europe Network South West 2009 The Eurostars Programme Kenny Legg R&D Funding for the Environmental Sector – 29 June 2010 European Commission.

© Enterprise Europe Network South West 2009 The Eurostars Programme Kenny Legg R&D Funding for the Environmental Sector – 29 June 2010 European Commission.
SDMX IT Tools SDMX use in practice in NA

SDMX IT Tools SDMX use in practice in NA

Similar presentations

Ads by Google