Presentation is loading. Please wait.

Presentation is loading. Please wait.

ESCWA SDMX Workshop Session: SDMX and Data. Session Objectives At the end of this session you will: –Know the SDMX model of a data structure definition.

Published byLogan Crawford Modified over 8 years ago

Similar presentations

Presentation on theme: "ESCWA SDMX Workshop Session: SDMX and Data. Session Objectives At the end of this session you will: –Know the SDMX model of a data structure definition."— Presentation transcript:

1 ESCWA SDMX Workshop Session: SDMX and Data

2 Session Objectives At the end of this session you will: –Know the SDMX model of a data structure definition –Understand the techniques to identify the structure of data –Identify the concepts in a simple data set –Be able to develop simple data structure definitions using SDMX tools

3 Data Set

4 Data Set: Structure

5 Data Set Structure Computers need to know the structure of data in terms of: –Concepts –Code Lists –Dimensionality –Additional metadata

6 First: Identify the Concepts A concept is a unit of knowledge created by a unique combination of characteristics (SDMX Information Model)

7 Unit Multiplier Unit Topic Time/Frequency Country Stock/Flow Data Set Structure: Concepts

8 Data Set Structure: Code Lists Code Lists TOPIC A Brady Bonds B Bank Loans C Debt Securities AR Argentina MX Mexico ZA South Africa COUNTRYSTOCK/FLOW 1 Stock 2 Flow CONCEPTS Topic Country Flow Concepts

9 16457 Q,ZA,B,1,1999-06-30=16457 Data Makes Sense

10 Data Set Structure: Defining Multi- dimensional Structures Comprises –Concepts that identify the observation value –Concepts that add additional metadata about the observation value –Concept that is the observation value –Any of these may be coded text date/time number etc. Dimensions Attributes Measure Representation

11 Data Set Structure: Concept Usage Unit Multiplier Unit Topic Time/Frequency Country Stock/Flow Observation (Dimension) (Attribute) (Dimension) (Attribute) (Measure)

12 has code list Code List Attributes concepts that add metadata has format concepts that identify groups of keys concepts that identify the observation Data Structure Definition Key Group Key Dimensions Concept Measures CONCEPTS Topic Country Flow takes semantic from has format takes semantic from has format concepts that are observed phenomenon TOPIC A Brady Bonds B Bank Loans C Debt Securities Representation Coded Non- coded

13 16457 Q,ZA,B,1,1999-06-30=16457 Data Makes Sense Frequency,Country,Topic,Stock/Flow,Time=Observation Quarterly, South Africa, Bank Loans, Stocks, 2 nd quarter 1999

14 Identifying Concepts Identifying Concepts - Sources –Existing data set tables From website From applications –Data Collection Instruments Questionnaires Excel spreadsheets –Regulations, Handbooks, User Guides Labour Statistics Convention, 1985 (No. 160), Recommendation, 1985 (No. 170) Council Regulation No: 311/76/EEC of 09/021976; OJ: L039 of 14/02/1976; Compilation of statistics on foreign workers –Database Tables –Existing Data Structure Definitions From other organisations

15 Identify Concepts – from website Source: FAO proof of concept project Measurement = 1,000 Kg

16 Concepts Reference Region Commodity Frequency and Time Observation Value Measure Type Unit and Unit Multiplier Measurement = 1,000 Kg

17 Exercise: Identify Concept Role

18 Concept Role: Reminder Dimensions –Are the concepts that identify the observation value Attributes –Are the concepts that add additional metadata about the observation value Measure –Is the concept that is the observation value

19 Concepts Reference Region Commodity Frequency and Time Observation Value Measure Type Unit and Unit Multiplier Measurement = 1,000 Kg

20 Exercise:Concept Role Reference Region Commodity Frequency and Time Observation Value Measure Type Unit and Unit Multiplier Measurement = 1,000 Kg (Dimension) (Dimensions) (Measure) (Dimension) (Attributes)

21 Data Set and Structure Dimension Concept FREQ REF_AREA_REG COMMODITY MEASURE_TYPE TIME Measure Concept OBS_VALUE Attribute Concept OBS_STATUS OBS_CONF UNIT UNIT_MULTIPLIER

22 Identify/Define Code Lists Purpose of a Code List –Constrains the value domain of concepts when used in a structure like a data structure definition –Defines a shortened language independent representation of the values –Gives semantic meaning to the values, possibly in multiple languages Agreeing on harmonised code lists is the most difficult aspect of defining a data structure definition

23 Code Lists Required Source: FAO proof of concept project Reference Region Commodity Frequency Measure Type Unit and Unit Multiplier Measurement = 1,000 Kg

24 Code Lists

25

26 Code Lists (CL_) For Time Series the SDMX Cross Domain Concepts recommend all observations have a status code (Concept = OBS_STATUS) and a confidentiality code (Concept = OBS_CONF)

27 Data Structure Definition

28 Key Group Key Dimensions Concept Attributes Measures takes semantic from has format takes semantic from has format concepts that add metadata concepts that identify the observation concepts that are observed phenomenon concepts that identify groups of keys Representation Coded Non- coded Code List has code list Data Structure Definition - Reminder

29 CL_FREQ CL_AREA_CTY CL_COMMODITY CL_MEASURE_ELEMENT Data Structure Definition - Agriculture CL_OBS_STATUS CL_OBS_CONF CL_UNIT CL_UNIT_MULT Data Structure Definition Key Group Key Dimensions Concept Attributes Measures AGRICULTURE_COMMODITY OBS_STATUS OBS_CONF UNIT UNIT_MULT FREQ REF_AREA_REG COMMODITY MEASURE_TYPE TIME OBS_VALUE Representation Coded Non- coded Code List

30 © Metadata Technology SDMX and Data Formats Exercise: Identify Concepts

31 Identifying Concepts Identifying Concepts - Sources –Existing data set tables From website From applications –Data Collection Instruments Questionnaires Excel spreadsheets –Regulations, Handbooks, User Guides Labour Statistics Convention, 1985 (No. 160), Recommendation, 1985 (No. 170) Council Regulation No: 311/76/EEC of 09/021976; OJ: L039 of 14/02/1976; Compilation of statistics on foreign workers –Database Tables –Existing Data Structure Definitions From other organisations

32 Identifying Concepts Identifying Concepts - Sources –Existing data set tables From website From applications –Data Collection Instruments Questionnaires Excel spreadsheets –Regulations, Handbooks, User Guides Labour Statistics Convention, 1985 (No. 160), Recommendation, 1985 (No. 170) Council Regulation No: 311/76/EEC of 09/021976; OJ: L039 of 14/02/1976; Compilation of statistics on foreign workers –Database Tables –Existing Data Structure Definitions From other organisations

33 Exercise: Identify Concepts – from collection instrument Source: UNESCO Institute for Statistics

34 Data Entry - Table 2.1 Source: UNESCO Institute for Statistics

35 Data Entry - Table 2.2 Source: UNESCO Institute for Statistics

36 Identifying Concepts Identifying Concepts - Sources –Existing data set tables From website From applications –Data Collection Instruments Questionnaires Excel spreadsheets –Regulations, Handbooks, User Guides Labour Statistics Convention, 1985 (No. 160), Recommendation, 1985 (No. 170) Council Regulation No: 311/76/EEC of 09/021976; OJ: L039 of 14/02/1976; Compilation of statistics on foreign workers –Database Tables –Existing Data Structure Definitions From other organisations

37 Exercise: Identify Dimension Concepts – from website Source: International Labor Organisation

38 Identify Concepts: Table 2A Source: International Labor Organisation

39 Identify Concepts: Table 2B Source: International Labor Organisation

40 Identify Concepts: Table 2C Source: International Labor Organisation

41 Identify Concepts: Table 2D Source: International Labor Organisation

42 Identify Concepts: Table 2E Source: International Labor Organisation

43 Dimension Concept

44 Identify Concepts: Table 2A Reference Area Sex Time PeriodFrequency Measure Type

45 Identify Concepts: Table 2B Economic Activity Measure Type

46 Identify Concepts: Table 2C OCCUPATION Measure Type

47 Identify Concepts: Table 2D Status in Employment Measure Type

48 Identify Concepts: Table 2E Measure Type

49 Exercise: Identify Concepts – from collection instrument Source: UNESCO Institute for Statistics Time Reference Area

50 Dimension Concepts - Tables 2.1/2.2 Source: UNESCO Institute for Statistics Education Level Sex Institution Type Measure Type Work Mode Programme Orientation

51 © Metadata Technology Labor Statistics: Data Structure Definition (Incomplete)

52 Dimension ConceptRepresentation Frequency (FREQ)CL_FREQ Reference Area (REF_AREA)CL_REF_AREA Education level (EDUC_LEVEL)CL_EDUCATLVTYP Sex (SEX)CL_SEX Programme Orientation (PROG_ORIENTATION) CL_PROG_ORIENTATION Institution Type (INSTITUTION_TYPE)CL_INSTITUTION_TYPE Work Mode (WORK_MODE)CL_WORK_MODE Measure Type (MEASURE_TYPE)CL_MEASURE_TYPE Time (TIME)Date/Time Measure ConceptRepresentation Observation Value (OBS_VAL)Numeric Education Statistics : Data Structure Definition (Incomplete)

53 Attribute ConceptAssignment Status AttachmentRepresentation Observation Status (OBS_STATUS) M(andatory)ObservationCL_OBS_STATUS Observation Confidentiality (OBS_CONF) C(onditional)ObservationCL_OBS_CONF Unit (UNIT)MSeriesCL_UNIT Unit Multiplier (UNIT_MULTIPLIER) MSeriesCL_UNIT_MULT Education Statistics : Data Structure Definition (Incomplete)

54 Identify Concepts from User Guide

Download ppt "ESCWA SDMX Workshop Session: SDMX and Data. Session Objectives At the end of this session you will: –Know the SDMX model of a data structure definition."

Similar presentations

National Institute of Statistics, Geography and Informatics (INEGI) Implementation of SDMX in Mexico.

National Institute of Statistics, Geography and Informatics (INEGI) Implementation of SDMX in Mexico.
CASE STUDY: IMPLEMENTING SDMX EXCHANGE WITH MEMBER COUNTRIES IN SHORT-TERM ECONOMIC STATISTICS (STES)

CASE STUDY: IMPLEMENTING SDMX EXCHANGE WITH MEMBER COUNTRIES IN SHORT-TERM ECONOMIC STATISTICS (STES)
SDMX training session on basic principles, data structure definitions and data file implementation 29 November 2007 2.

SDMX training session on basic principles, data structure definitions and data file implementation 29 November
1 The BIS-IMF-OECD-World Bank Joint External Debt Hub Presentation by Ibrahim Levent, Development Data Group, WB René Piché, Statistics Department, IMF.

1 The BIS-IMF-OECD-World Bank Joint External Debt Hub Presentation by Ibrahim Levent, Development Data Group, WB René Piché, Statistics Department, IMF.
© Metadata Technology ESCWA SDMX Workshop Session: Data Formats.

© Metadata Technology ESCWA SDMX Workshop Session: Data Formats.
The 2010 World Programme on Population and Housing Censuses Paul Cheung, Director United Nations Statistics Division.

The 2010 World Programme on Population and Housing Censuses Paul Cheung, Director United Nations Statistics Division.
Statistical Databases: A short review Slide: 1 Demonstration of the Prototype model for ECA Statistical Database Statistical Database Application March.

Statistical Databases: A short review Slide: 1 Demonstration of the Prototype model for ECA Statistical Database Statistical Database Application March.
Sdmx web services Strutural data

Sdmx web services Strutural data
REFERENCE METADATA FOR DATA TEMPLATE Ales Capek EUROSTAT.

REFERENCE METADATA FOR DATA TEMPLATE Ales Capek EUROSTAT.
SDMX AND DATA DISSEMINATION SDMX Training BANK INDONESIA 16-18 SEPTEMBER 2015 YOGYAKARTA, INDONESIA.

SDMX AND DATA DISSEMINATION SDMX Training BANK INDONESIA SEPTEMBER 2015 YOGYAKARTA, INDONESIA.
EXERCISE: IDENTIFY CONCEPTS SDMX Training BANK INDONESIA 16-18 SEPTEMBER 2015 YOGYAKARTA, INDONESIA.

EXERCISE: IDENTIFY CONCEPTS SDMX Training BANK INDONESIA SEPTEMBER 2015 YOGYAKARTA, INDONESIA.
CountryData Technologies for Data Exchange SDMX Information Model: An Introduction.

CountryData Technologies for Data Exchange SDMX Information Model: An Introduction.
Eurostat Unit B3 – IT and standards for data and metadata exchange SDMX Basics Training – 2012 IT architectures for data exchange SDMX-RI and the Hub approach.

Eurostat Unit B3 – IT and standards for data and metadata exchange SDMX Basics Training – 2012 IT architectures for data exchange SDMX-RI and the Hub approach.
Eurostat – Directorate B: Corporate statistical and IT services SDMX Basics Training – 2013 SDMX basics Marco Pellegrino Eurostat, Directorate B.

Eurostat – Directorate B: Corporate statistical and IT services SDMX Basics Training – 2013 SDMX basics Marco Pellegrino Eurostat, Directorate B.
« 8-11 July 2008 « Metadata Life Cycle « STATISTICS PORTUGAL.

« 8-11 July 2008 « Metadata Life Cycle « STATISTICS PORTUGAL.
Monitoring public satisfaction through user satisfaction surveys Committee for the Coordination of Statistical Activities Helsinki 6-7 May 2010 Steve.

Monitoring public satisfaction through user satisfaction surveys Committee for the Coordination of Statistical Activities Helsinki 6-7 May 2010 Steve.
13-Jul-07 Implementation of SDMX for data and metadata exchange Balance of Payments Working Group 2-3 April 2012 Daniel Suranyi Eurostat B5 Management.

13-Jul-07 Implementation of SDMX for data and metadata exchange Balance of Payments Working Group 2-3 April 2012 Daniel Suranyi Eurostat B5 Management.
SDMX DATA STRUCTURE DEFINITION SDMX Training BANK INDONESIA 16-18 SEPTEMBER 2015 YOGYAKARTA, INDONESIA.

SDMX DATA STRUCTURE DEFINITION SDMX Training BANK INDONESIA SEPTEMBER 2015 YOGYAKARTA, INDONESIA.
1 Eurostat Unit B5 – Statistical Information Technologies SDMX Basics – October 2011 SDMX Basics Core Elements Information Model Data Structure Definition.

1 Eurostat Unit B5 – Statistical Information Technologies SDMX Basics – October 2011 SDMX Basics Core Elements Information Model Data Structure Definition.
Legal and institutional foundation of economic statistics Overview of international experience Regional Workshop for African Countries on Compilation of.

Legal and institutional foundation of economic statistics Overview of international experience Regional Workshop for African Countries on Compilation of.

Similar presentations

Ads by Google