1. Day 4 Metadata Statistics Canada December 1 st 2011 SIMPII – Workshop on Information Technology

2. 27/10/2014 Statistics Canada Statistique Canada 2 Outline  What is metadata?  Standards  Why is it important?  Implementation example with Social Surveys Common Tools

3. 27/10/2014 Statistics Canada Statistique Canada 3 What is metadata?  Definition: “Metadata is structured information that describes, explains, locates, or otherwise makes it easier to retrieve, use, or manage an information resource” * *NISO (2004) Understanding Metadata. Bethesda, NISO Press  Describes content, quality, condition and other characteristics about data

4. 27/10/2014 Statistics Canada Statistique Canada 4 What is metadata?  Metadata answers questions about your data: What is the concept? Where is the input source? What is it used for? When did it changed? Who changed the variable last?  Helps to improve the communication between: Data developers, Data users and Organizations

5. 27/10/2014 Statistics Canada Statistique Canada 5 Standards  Intended to establish a common understanding of the meaning or semantics of the data  As an example in StatCan, we use : DDI : standard for technical documentation describing social science data

6. 27/10/2014 Statistics Canada Statistique Canada 6 Why is it important?  Records basic information about your data  Provides a common understanding of your data  Allows for reuse during Survey Development Life Cycle  Facilitates connections between systems & services  Support archiving & preservation

7. 27/10/2014 Statistics Canada Statistique Canada 7 Example “dog” “golden retriever puppy” Clearly, this more specific search term is better. But it only works if someone has taken the time to associate the metadata.

8. 27/10/2014 Statistics Canada Statistique Canada 8 Example This puppy example illustrates not only the effectiveness of metadata but also the importance of tagging content with metadata. If users don’t take the time to attach metadata when they create, upload, or edit documents the benefits will be lost. DOC document name audience expiration date version department project

9. 27/10/2014 Statistics Canada Statistique Canada 9 Enterprise Metadata Classification

10. 27/10/2014 Statistics Canada Statistique Canada 10 Common Tools Logo

11. 27/10/2014 Statistics Canada Statistique Canada 11 Common Tools Technical Architecture

12. 27/10/2014 Statistics Canada Statistique Canada 12 Solution Overview  Social Survey Metadata Environment (SSME) Supporting environment of a metadata driven processing system  Interfaces are developed to access and manipulate appropriate metadata in support of a particular business process Questionnaire Development (QDT) Data Dictionary (DDT) Processing and Specifications (PST) Derived Variable (DVT)

13. 27/10/2014 Statistics Canada Statistique Canada 13 Solution Overview  Social Survey Processing Environment (SSPE) A set of generalized processes that can be used in the processing activities of the Survey Life Cycle.  The purpose of these processes is to allow subject matter and survey support staff to specify and run the processing of a survey in a timely fashion with high quality outputs.

14. 27/10/2014 Statistics Canada Statistique Canada 14 Questionnaire Development Tool screenshot

15. 27/10/2014 Statistics Canada Statistique Canada 15 Questionnaire Development Tool screenshot

16. 27/10/2014 Statistics Canada Statistique Canada 16 QDT Auto-generated Report CELL_Q03For which of the following reasons did she get her cell phone? Pour quelles raisons, parmi les suivantes, a-t-elle acquis son téléphone cellulaire? INTERVIEWER: Read categories to respondent. Mark all that apply. INTERVIEWEUR : Lisez les catégories au répondant. Choisissez toutes les réponses appropriées. 01It was a giftC'était un cadeau 02In case of emergencyEn cas d'urgence 03Peer influenceInfluence des pairs 04Work requires itRequis pour le travail 05To browse the InternetPour naviguer Internet 06To replace a regular landline phone Pour remplacer un téléphone régulier 07To replace another multimedia player Pour remplacer un autre appareil multimédia 08OtherAutres DK, RFNSP, RF

17. 27/10/2014 Statistics Canada Statistique Canada 17 Processing Specifications Tool

18. 27/10/2014 Statistics Canada Statistique Canada 18 Processing Specifications Tool

19. 27/10/2014 Statistics Canada Statistique Canada 19 Processing Specifications Tool

20. 27/10/2014 Statistics Canada Statistique Canada 20 Data Dictionary Tool output CodeAnswer CategoriesFrequenciesPopulation% 1Yes22,3454,746,56117 2No108,65523,080,67082 6Valid skip950 201,801 1 7Don’t know3 637 0 8Refusal12120 9Not Stated510620 Total131,95928,030,943100 Variable Name: CELL_03A Length: 1 Position: 5 Question Name: CELL_Q03 Concept: Reasons to get a cell phone – Gift Question:For which of the following reasons did you get your cell phone ? – Gift Universe: Respondents who answered CELL_1=1

21. 27/10/2014 Statistics Canada Statistique Canada 21 Common Tools Entity Relationship Diagram

22. 27/10/2014 Statistics Canada Statistique Canada 22 Common Tools Portal

23. SDMX  Statistical Data and Metadata eXchange (born in 2002) - Standardization for statistical data and metadata access and exchange - Between NSO’s and international organizations - Within a national statistical system - Within an organization - For dissemination  Sponsors: BIS, ECB, EUROSTAT, IMF, OECD, UN, World Bank  1) Technical standards (v1: ISO 17369) - XML-based message formats (SDMX-ML) - GESMES and the UN/EDIFACT-based message formats - Guidelines for SDMX web service implementations - SDMX registry specification (“yellow pages”)  2) SDMX Content-Oriented Guidelines - Statistical subject-matter domains (to locate data and working groups) - Cross-domain concepts/code lists (incl. metadata concepts, mapping if difficult to agree) - Metadata common vocabulary (terminology)

24. SDMX Plans for Statistics Canada  Create SDMX-ML outputs from CANSIM  Investigate OECD implementation of SDMX using.STAT software  Participate in Statistical network -- Innovation in dissemination, Machine to machine transfer stream with Stats New Zealand, Australian Bureau of Statistics  Investigate implementation of SDMX Reference Infrastructure from Eurostat

25. 27/10/2014 Statistics Canada Statistique Canada 25 Conclusion  Communication is key to collaboration  Help for decision making  Reduces system and data redundancy  Enables enterprise-wide application development

26. 27/10/2014 Statistics Canada Statistique Canada 26 Jean Labbé Field IT Manager Statistical Information System Division Informatics Branch (613) 951-2584 Jean.Labbe@statcan.gc.ca Xie xie