1. International Household Survey Network (IHSN) Microdata Management Toolkit Trevor Croft tcroft@blancroft.com MICS3 Data Archiving, Dissemination and Further Analysis Workshop Geneva - November 6 th, 2006 Original presentation by Pascal Heus, pascal.heus@gmail.com

2. IHSN Background International Household Survey Network Marrakech Action Plan for Statistics (MAPS) (Feb 2004) Established in September 2004 International organizations actively sponsoring household surveys + NSO Coordination: ILO, DFID, Paris 21, UNICEF, UNSD, WHO, World Bank http://www.surveynetwork.org Survey under-use: limited access, poor return on investment, limited impact, difficulties in policy making Common obstacles: quality, technical capacity, management / political / legal / ethical issues Common problems: Accessibility, Timeliness, Coherence, Lack of metadata / documentation / data, poorly organized archives

3. IHSN Activities Coordinating survey programs Web based information on planned surveys Harmonizing concepts & methods Planning, sampling, questionnaire design, processing, data disclosure & confidentiality, etc. Maintaining a survey catalog Web based central catalog Developing data dissemination tools Microdata Management Toolkit

4. IHSN Website www.internationalsurveynetwork.org

5. Microdata Management Toolkit Tools to facilitate archive and dissemination of surveys

6. Toolkit Requirements User friendly software for microdata Facilitate metadata exchange (DDI, Dublin Core) Facilitate archiving (metadata and data, quality control) Facilitate preservation/dissemination: network, CD / DVD, web sites Works with common data formats Multilingual support Free or Inexpensive Availability of technical support and training Supported by national, international and research communities

7. DESCRIPTIVE INFORMATION RESOURCE DESCRIPTION FRAMEWORK W3C CONSORTIUM RSS NEWS XML TAGS MARKUP DOCUMENT What is metadata? eXtensible Markup Language XML, XSL, DTD, XSD, SOAP, XQUERY,… METADATA EXCHANGE SEMANTIC WEB WEB 2.0 STANDARD SPECIFICATIONS EASY TO TRANSFORM HUMAN READABLE SEARCHABLE MACHINE PROCESSABLE WEB SERVICES ONTOLOGIES

8. What is metadata? DATA ABOUT DATA

9. What is metadata? Unlabeled stuffLabeled stuff The bean example is taken from: A Managers Introduction to Adobe eXtensible Metadata Platform, http://www.adobe.com/products/xmp/pdfs/whitepaper.pdf Source: Courtesy of Jostein Ryssevik, Nesstar Inc.

10. Metadata – user view HTML PDF

11. International Standards Data Documentation Initiative (DDI) Surveys (description, files, variables) Microdata Management Toolkit, DevInfo Statistical Data and Metadata Exchange (SDMX) Time series, indicators DevInfo5 Dublin Core (DC) Documents, images, maps, other resources Microdata Management Toolkit, DevInfo5, CDS/WinISIS ISO 19115 Geographical Information Systems (GIS) DevInfo5 ISO 11179 Registries, classifications, life cycle future catalog tools Important to adopt and to promote use

12. Generate HTML based CD-ROM Import metadata and prepare CD-ROM Import data and compile metadata The Toolkit Process 1 2 3

13. Toolkit Components Archiving: Metadata Editor (World Bank / Nesstar Ltd.) Compile survey data, documentation and metadata in a standard format (Nesstar/DDI). Free data reader for users. Built on Nesstar Publisher Dissemination: CD Builder (World Bank / Mark Diggory) Facilitate the publication of survey data, documentation and metadata on CD-ROM and on the web (transforms DDI into HTML based navigation) Based on Eclipse Platform, open source Nesstar Explorer Free software Access metadata & data and export to common statistical formats IHSN Tools Reporting, diagnostics and other utilities

14. What is the Metadata Editor? DDI / Dublin Core specialized editor Template driven Import/Export common data files Integrated interface, multilingual support Metadata and data in single file Export to DDI / DC Licensing agreement for developing countries and IHSN members

15. What is the Metadata Editor?

16. What is the CD Builder? Publish survey metadata, documents and data on a CD-Rom or web site) Transforms DDI into an HTML based interface User can customize the layout (branding) and content of the CD (single or multi-surveys) Based on DDI / Dublin Core Integrates with Metadata Editor

17. CD Builder Process Create new CD-ROM Project Add a survey to the project and select its type and branding 1 2 Selecting a survey consist in opening the DDI-XML or Nesstar file The survey branding determines the overall look and feel of the CD The survey type determines the default metadata content Selecting a survey consist in opening the DDI-XML or Nesstar file The survey branding determines the overall look and feel of the CD The survey type determines the default metadata content Click the Save button to generate the HTML interface 3 After a few minutes, your CD Project is ready for publishing! 4

18. Sample output

19. Nesstar Explorer Free software PDF philosophy Access to survey metadata Access to data (no need for specialized software) Export to common formats Single file holds data and metadata

20. IHSN Tools Free / Open Source Utilities to complement existing software Integrates in Editor Diagnostics Reporting Metadata sharing

21. Benefits and Users Improves and facilitates documentation preservation, cataloguing, metadata exchange, dissemination, quality, etc. Users: survey producers, survey sponsors, data archives Benefits: data producers, national & international survey sponsors, survey data repositories, data analysts, policy makers, DDI Community

22. Status & Availability Metadata Editor: Available in English, Commercial release in Nesstar 4.0 (end 2006?), testing French, Spanish & Russian CD Builder Available in English & French, testing Spanish & Russian IHSN Tools Diagnostics reports available, Survey Documentation in PDF Other I18n: working on French, Spanish, Bosnian, Russian, Portuguese,… Users Guide in English, draft in French, Spanish, Russian Open source project components hosted on SourceForge at http://www.sourceforge.net/projects/toolkit

23. Roll-out Roll-out in in 2006/2007 (global) Completed training / pilot in about 20 countries, mainly in the Africa region Use by UNICEF for Multiple Indicators Cluster Surveys (MICS) Asia: Partnership with United Nations Economic and Social Commission for Asia and the Pacific (ESCAP) Latin America: partnership with Inter-American Development Bank (IADB) and Pan-American health Organization (PAHO) Used by IHSN member agencies (WHO, ILO, etc.) Component of World Bank Accelerated Data Program (ADP)

24. Thank you! QUESTION / ANSWER