ESCWA SDMX Workshop Session: Role in the Statistical Lifecycle and Relationship with DDI (Data Documentation Initiative)

Slides:

Advertisements

Similar presentations

IZA Data Service Center DDI/SDMX Workshop Wiesbaden, Germany, June 18th 2008 The Data Documentation Initiative (DDI) Arofan Gregory / Pascal Heus

Advertisements

Workshop on Metadata Standards and Best Practices November th, 2007 Session 4 The Data Documentation Initiative Technical Overview Pascal Heus Open.

3rd International Digital Curation Conference Washington, DC, Dec 2007 Paper Presentations: Interoperability, Metadata & Standards Data Documentation Initiative:

Wendy Thomas November 28-29, 2012

Making the Case for Metadata at SRS-NSF National Science Foundation Division of Science Resources Statistics Jeri Mulrow, Geetha Srinivasarao, and John.

ESSnet on SDMX phase II Laura Vignola ISTAT Rome, 3-4 December 2012.

[Meta]-Data Management using DDI

DDI URN Enabling identification and reuse of DDI metadata IDSC of IZA/GESIS/RatSWD Workshop: Persistent Identifiers for the Social Sciences Joachim Wackerow.

DDI 3.0 Conceptual Model Chris Nelson. Why Have a Model Non syntactic representation of the business domain Useful for identifying common constructs –Identification,

SDMX and DDI: How Do They Fit Together in Practical Terms? Arofan Gregory The Open Data Foundation European DDI User’s Group 2011 Gothenburg, Sweden.

Präsentationstitel IAB-ITM Find the right tags in DDI IASSIST 2009, 27th-30th Mai 2009 IAB-ITM Finding the Right Tags in DDI 3.0: A Beginner's Experience.

Codebook Centric to Life-Cycle Centric In the beginning….

Managing the Metadata Lifecycle The Future of DDI at GESIS and ICPSR Peter Granda, ICPSR Meinhard Moschner, GESIS Mary Vardigan, ICPSR Joachim Wackerow,

Reducing Metadata Objects Dan Gillman November 14, 2014.

 Name and organization  Have you worked with DDI before? (2 or 3)  If not, are you familiar with XML?  What kind of CAI systems do you use?  Goals.

The education variables in the European Social Survey: Advantages in using the DDI for documentation Hilde Orten and Hege Midtsæter Norwegian Social Science.

Course on DDI 3: Putting DDI to Work for You 8 December 2010 Wendy Thomas, Minnesota Population Center 2 nd Annual European DDI Users Group Meeting Learning.

Environment Change Information Request Change Definition has subtype of Business Case based upon ConceptPopulation Gives context for Statistical Program.

ISO as the metadata standard for Statistics South Africa

Data Documentation Initiative (DDI): Goals and Benefits Mary Vardigan Director, DDI Alliance.

Overview of DDI Arofan Gregory METIS October 5-7, 2011.

WP.5 - DDI-SDMX Integration

WP.5 - DDI-SDMX Integration E.S.S. cross-cutting project on Information Models and Standards Marco Pellegrino, Denis Grofils Eurostat METIS Work Session6-8.

NSI 1 Collect Process AnalyseDisseminate Survey A Survey B Historically statistical organisations have produced specialised business processes and IT.

Case Studies: Statistics Canada (WP 11) Alice Born Statistics UNECE Workshop on Statistical Metadata.

Survey Data Management and Combined use of DDI and SDMX DDI and SDMX use case Labor Force Statistics.

Statistics New Zealand Classification Management System Andrew Hancock Statistics New Zealand Prepared for 2013 Meeting of the UN Expert Group on International.

Statistics New Zealand Classification Management System Andrew Hancock Statistics New Zealand Prepared for 2013 Meeting of the UN Expert Group on International.

Just Enough DDI 3 for the “DDI: Managing Metadata for Longitudinal Data — Best Practices”

SDMX and DDI Working Together Technical Workshop 5-7 June 2013

DDI: Capturing metadata throughout the research process for preservation and discovery Wendy Thomas NADDI 2012 University of Kansas.

3 rd Annual European DDI Users Group Meeting, 5-6 December 2011 The Ongoing Work for a Technical Vocabulary of DDI and SDMX Terms Marco Pellegrino Eurostat.

IMS Proof of Concept for Data Capture using Metadata Bryan Fitzpatrick Rapanea Consulting Limited June 2014.

DDI 3.0 Overview Sanda Ionescu, ICPSR. DDI Background Development History 1995 – A grant-funded project initiated and organized by ICPSR proposes to create.

DDI-RDF Discovery Vocabulary A Metadata Vocabulary for Documenting Research and Survey Data Linked Data on the Web (LDOW 2013) Thomas Bosch.

4 April 2007METIS Work Session1 Metadata Standards and Their Support of Data Management Needs Daniel W. Gillman Bureau of Labor Statistics Paul Johanis.

CountryData Technologies for Data Exchange SDMX Information Model: An Introduction.

Metadata Portal Project: Using DDI to Enhance Data Access and Dissemination Mary Vardigan Assistant Director, ICPSR Director, DDI Alliance.

SDMX Standards Relationships to ISO/IEC 11179/CMR Arofan Gregory Chris Nelson Joint UNECE/Eurostat/OECD workshop on statistical metadata (METIS): Geneva.

Describing Statistical registers in SDMX and DDI: A Comparison Arofan Gregory Metadata Technology Eurostat, June 4-6, 2013 Luxembourg.

DDI-RDF Leveraging the DDI Model for the Linked Data Web.

Environment Change Information Request Change Definition has subtype of Business Case based upon ConceptPopulation Gives context for Statistical Program.

Evolution of Data Documentation Providing Social Science Data Services Jim Jacobs, 2008.

Session: General Statistical Business Process Model (GSBPM)

DDI and the Lifecycle of Longitudinal Surveys Larry Hoyle, IPSR, Univ. of Kansas Joachim Wackerow, GESIS - Leibniz Institute for the Social Sciences.

ILO Department of Statistics Edgardo Greising

Survey Data Management and the Combined Use of DDI and SDMX Arofan Gregory Chris Nelson Metadata Technology Eurostat, June

Eurostat SDMX and Global Standardisation Marco Pellegrino Eurostat, Statistical Office of the European Union Bangkok,

SDMX IT Tools Introduction

Statistical Metadata Extensions to the X3.285 Metamodel By Daniel W. Gillman Chairman, NCITS/L8 U.S. Bureau of the Census.

TIC Updates EDDI 2010 Wendy Thomas – 6 Dec Schedule and Process Changes Production schedule is moving to: – Summer / Winter release schedule January.

SDMX Basics course, March 2016 Eurostat SDMX Basics course, March Introducing the Roadmap Marco Pellegrino Eurostat Unit B5: “Data and.

Writing a HOWTO Guide for DDI An approach for getting started.

>> Metadata What is it, and what could it be? EU Twinning Project Activity E.2 26 May 2013.

Metadata requirements for archiving structured data Alice Born Statistics Canada Joint UNECE/Eurostat/OECD Work Session on Statistical Metadata (9-11 April.

Metadata models to support the statistical cycle: IMDB

DDI and GSIM – Impacts, Context, and Future Possibilities

Taking Advantage of DDI 3.0

Topic 2 (ii) Metadata concepts, standards, models and registries

End-to-End Management of the Statistical Process An Initiative by ABS

ESCWA SDMX Workshop Session: Constraints.

Session: Metadata Repository

SDMX Information Model: An Introduction

The problem we are trying to solve

Question Banks, Reusability, and DDI 3.2 (Use Parameters)

Arofan Gregory METIS October 5-7, 2011

SISAI STATISTICAL INFORMATION SYSTEMS ARCHITECTURE AND INTEGRATION

Presentation to SISAI Luxembourg, 12 June 2012

The role of metadata in census data dissemination

DDI and GSIM – Impacts, Context, and Future Possibilities

Presentation transcript:

ESCWA SDMX Workshop Session: Role in the Statistical Lifecycle and Relationship with DDI (Data Documentation Initiative)

© Metadata Technology General Statistical Business Process Model and SDMX

© Metadata Technology General Statistical Process Business Model and SDMX DDI SDMX

© Metadata Technology General Statistical Process Model and SDMX DDI SDMX

© Metadata Technology DDI 3.0 and the Survey Life Cycle A survey is not a static process: It dynamically evolved across time and involves many agencies/individuals DDI 2.x is about archiving, DDI 3.0 across the entire “life cycle” 3.0 focus on metadata reuse (minimizes redundancies/discrepancies, support comparison) Also supports multilingual, grouping, geography, and others 3.0 is extensible

© Metadata Technology DDI 3.0 Modules Conceptual Components (concepts, universes) Data Collection (survey instruments and collection processing) Logical Products (variables, categories, code lists) Physical data product (descriptions of file structures) Physical Instance (instances of data files) Archiving (information about holding, storage, and organizations) Comparative (mapping schemes) Grouping (for comparison, and longitudinal studies, panels, and series) Instance (the wrapper) DDI Profile (describes which DDI 3 elements are used) Study Unit (describes a single study)

© Metadata Technology Technical Specifications - Maintainable Schemes (that’s with an ‘e’ not an ‘a’) Category Scheme Code Scheme Concept Scheme Control Construct Scheme GeographicStructureScheme GeographicLocationScheme InterviewerInstructionScheme Question Scheme NCubeScheme Organization Scheme Physical Structure Scheme Record Layout Scheme Universe Scheme Variable Scheme Packages of reusable metadata maintained by a single agency

© Metadata Technology DDI “Mind Map”

© Metadata Technology DDI – “Mind Map”

© Metadata Technology Technical Specifications – XML Schemas archive comparative conceptualcomponent datacollection dataset dcelements DDIprofile ddi-xhtml11 ddi-xhtml11-model-1 ddi-xhtml11-modules-1 group inline_ncube_recordlayout instance logicalproduct ncube_recordlayout physicaldataproduct physicalinstance proprietary_record_layout (beta) reusable simpledc studyunit tabular_ncube_recordlayout xml set of xml schemas to support xhtml

© Metadata Technology Use within data collection Concepts Universe Methods Purpose People/Orgs Questions Instrument Data Collection Data Processing Funding Revisions Submitted Proposal $ € £ Presentations Archive/ Repository Publication Variables Physical Stores Principal Investigator Collaborators Research Staff Data

© Metadata Technology DDI and Generic Process Example Survey/Register Raw Data Set Anonymization, cleaning, recoding, etc. Micro-Data Set/ Public Use Files Tabulation, processing, case selection, etc. Aggregation,harmonization Aggregation,harmonization Aggregate Data Set (Lower level) Aggregate Data Set (Higher Level) DDI SDMX Indicators

© Metadata Technology Concept Scheme Code List, HCL, SDMX Registry SDMX DSD and data set SDMX MSD and metadata set SDMX General Statistical Process Model and SDMX

SDMX SDMX Process Model

© Metadata Technology Process - Schematic Process Computation Transition Process Step Process Artefact inputs outputs target

© Metadata Technology Process - UML