The Role of Metadata in Census Data Dissemination United Nations Regional Seminar on Census Data Dissemination and Spatial Analysis Amman, Jordan 16-19 May 2011 Background When talking about strategies for data dissemination, we cannot avoide the subject of MD. It is critical for dissemination and corrct use of data. -Also, as the exchange of statistical information grows, there are stronger calls for comprehensive and accessible metadata systems. -

Outline What is metadata? What does metadata do? Types of metadata Metadata in census data dissemination Metadata standards: current situation Conclusion

What is metadata? It is structured information or documentation about data which: informs users about the content, quality and condition of data; describes the structure of datasets, explains, locates, or makes it easier to retrieve, use, or manage data; provides information on the processes of data production; guides on proper usage or interpretation of data. “Data that define and describe other data” (ISO definition) “Information about information” (Dion, 2006) What is metadata? There are many statements about MD, but no single agreed conceptual definition. - MD is structured information (or documentation) about statistical data which: -informs users about the content, quality and condition of the data. -describes the structure of dataset, explains, locates, or otherwise makes it easier to retrieve, use, or manage an information resource. -provides information on the processes of production -guides on proper usage or interpretation of data, and is instrumental in transforming data into meaningful information. Putting it simply.... Organization for Standardization (ISO) which defines metadata as “data that defines and describes other data” 3

What does metadata do? Assist in retrieving and processing data Support correct use of data Provide transparency in data Enhance interoperability Improve archiving, preservation, institutional memory over time Statistical metadata systems have several basic roles, some of which support end-users and others support statistical production. 1) In a computerized environment, metadata assists in retrieving and processing data in various statistical applications. 2) Support correct use of data , because Good documentation reduces the likelihood of misuse of data. 3) Comments on data quality, coverage, difficulties in the process of data collection, any deviations from recommended definitions/classifications enhance transparency in data. 4) Enhance interoperability Describing a data resource with structured metadata allows to exchange data with minimal loss of content and functionality. 5) Archiving, preservation, institutional memory Metadata ensures that data resources will survive and continue to be accessible into the future. THINK that dDigitally available information may become unusable in the future as hardware and software technologies change. A well-structured metadata system however can help in migrating data properly across various technologies, thus preserving data. 4

Types of metadata Structural metadata provide information about the structure of the dataset act as identifiers and descriptors of the data, making it possible to properly identify, retrieve and browse the data Reference metadata allow a thorough understanding and interpretation of the corresponding statistical data Describe the concepts, definitions, methodology and quality of data; production and dissemination process, data access conditions, release policy, confidentiality, etc Broadly speaking..... There are two types of metadata: Structural metadata – Structural metadata are those metadata which provide information about the structure of the dataset. They act as identifiers and descriptors of the data structure, making it possible to properly identify, retrieve and browse the data. Reference metadata -allow a thorough understanding and interpretation of the corresponding statistical data - Reference metadata (also known as explanatory metadata) are generally in a textual format and describe the content of the data, including the concepts, methodology and quality, and so on.. 5

Metadata in Census Data Dissemination All tabulations of census data should include: Census questions Reason why the questions are asked Conceptual definitions Geographic hierarchies used Changes since the previous census, regarding content, operational methods or geographic boundaries Quality indicators (ex. coverage rates and item non-response) Methodological note on the rules and methods applied If a long-form sample is used... Sampling design, size, sampling variability of the results (P&R, 2008) Metadata is a key element of census dissemination to ensure that the underlying concepts on the issues are well understood and that the results are properly interpreted. According to the P&R.....

Metadata standards: current situation 　Currently, data dissemination and exchange take place in ad-hoc manner, using all kinds of non-standard format. Common standards and guidelines are needed to enable more efficient exchange of statistical data. Standard metadata system will: ensures consistency and comparability of content avoids duplication and diversity of definitions ensures reduction in cost of data development - Currently, data dissemination and exchange take place in ad-hoc manner, using all kinds of non-standard format. Common standards and guidelines are needed to enable more efficient exchange of statistical data. - We should realize that considerable gains from adopting common approaches to metadata management. Standard metadata system will: ensures consistency and comparability of content avoids duplication and diversity of definitions ensures reduction in cost of data development 7

Metadata standards (cont’d) Two international metadata standards are becoming well established: - SDMX (Statistical Data and Metadata Exchange) a number of international agencies have endorsed SDMX; supported by the UN Statistical Commission -DDI (Data Dissemination Initiative) Microdata Management Toolkit of WB uses the DDI metadata standard At the international level, many different metadata schems are being developed. Among these, Two international metadata standared for data exchanges. １．SDMX (Statistical Data and Metadata Exchange ２．DDI (Data Dissemination Initiative ) - a number of international agencies, including IMF, OECD, FAO, have moved to use SDMX for standerdized data exchange. 8DSMX ensures MD always come along with the data, making the information immediately understandable, - aggregate level?) - IHSN (International Household Survey Network) of WB offers developing countries a *Macrodata Management toolkit”, in which DDI metadata standared is used. (microdata documentation – unit base??) 8

Conclusion Metadata are at the heart of the information management. They should be an integral part of statistical dissemination strategies. Standards for metadata management are important to develop. Regional and international collaboration between NSOs is an important consideration 9

Thank you !

Users of metadata Metadata support the knowledge of potential user of statistical information. The major users include: Users of statistical data Producers of statistical data Researchers on the development of statistical systems Who needs metadata? -Users and producers of statistical data are obvious users of statistical metadata. -Users of statistical data: The user of statistical data will need some metadata to analyze and interpret the statistical data. The level of detail of the metadata needed will vary depending on the type of the user. Different categories of users have different needs and requirements for the level of detail. As a result statistical services will have to offer a wide range of metadata to suit different users. - Producers of statistical data: A broad range of people involved in the production of statistical data and information products including -- designers of data collections programs, subject matter statisticians, statistical methodologists, and information syst em specialists as well as providers of input data and respondents of surveys – all need metadata in their work. -Researchers on the development and methodology of statistical systems also need metadata 11

Points for discussion - What are the success factors of metadata? Strategies? Infrastrucure? Resources? - How to ensure the security of metadata? - How “open” or “transparent” the metadata can be? - How to improve standerdization of metadata?