Integrated metadata systems History Status Vision Roadmap

Integrated Metadata Systems  Stove-piped statistical production (systems) with no, or at the best, encapsulated metainformation, represents our remains from the IT stone-age.  First steps towards the consciousness of metadata(structures) were taken some 20 years ago: metadatadriven on-line systems file description- and other archives of structured documentation  The technological evolution has been the driving force towards a vision of a coherent statistical (IT) system  However, the state-of-the-art-technology has also at all times represented one of the most important obstacles to success  in addition to the human- and organisational barriers that we also discuss

Technological barriers  Lack of processor speed and data storage capacity  lack of access possibilities across different IT systems  Lack of database functionality and flexibility  Lack of awareness of metainformation as a whole in the IT industry (handling of technical meta- information at the most), i.e the kind of metainformation that was handled in the first datawarehouse solutions  Lack of (IT) standards,  but anyhow; why didn’t we achieve more when we had all our information systems within one mainframe ?  due to the human and organisational barriers ?

Our current advantages  WWW  Open standards on connectivity  LAN/WAN communication  database connectivity standardised exchange of data on  protocol level  syntactic level  Object orientation  Web services ! But what about the semantic level ?

A vision for a coherent statistical system  The basic architecture of a coherent statistical system is formed by the structure, content and handling of metainformation  The IT system will never reflect anything else but the level of standardisation and coordination of the statistical production within the organisation  NSI’s must take into account all statistical IT systems currently running, having been developed over the past 20 years, which would need to fit into a new or upgraded system  A coherent statistical system based on integration of what you already have, or convert everything to a new (gigantic) system ?

Objective Content Design and planning Population Sample Collection methods Process methods Dissemi- nation Evaluation Operation Establish population & sample Data collect- ion & Edit Presentation Dissemination Estimation Aggregation Expert knowledge: -Guidelines -Articles -Methods -People Input data Input data Stat. data Knowledge base Local metadata Global metadata Classifications Standards -Datadoc -Stat.Activities -Stat.doc -Quality decl. -Structured metadata Datawarehouse Populations Local prod.data Observation register Local prod.data Dissemination database Source: Bo Sundgren A vision for a coherent statistical system Know- ledge

Metadata File descript. Classifications Macro database Variable definitions Local metadata Question- naire repository Content (Quality) declaration Statistical activities Local metadata Local metadata

Metadata File descript. Classifications Macro database Variable definitions Local metadata Question- naire repository Content (Quality) declaration Statistical activities Local metadata Local metadata Census/ Survey

Metadata File descript. Classifications Macro database Variable definitions Local metadata Question- naire repository Content (Quality) declaration Statistical activities Local metadata Local metadata What information is needed to establish consistent links between the components of your (structured) metainformation system ?

Metadata File descript. Classifications Macro database Variable definitions Local metadata Question- naire repository Content (Quality) declaration Statistical activities Local metadata Local metadata

Metadata File descript. Classifications Macro database Variable definitions Local metadata Question- naire repository Content (Quality) declaration Statistical activities Local metadata Local metadata XML

Metadata components File descript. Classifications Macro database Variable definitions Local metadata Question- naire repository Content (Quality) declaration Statistical activities Local metadata Local metadata

Metadata components File descript. Classifications Macro database Variable definitions Local metadata Question- naire repository Content (Quality) declaration Statistical activities Local metadata Local metadata Linking/Mapping Metamodel Metadata Data Three layered model

Metadata components File descript. Classifications Macro database Variable definitions Local metadata Question- naire repository Content (Quality) declaration Statistical activities Local metadata Local metadata Linking/Mapping Collection Aggregation Estimation Data Editing Dissemination Process

Metadata components File descript. Classifications Macro database Variable definitions Local metadata Question- naire repository Content (Quality) declaration Statistical activities Local metadata Local metadata Linking/Mapping Different domains Domain 2 Domain 1 Domain n

Metadata components File descript. Classifications Macro database Variable definitions Local metadata Question- naire repository Content (Quality) declaration Statistical activities Local metadata Local metadata Access End user needs

Non-structured metainformation  Text  Text-mining  Knowledge systems  Challenge, and upcoming reality: How shall we be able to store, retrieve and maintain the knowledge of the organisation much more independent of their (shifting) staff ?

Metadata in the statistical production  Data input  Data throughput  Data dissemination

I Data collection BS CRDS NSI OCR ELQ Metadata P Internal Business Systems Mapping between statistical and in-house data definitions Electronic Questionnaires Paper Questionnaires Subject matter systems Optical char. recognition, intrepretation verifiying Data Definitions Questions Rules/Checks Questionnaires Central Raw Data Storage XML Questionnaire generation Links to a (national) repository of Data definitions/Questionnaires Linked to Business Register