1 SC 32/WG 2 Tutorial Metadata Registry Standards July 16, 2007 Bruce Bargmeyer University of California, Berkeley and Lawrence Berkley National Laboratory Tel: JTC1 SC32 N1649

2 Topics F Standards development: OMG, ISO (TC 37 & JTC 1/SC 32), W3C, OASIS u Align, Coordinate, Integrate: Standards, Recommendations, Specifications F Semantics Challenges and Future Directions

Align, Coordinate, Integrate Standards E WG 2 doing OK internally:

Align, Coordinate, Integrate Standards 4 WG 1 WG 2 WG 3 WG 4 SC 32? Clearwater meeting a step forward

5 Align, Coordinate, Integrate Standards/Recommendations/Specifications for Semantic Computing ISO/IEC JTC 1/SC 32 Us er s ISO/IEC Metadata Registries Metadata Registry Terminology Thesaurus Taxonomy Data Standards Ontology Structured Metadata Terminology CONCEPT Referent Refers To Symbolizes Stands For “Rose”, “ClipArt Rose” ISO TC 37 Semantic Web W3C Object Management MOF ODM CWM IMM OMG Node Edge Subject Predicate Object Graph RDF

Standards Development Semantics Management and Semantics Services – Semantic Computing 6 OMG W3C ISO/IEC JTC 1 SC 32 Align, Co-develop, Fast Track, PAS Submission … ISO TC 37

Standards Development Semantics Management and Semantics Services – Semantic Computing 7 OMG W3C ISO/IEC JTC 1 SC 32 Align, integrate, co-develop, Fast Track, PAS Submission … Can we coordinate content? W3C

A Success 8 OMG ISO/IEC JTC 1 SC 32 Some text and figures are identical in the two standards. ISO/IEC OMG ODM ISO/IEC – Common Logic OMG Ontology Definition Metamodel

Standards Development Semantics Management and Semantics Services – Semantic Computing 9 ISO/IEC (Edition 3) ISO/IEC JTC 1 SC 32 Ongoing effort

Standards Development Semantics Management and Semantics Services – Semantic Computing 10 Possible effort E3 proposals OMG RFP - MOF? IMM

Standards Development Semantics Management and Semantics Services – Semantic Computing 11 ISO/IEC (Edition 3) ISO/IEC JTC 1 SC 32 Hopeful? OMG IMM &

Other Possibilities F OASIS ebXML Registry F W3C Semantic Web Deployment WG F TC 37 12

Getting the information that we need, when we need it, without afflicting the excellent minds of humans with toil and drudgery The litany: F Too much or too little, irrelevant, not authoritative, out of date F Unknown quality, not trustable, lacks provenance, no certainty measures F Difficult to find, difficult to access, difficult to use F Meaning not clear, relationship to other information not clear F Data creators do not have the same understanding of the data as end users F Recorded data loses much real world meaning, context, relationships F Much of the meaning of data is buried in the processes used to manipulate the data (e.g., in computer code) F Need improvements in efficiency and effectiveness Every time we solve it, we re-create it. The Ageless Information Problem cf: Data, Information, Knowledge, Wisdom

F Improve traditional data management/data administration u Use stronger semantics management and semantics services capabilities F Enable something new u Semantic computing New Semantics Capabilities Proposed for ISO/IEC MDR (Edition 3)

F Processing that takes “meaning” into account u Makes use of concept systems, e.g., thesauri and/or ontologies u Moves some of the “meaning” of data from computer code to managed semantics F Processing that uses (e.g., reasons across) the relations between things not just computing about the things themselves. F Processing that helps to take people out of the computation, reducing the human toil u Semantics “grounding” for data, data discovery, extraction, mapping, translation, formatting, validation, inferencing, … F Delivering higher-level results that are more helpful for the user’s thought and action Semantic Computing: The Nub of It

In The Epic Information Struggle We Have Made Heroic Progress Files Machine Processing Computer Processing Cards Tape Disk

In structuring data and text -- F Structured Data u Columns on cards & tape (possibly comma separated) u Hierarchical (DBMS) u Network u Table (relational DBMS) u Hierarchy (XML) u Graph (RDF) F Semi-structured text u Nrof, trof, LaTeX … u SGML u HTML u XML In The Epic Information Struggle We Have Made Heroic Progress

In documenting data and text (e.g., semantics management) – F Data Standards u Code sets F (Meta)Data Standards u Data element definitions, valid values, value meanings u Metadata registries (MDR, ISO/IEC 11179) u Other standards as presented at this conference F Concept systems (or KOS) u Glossaries u Dictionaries u Thesauri u Taxonomies u Ontologies u Graphs In The Epic Information Struggle We Have Made Heroic Progress

F Improve data management through use of stronger semantics management u Databases u XML data u Other “traditional” data F Enable new wave of semantic computing u Take meaning of data into account u Process across relations as well as properties u May use reasoning engines, e.g., to draw inferences Semantic Management Proposals for Edition 3

Semantics Improve Data Management/Data Administration Object Class Chemopreventive Agent Property NSCNumber Conceptual Domain Agent Data Element Concept Chemopreventive Agent NSC Number Data Element Chemopreventive Agent Name Value Domain NSC Code Context caCORE Representation Code Classification Schemes caDSRTraining Valid Values Cyclooxygenase Inhibitor Doxercalciferol Eflornithine … Ursodiol Source: Denise Warzel, National Cancer Institute Enterprise Vocabulary Services (EVS) Concepts Unite NCI MDR

Semantic Computing Application: Find and process non-explicit data Analgesic Agent Non-Narcotic Analgesic AcetominophenNonsteroidal Antiinflammatory Drug Analgesic and Antipyretic Datril Anacin-3Tylenol For example… Patient data on drugs contains brand names (e.g. Tylenol, Anacin-3, Datril,…); However, want to study patients taking analgesic agents

A Semantics Application: Specify and compute across Relations, e.g., within a food web in an Arctic ecosystem An organism is connected to another organism for which it is a source of food energy and material by an arrow representing the direction of biomass transfer. Source: (from SPIRE)

Semantics Application: Combine Data, Metadata & Concept Systems NameDatatypeDefinitionUnits IDtext Monitoring Station Identifier not applicable DatedateDateyy-mm-dd Tempnumber Temperature (to 0.1 degree C) degrees Celcius Hgnumber Mercury contamination micrograms per liter IDDateTempHg A B X Inference Search Query: “find water bodies downstream from Fletcher Creek where chemical contamination was over 10 micrograms per liter between December 2001 and March 2003” Data: Metadata: BiologicalRadioactive Contamination leadcadmium mercury Chemical Concept system:

Semantics Application: Use data from systems that record the same facts with different terms F Reduce the human toil of drawing information together and performing analysis.

Challenge: Use data from systems that record the same facts with different terms Common Content OASIS/ebXML Registries Common Content ISO Registries Common Content Ontological Registries Common Content CASE Tool Repositories Common Content UDDI Registries Country Identifier Data Element XML Tag Term Hierarchy Attribute Business Specification Table Column Software Component Registries Common Content Database Catalogs Business Object Dublin Core Registries Common Content Coverage

Data Elements DZ BE CN DK EG FR... ZW ISO 3166 English Name ISO Numeric Code ISO Alpha Code Algeria Belgium China Denmark Egypt France... Zimbabwe Name: Context: Definition: Unique ID: 4572 Value Domain: Maintenance Org. Steward: Classification: Registration Authority: Others ISO 3166 French Name L`Algérie Belgique Chine Danemark Egypte La France... Zimbabwe DZA BEL CHN DNK EGY FRA... ZWE ISO Alpha Code Same Fact, Different Terms Algeria Belgium China Denmark Egypt France... Zimbabwe Name: Country Identifiers Context: Definition: Unique ID: 5769 Conceptual Domain: Maintenance Org.: Steward: Classification: Registration Authority: Others Data Element Concept

Challenge: Draw information together from a broad range of studies, databases, reports, etc.

A semantics application: Information Extraction and Use Segment Classify Associate Normalize Deduplicate Discover patterns Select models Fit parameters Inference Report results Actionable Information Decision Support Extraction Engine (E3) XMDR

Extraction Engines F Find concepts and relations between concepts in text, tables, data, audio, video, … F Produce databases (relational tables, graph structures), and other output F Functions: u Segment – find text snippets (boundaries important) u Classify – determines database field for text segment u Association – which text segments belong together u Normalization – put information into standard form u Deduplication – collapse redundant information

Metadata Registries are Useful Registered semantics F For “training” extraction engines F The “Normalize” function can make use of standard code sets that have mapping between representation forms. F The “Classify” function can interact with pre-established concept systems. Provenance F High precision for proper nouns, less precision (e.g., 70%) for other concepts -> impacts downstream processing, Need to track precision

Challenge: Gain Common Understanding of meaning between Data Creators and Data Users Users Information systems Data Creation Users EEA USGS DoD EPA environ agriculture climate human health industry tourism soil water air textdata environ agriculture climate human health industry tourism soil water air text ambiente agricultura tiempo salud hunano industria turismo tierra agua aero textdata environ agriculture climate human health industry tourism soil water air textdata Others... ambiente agricultura tiempo salud huno industria turismo tierra agua aero textdata A common interpretation of what the data represents

F Vocabulary Management is essential for use of semantic technologies u Define concepts and relationships u Harmonize terminology, resolve conflicts u Collaborate with stakeholders F An approach u Select a domain of interest u Enter core concepts and relationships u Engage community in vocabulary review u Harmonize, validate and vet the vocabulary u Enter metadata describing enterprise data u Link concept system to metadata Practical Vocabulary Management

F For vocabulary repository u Register, harmonize, validate, and vet definitions and relations F To register mappings between multiple vocabularies F To register mappings of concepts to data F To provide semantics services F To register and manage the provenance of data (E3) is part of the infrastructure for semantics and data management. These capabilities are proposed for ISO/IEC Edition 3 Use eXtended MDR Capabilities

F Upside u Collaborative n Supports interaction with community of interest n Shared evolution and dissemination n Enables Review Cycle u Standards-based – don’t lock semantics into proprietary technology u Foundation for strategic data centric applications u Lays the foundation for Ontology-based Information Management u Content is reusable for many purposes F Downside u Managing semantics is HARD WORK - No matter how friendly the tools u Needs integration with other components (E3) Use

F Data management and metadata management must evolve to address more complex data structures (relational, object, hierarchies, graphs) u Query capabilities n More than SQL, XQuery, SPARQL u Discovery mechanisms n More than Google u Access, mining, extraction We need stronger semantics management Some Challenges

F Registering and mapping ontologies F Ontology Evolution F Registering Process Ontologies Metadata Registry Support for

Thank You F Acknowledgements u Karlo Berket, LBNL u Kevin Keck, LBNL u John McCarthy, LBNL u Harold Solbrig, Apelon This material is based upon work supported by the National Science Foundation under Grant No , USEPA and USDOD. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation, USEPA or USDOD. 37 Bruce Bargmeyer Lawrence Berkeley National Laboratory & Berkeley Water Center University of California, Berkeley Tel: