1. Ontology Assessment – Proposed Framework and Methodology

2. Goals & Objectives Framework against which any so-called ontology can be neutrally assessed and characterized Assessment for the purpose of informing users about an ontology Providing developers with methodology for comparison and improvement Additional purpose…. Setting minimum standards for what is/is not an ontology Defining thresholds for formal and informal ontologies

3. Which one is an ontology? Why/not? Biological Classification Scheme AMS Classification Scheme NASA Thesaurus Library of Congress Subject Headings Dublin Core Metadata Scheme Organizational Chart ISO Country List Metadata repository scheme Master Data Repository Content architecture models (OO models) SCORM XML Schema for Directory Records Classification Scheme Records Classification Scheme Social Network Representation Social Network Representation Folksomony Folksomony Domain Knowledge Map Domain Knowledge Map Visual representation of concept clusters Visual representation of concept clusters Financial ratios Economic indicators Mathematical formula XML structured electronic journal issue WordNet

4. Proposed Ontology Assessment Methodology Requires that we suspend our current terminology of things that we call ontologies and adopt a neutral mental model – convince others to do the same Requires that we suspend our current terminology of things that we call ontologies and adopt a neutral mental model – convince others to do the same Suggest that a factor analysis methodology can be applied to ontologies, just as it is applicated to knowledge economies Suggest that a factor analysis methodology can be applied to ontologies, just as it is applicated to knowledge economies Factor analysis involves … Factor analysis involves … 1. defining the essential dimensions of an ontology 2. defining those factors which characterize each dimension 3. quantifying the factors 4. analyzing the factors for any given application (factor analysis) or comparison 5. visually representing the analysis for a single ontology and/or for comparisons of ontologies Let me explain how factor analysis might be used Let me explain how factor analysis might be used

5. Factor Analysis Methodology Methodology is currently used to calculate and visually display factors which Contribute to the development or knowledge economies.

6. Dimensionality of Ontologies Dimensionality proposed in the framework includes: structure, expressiveness, representational granularity, intended use, automated reasoning, descriptive/prescriptive, and design methodology Are these theoretical or practical dimensions? Do they work at an analytical level? Other possible dimensions …. A. A. Concepts – the nature of the content or values that are delivered or accessed through the ontology such as type, granularity, B. B. Relationships – nature, type, extent, specification of relationships, logic associated with relationships C. C. Context – the context for which the ontology was developed and in which it may be used, including knowledge domain, application domain, D. D. Governance – control and management of the concepts, relationships and context exercised by the developer or current user

7. Representation of Ontological Assessments Methodology could be used to generate an ontological factor index for ontologies, and to rank and compare ontologies. ConceptsContext Relationships Governance Another Dimensionality Framework

8. Representation of Ontological Assessments Structure Represetational Granularity Expressiveness Descriptive vs. Prescriptive Dimensionality Suggested in the Framework Paper Intended Use Use of Automated Reasoning Critical Question: Are these dimensions orthogonal, mutually exclusive and clean enough for analysis?

9. Representation of Ontological Assessments Methodology could be used to generate an ontological factor index for ontological things, and to rank and compare ontologies. Concepts Context Relationships Governance Sample assessment of a folksonomy

10. Representation of Ontological Assessments Concepts Context Relationships Governance Sample assessment of a medical disease classification scheme

11. Representation of Ontological Assessments Concepts Context Relationships Governance Sample assessment of an institutional records classification scheme

12. Defining and Quantifying Factors For each component an orthogonal, independent set of factors must be defined For each component an orthogonal, independent set of factors must be defined Factors must be independent of any particular pre-existing ontology (neutral) Factors must be independent of any particular pre-existing ontology (neutral) Each factor must have a quantifiable method of representation that lends itself to scoring, analysis and comparison Each factor must have a quantifiable method of representation that lends itself to scoring, analysis and comparison Factors must have agreed upon definitions, be easily interpreted by people and machines, and be inclusive in their coverage of values/conditions Factors must have agreed upon definitions, be easily interpreted by people and machines, and be inclusive in their coverage of values/conditions To illustrate the idea, selected examples are presented in following slides To illustrate the idea, selected examples are presented in following slides (please note – these were brainstormed with about 1 hour of thinking and sleep… NEEDS IMPROVEMENT AND DEFINITELY NOT RIGOROUS)

13. Selected Examples of Concept Factors Concept types Concept types Data/numbers Data/numbers Calculation/ratios Calculation/ratios Words Words Grammatical fragment Grammatical fragment Logical statement Logical statement Rule expression Rule expression Engineering equations Engineering equations Degree of ambiguity Degree of ambiguity Context sensitivity/insensitivity of definition Context sensitivity/insensitivity of definition Representational form Representational form Usable encoding method Availability of representational specifications (Strings vs. syntax) Degree of conceptualization/ specification Degree of conceptualization/ specification Theoretical to commital What else…? What else…?

14. Selected Examples of Relationship Factors Simple expressive form of relationships Simple expressive form of relationships Grammatical Grammatical Mathematical Mathematical Logical Logical Relationship behavior Relationship behavior Membership dependence Membership dependence Representation or instance Representation or instance Equivalence Equivalence Causal dependence Causal dependence Derivational dependence Derivational dependence Degree of Relationship Validation/Rigor Degree of Relationship Validation/Rigor Fully Subjective Grammatical validation Mathematical validation Logical rigor/validation What else? What else?

15. Selected Examples of Context Factors Knowledge Context Knowledge Context Formal vs. informal knowledge domain Formal vs. informal knowledge domain Application Context Application Context System vs. human application/ consumption System vs. human application/ consumption Managed/standardized application vs. home grown Managed/standardized application vs. home grown Functional context Functional context Search Mathematical or statistical analysis Logical inference Classification Dynamic clustering Metadata representation Concept indexing What else…? What else…?

16. Selected Examples of Governance Factors Standards Availability Standards Availability Published formal vs. guidelines vs. ad hoc concepts Published formal vs. guidelines vs. ad hoc concepts Published formal vs. guidelines vs. ad hoc relationships Published formal vs. guidelines vs. ad hoc relationships Prescriptive vs. Descriptive Governance Prescriptive vs. Descriptive Governance Enforcement of standards Enforcement of standards Design Guidelines Design Guidelines Top-down (model) vs. Bottom-up (empirical) Top-down (model) vs. Bottom-up (empirical) Extensibility Extensibility Degree to which others can add to or extend either the concepts or the relationships Currency Currency Degree to which the concepts and/or relationships represent our current view or knowledge of the context What else…? What else…?

17. Factor Analysis Factor analysis could be conducted: Factor analysis could be conducted: At the component level on that subset of factors At the component level on that subset of factors At the ontology level, across all factors At the ontology level, across all factors Developers or users could determine what the optimal dimensionality was for their particular use Developers or users could determine what the optimal dimensionality was for their particular use Summit members and the Ontology community could identify minimun factor scores that define what is/is not an ontology, and what constitutes a full, formal ontology Summit members and the Ontology community could identify minimun factor scores that define what is/is not an ontology, and what constitutes a full, formal ontology Ultimately, this may provide us with an ecumenical vs. evangelical approach to ontological standards development and assessment Ultimately, this may provide us with an ecumenical vs. evangelical approach to ontological standards development and assessment