1. Ontology Learning and Population from Text: Algorithms, Evaluation and Applications
Chapters 1 - 5
Presented by Sole

2. Introduction Artificial intelligence
Build systems that incorporate knowledge about a domain to reason on the basis of this knowledge and solve problems not encountered before
Include explicit and symbolic representation of knowledge about a domain
Symbolic representation and procedural aspects are separated so that it can be reused across systems
Example for Yoyo the cat
Which symbols to use and what they stand for?

3. Introduction Ontology
Defines what is important in a domain and how concepts are related
Knowledge-based system: determine which symbols are needed and how they are interpreted
Logical level: interpretation can be constraint according to the ontology by axiomatizing symbols
Issues
Costly to construct
Time-consuming
Significant coverage of domain is needed
Meaning and consistent generalization are required
The trade-off between modeling a large amount of knowledge and providing as many abstractions as possible to keep the model concise makes ontology engineering indeed a challenging
Knowledge Acquisition Bottleneck

4. Introduction Solution Automatically learn ontologies from data
Goal: bridging the gap between
World of symbols (words used in natural language)
World of concepts (abstractions of human thought)
Challenge
Correctness and consistency of the model can not be guaranteed
Human post-processing definitely necessary
Automatically learned ontologies need to be inspected, validated, and modified by humans before they can be applied for applications relying on logical reasoning
Rest of book will be about explaining in more details ontology learning

5. Ontologies Definition Philosophical discipline Computer Science
Science of existence or the study of being
Computer Science
Formal specifications of a conceptualization
Resources representing the conceptual model underlying a certain domain, describing it in a declarative fashion and thus cleanly separating it from procedural aspects

6. Ontologies
Example

7. Learning from Text Ontology learning Acquire a domain model from data
Lifting : XML-DTDs, UML diagrams, databases
Semi-structured sources: HTML, XML
Unstructured sources: ontology learning from text
The task is inherently complex and challenging mainly due to two reasons.
* There is typically only a small part of the authors' domain knowledge involved in the creation process, such that the process of reverse engineering can, at best, only partially reconstruct the
authors' model.
*World knowledge – unless we are considering a text book or dictionary - is rarely mentioned explicitly.

8. Learning from Text Meaning triangle
Every language has symbols that evoke a concept that refers to a concrete individual in the world

9. Learning from Text Ontology population Learning concepts and relations
Knowledge markup or annotation: select text fragments and assign them to an ontological concept
Applications
Several methods have been developed in recent years
Challenge
No consensus within ontology learning community on concrete tasks for ontology learning
Comparison between approaches is difficult

10. Learning from Text
Ontology learning tasks (layer cake)

11. Learning from Text Terms:
Task: find a set of relevant concepts and relations
E.g., words, multi-word compounds
State-of-the-art
IR methods
NLP methods: POS tagger, statistical approaches
Linguistic realization of domain specific concepts (keywords in a domain)
Input: text
Output: concepts

12. Learning from Text Synonyms:
Task: find words which denote the same concept
E.g., synsets on WordNet
State-of-the-art
Semantically-similar words
Sense disambiguation and synonym discovery
Latent Semantic Indexing (LSI)
Statistical information measures defined over the Web to detect synonyms

13. Learning from Text Concepts:
Task: find intentional definitions of concept, their extension, and lexical signs used to refer to them
State-of-the-art
Clusters of related terms
LSI-based techniques
Discovery of hierarchies of named entities
Know-it-all system
OntoLearn system
Intension: natural language description of concept
The Know-It-All system [Etzioni et al., 2004a] also aims at learning
the extension of given concepts, such as, for example, all the actors appearing
on the Web. In the approach of Evans [Evans, 2003], the concepts as
well as their extensions are thus derived automatically, while Etzioni et al.
[Etzioni et al., 2004a] essentially learn the extension of existing concepts.
Finally, other systems learn concepts intentionally. The OntoLearn system
[Velardi et al., 2005], for example, derives WordNet-like glosses for domain specific
concepts on the basis of a compositional interpretation of the meaning
of compounds.

14. Learning from Text Hierarchies:
Task: concept hierarchy induction, refinement and lexical extension
State-of-the-art
Lexico-syntactic patterns
Clustering algorithm to automatically derive concept hierarchies
Analysis of term co-occurrence in same sentence/document

15. Learning from Text Relations:
Task: learn relations identifiers or labels as well as their appropriate domain and range
State-of-the-art
Association rules
Syntactic-dependencies
Very few approaches address the issue of learning ontology relations from text

16. Learning from Text Axiom schemata instantiations: General axioms
Task: learn which concepts, relations, or pair of concepts the axioms in a given system apply to
General axioms
Task: derive more complex relationships and connections between concepts and relations
Logical interpretations constraining the interpretation of concepts and relations

17. Learning from Text Population:
Task: learn instances of concepts and relations
State-of-the-art
Associated to well-known tasks for which a variety of approaches have been developed
Information extraction
Named entity recognition

18. Basics Natural Language Processing
Basic formalisms and techniques necessary for understanding the rest of the book
NLP-> “green car” green is the color value to describe car
Co-references John Adams J. Adams

19. Syntactic analysis: parsing
Basics
NLP
Pre-processing steps
Chunking, also called shallow or partial parsing, applies shallow processing
techniques (typically regular expressions and finite automata) to group together
words to larger syntactic and meaning-bearing constituents, typically
with a head which is modified by other words in the unit.
Chunking
Syntactic analysis: parsing

20. Syntactic dependencies
Basics
NLP
Pre-processing
The museum houses an impressive collection of medieval and modern art. The building combines geometric abstraction with classical references that allude to the Roman influence on the region.
Bank
River
Financial
Institution
Contextual features
Syntactic dependencies

21. Basics Similarity measures NLP
Context is often represented as vector in high dimensional space E", the dimensions
corresponding to words found in the context of the word in question.
This vector-based context representation constitutes the core of the so called
vector space model used in information retrieval

22. Basics Similarity measures Binary similarity measures
NLP
Similarity measures
Binary similarity measures
Geometric similarity measures

23. Basics Similarity measures Measures based on probability distribution
NLP
Similarity measures
Measures based on probability distribution
Hypothesis testing
When assessing the degree of association between words, the HQ hypothesis
assumes that the probability of the two words is independent of each
other, i.e. Piwi,W2) = Piwi) P{w2)
The independence hypothesis is rejected in case the observed probability is
found to significantly differ from P{wi,W2) as defined above.

24. Basics Term relevance Weight the importance of a term in a document
NLP
Term relevance
Weight the importance of a term in a document

25. Basics
NLP
WordNet
Lexical database for the English language

26. Basics Formal concept analysis Formal objects: concepts +
Formal attributes: characteristics describing objects
Incidence relation: information about which attributes hold for each object =
Formal context

27. Basics
FCA
Example

28. Basics
FCA
Example
Mention that there exist algorithms that can be used to create FC

29. Basics Machine learning
Automatic recognition/detection of patterns and regularities within sample data
Patterns can be used to understand/describe the data or to make predictions
Learning process
Supervised
Predicts the appropriate category for an example from a set of categories represented by a set of labels
Unsupervised
Search for common and frequent structures within the data (data exploration)

30. Basics Supervised learning Regression Classification ML
Numeric prediction (labels are continue values)
Classification
Assign proper category to a given example
Target value
Feature vector

31. Basics Classifiers Tools Bayesian Classifiers Decision TreesML
Classifiers
Bayesian Classifiers
Decision Trees
Instance-Based Learning
Support Vector Machines
Artificial Neural Networks
Tools
WEKA
RapidMiner

32. Basics ML
Examples

33. Basics Unsupervised learningML
Unsupervised learning
Clustering: find groups of similar objects in data
There is no labeled data to train from
Classification
Hierarchical vs. non-hierarchical
Non-hierarchical algorithms produce a set of groups
Hierarchical algorithms order groups in a tree structure
Hard vs. soft
Hard: elements are assigned to distinct clusters
Soft: elements are assigned to clusters with a certain degree of membership

34. Basics Algorithms K-means Hierarchical clusteringML
Algorithms
K-means
Hierarchical clustering
Hierarchical Agglomerative (Bottom-Up) Clustering
Divisive (Top-Down) Clustering

35. Datasets
Corpus description

36. Datasets
Concept hierarchies