1. Hierarchical, Perceptron-like Learning for OBIE
Yaoyong Li, Kalina Bontcheva
Department of Computer Science
University of Sheffield

2. Outline Ontology based information extraction (OBIE) and Semantic Web
Perceptron-like algorithm for OBIE
Experimental results
2(17)

3. Semantic Annotation Most material in Web involve textual language.
Annotating text according to an ontology is an important aspect of Semantic Web.
Automatic annotating is desirable.
3(17)

4. Ontology Based Information Extraction
Information extraction (IE): extract the pre-defined information in text, automatically.
Ontology based IE (OBIE): given some text, identify the mentions of the concepts of ontology.
4(17)

5. Machine Learning (ML) for OBIE
OBIE systems based on ML
Magpie, SemTag: match text with instances in ontology.
C-PANKOW: unsupervised learning.
KIM: rule based, human-engineered, using ontology structure during pattern matching.
Few OBIE systems explored the ontology structure.
5(17)

6. Hierarchical Classification (HC)
Class labels are organised in a hierarchical fashion.
Learning algorithm takes into account the relations among labels.
Thus it can achieve better performance than the flat learning algorithm for HC problem.
6(17)

7. Adapting Hieron for OBIE
Hieron is an effective and efficient learning algorithm for HC, proposed in Dekel etc
OBIE is somehow different from HC.
IE vs. classification.
Ontology vs. taxonomy.
Adapting Hieron for OBIE.
7(17)

8. Hieron Learning Learn a Perceptron classifier for each concept.
The difference between two classifiers is proportional to the cost of misclassifying one concept as another.
Given one training example, update the Perceptrons along the path from the true concept to the predicted one.
8(17)

9. Our Modification on Hieron
Added a regularisation parameter for learning.
So it will stop after a finite learning loops on any training data.
9(17)

10. Adaptation to OBIE
Learn two Hierons, one for start tokens of information entities, another for end tokens.
Add one concept into ontology, representing the non-class.
More than one path between two concepts: select the shortest path during training.
10(17)

11. Ontology Sensitive F-measure
Cost of misclassifying an example of concept A as another concept B: ecost(A, B)
Overall accuracy An for n entities: Sum of n accuracies (1- ecost(Ai, Bi))
precision = An/(An+Nspurious)
recall = An/(An+Nmissing)
F1 = 2*precision*recall/(precision+recall)
11(17)

12. Experimental Dataset Sekt ontology-annotated news corpus
Consist of 290 news articles, divided into three themes: business, international and UK politics.
Manually annotated according to the Proton ontology: 146 concepts were used for annotation.
Created within the EU project Sekt.
Pre-processed the corpus using ANNIE
Obtained the domain-independent linguistic features, such as token’s form, lemma, simple types, POS, named entity types.
12(17)

13. Experimental Results (1)
PAUM
SVM
Hieron
Business
0.741
0.753
0.827
Int.-politics
0.771
0.801
0.833
UK-politics
0.820
0.829
0.825
Conventional F1
13(17)

14. Experimental Results (2)
PAUM
SVM
Hieron
Business
0.788
0.793
0.912
Int.-politics
0.830
0.859
0.913
UK-politics
0.836
0.844
0.901
Ontology Based F1
14(17)

15. Experimental Results (3)
Computational time
PAUM
SVM
Hieron
Training
552s
11450s
3815s
Application
33s
111s
109s
15(17)

16. Regularization Parameter for Hieron
Single loop
300 loops
Regularisation F1
0.798
0.813
0.825
Ontology F1
0.890
0.893
0.901
Training time
510s
54173s
3815s
16(17)

17. Conclusions Explore the structure of ontology in semantic annotation.
The Hieron, after adaptation, performed well for OBIE.
Future research:
Use other cost measures instead of distance.
SVM based learning algorithms.
17(17)