Hierarchical, Perceptron-like Learning for OBIE Yaoyong Li, Kalina Bontcheva Department of Computer Science University of Sheffield {yaoyong,kalina}@dcs.shef.ac.uk http://gate.ac.uk/ http://nlp.shef.ac.uk/

Outline Ontology based information extraction (OBIE) and Semantic Web Perceptron-like algorithm for OBIE Experimental results 2(17)

Semantic Annotation Most material in Web involve textual language. Annotating text according to an ontology is an important aspect of Semantic Web. Automatic annotating is desirable. 3(17)

Ontology Based Information Extraction Information extraction (IE): extract the pre-defined information in text, automatically. Ontology based IE (OBIE): given some text, identify the mentions of the concepts of ontology. 4(17)

Machine Learning (ML) for OBIE OBIE systems based on ML Magpie, SemTag: match text with instances in ontology. C-PANKOW: unsupervised learning. KIM: rule based, human-engineered, using ontology structure during pattern matching. Few OBIE systems explored the ontology structure. 5(17)

Hierarchical Classification (HC) Class labels are organised in a hierarchical fashion. Learning algorithm takes into account the relations among labels. Thus it can achieve better performance than the flat learning algorithm for HC problem. 6(17)

Adapting Hieron for OBIE Hieron is an effective and efficient learning algorithm for HC, proposed in Dekel etc. 2004. OBIE is somehow different from HC. IE vs. classification. Ontology vs. taxonomy. Adapting Hieron for OBIE. 7(17)

Hieron Learning Learn a Perceptron classifier for each concept. The difference between two classifiers is proportional to the cost of misclassifying one concept as another. Given one training example, update the Perceptrons along the path from the true concept to the predicted one. 8(17)

Our Modification on Hieron Added a regularisation parameter for learning. So it will stop after a finite learning loops on any training data. 9(17)

Adaptation to OBIE Learn two Hierons, one for start tokens of information entities, another for end tokens. Add one concept into ontology, representing the non-class. More than one path between two concepts: select the shortest path during training. 10(17)

Ontology Sensitive F-measure Cost of misclassifying an example of concept A as another concept B: ecost(A, B) Overall accuracy An for n entities: Sum of n accuracies (1- ecost(Ai, Bi)) precision = An/(An+Nspurious) recall = An/(An+Nmissing) F1 = 2*precision*recall/(precision+recall) 11(17)

Experimental Dataset Sekt ontology-annotated news corpus Consist of 290 news articles, divided into three themes: business, international and UK politics. Manually annotated according to the Proton ontology: 146 concepts were used for annotation. Created within the EU project Sekt. Pre-processed the corpus using ANNIE Obtained the domain-independent linguistic features, such as token’s form, lemma, simple types, POS, named entity types. 12(17)

Experimental Results (1) PAUM SVM Hieron Business 0.741 0.753 0.827 Int.-politics 0.771 0.801 0.833 UK-politics 0.820 0.829 0.825 Conventional F1 13(17)

Experimental Results (2) PAUM SVM Hieron Business 0.788 0.793 0.912 Int.-politics 0.830 0.859 0.913 UK-politics 0.836 0.844 0.901 Ontology Based F1 14(17)

Experimental Results (3) Computational time PAUM SVM Hieron Training 552s 11450s 3815s Application 33s 111s 109s 15(17)

Regularization Parameter for Hieron Single loop 300 loops Regularisation F1 0.798 0.813 0.825 Ontology F1 0.890 0.893 0.901 Training time 510s 54173s 3815s 16(17)

Conclusions Explore the structure of ontology in semantic annotation. The Hieron, after adaptation, performed well for OBIE. Future research: Use other cost measures instead of distance. SVM based learning algorithms. 17(17)