SVM Based Learning System for F-term Patent Classification Yaoyong Li, Kalina Bontcheva, Hamish Cunningham Department of Computer Science University of Sheffield {yaoyong,kalina,hamish}@dcs.shef.ac.uk http://gate.ac.uk/ http://nlp.shef.ac.uk/

SVM for F-term Classification Experiments with different contents of patent. Adapt the SVM for F-term patent classification sub-task. Difference between normal document classification and F-term patent classification. Hierarchical SVM for the F-term taxonomy. 2(10)

Using Different Parts of Patent Results using different contents of patent. A-Precision R-Precision F-measure Abstract only 0.4279 0.3908 0.3647 Full text of patent 0.4688 0.4270 0.3998 Full text + F-term description 0.4779 0.4363 0.4125 3(10)

Evaluation of F-term classification Document classification, based on category Learn a classifier for one category, and evaluate it on new documents. F-term patent classification: based on patent Compare the scores of all classifiers for one patent. So need normalise the scores of different classifiers for fair comparison. 4(10)

Benefit of Normalisation Comparison between the results with and without normalisation. A-Precision R-Precision F-measure Without normalisation 0.4643 0.4330 0.3677 With normalisation 0.4779 0.4363 0.4125 5(10)

Hierarchical SVM Flat SVM Hierarchical SVM for taxonomy Learn an SVM classifier for one class using the one vs. all others approach. Ignore the relations between classes. Hierarchical SVM for taxonomy Learn an SVM classifier for one class by using only the training examples which are the positive examples of the parent class. 6(10)

Comparison of H-SVM and Flat SVM Using the conventional measure. A-Precision R-Precision F-measure Flat SVM 0.4779 0.4363 0.4125 H-SVM 0.2376 0.2164 0.2257 7(10)

Comparison of H-SVM and Flat SVM Using the measures taking into account the relations between F-terms. A-Precision R-Precision F-measure Flat SVM 0.6269 0.6194 0.4429 H-SVM 0.5193 0.5414 0.3605 8(10)

Possible Reasons of Failure of H-SVM F-term classification is a multi-label problem, One instance may have more than one true class. But the H-SVM we used was designed for the case that each instance has only one true class. F-terms under a given theme are not hierarchically related with each other in the strict sense. 9(10)

Conclusions Adapted successfully the SVM for the F-term patent classification subtask. Demonstrated that more information are helpful. Analysis of the failure of H-SVM How to adapt the H-SVM? 10(10)