1. A Memory-Based Approach to Semantic Role Labeling Beata Kouchnir Tübingen University 05/07/04

2. Introduction ● Applying Memory-Based Learning to the task of Semantic Role Labeling (using the TiMBL software) ● Data is processed by chunks, except for the target verb chunks, which can contain modals and negation ● Task is split into two modules: – Recognition: identifies the arguments of a target verb – Labeling: assigns a semantic role to each argument 1Beata Kouchnir05/07/04

3. Memory-Based Learning ● Training instances are stored without abstraction ● Test instances are assigned the most frequent class within a set of k most similar examples (k-nearest neighbors) ● Similarity is computed based on a distance metric: – Overlap: 1 if two values are the same, 0 otherwise (for symbolic values) – Modified value difference: determines similarity based on co-occurrence of values with classes 2Beata Kouchnir05/07/04

4. Recognition Features ● Head word and POS of the focus element (head is last word of chunk) ● Chunk type: one of the 12 chunks types ● Position in clause: beginning, end or inside. ● Directionality with respect to target verb: before, after, coincides ● Numerical distance (1.. n) to the target verb. ● Adjacency to target chunk: adjacent, not adjacent, inside the target chunk 3Beata Kouchnir05/07/04

5. Recognition Features (contd.) ● Target verb and voice: passive if target verb is a past participle preceded by a form of to be ● Context: the features head word, part of speech, chunk type and adjacency of the three chunks each to the left and right of the focus chunk 4Beata Kouchnir05/07/04

6. Labeling Features ● Word, POS and chunk sequence of the head words of all the chunks in the argument; each sequence represents one value ● Clause information: is argument a complete clause? ● Length of the argument in chunks ● Directionality, adjacency, target verb and voice ● Prop Bank roleset of target verb's first sense (86% of targets use first sense) 5Beata Kouchnir05/07/04

7. Evaluation ● Recognition module: Prec. 53.21%, Rec. 74.97%, F 62.25 – All features improved performance; MVDM, k=7 best parameter setting ● Labeling module: Prec. 75.71%, Rec. 74.60%, F 75.15 – POS-sequence and length worsen performance; MVDM, k=1 best parameter setting ● Overall development: Prec. 44.93%, Rec. 63.12%, F 52.50 ● Overall test: Prec. 56.86%, Rec. 49.95%, F 53.18 6Beata Kouchnir05/07/04

8. Conclusion and Future Work ● Recognizing arguments is more difficult than labeling – Removing multiple A0-A5 arguments can increase precision – IOB2 might not be the best representation ● Some chunkers can recognize recursive noun phrases – Could improve results without adding too much comlexity ● Changing classifier's default parameters considerably improves performance 7Beata Kouchnir05/07/04