1. SEMANTIC ROLE LABELING BY TAGGING SYNTACTIC CHUNKS
Kadri Hacioglu1, Sameer Pradhan1, Wayne Ward1
James H. Martin1, Daniel Jurafsky2
1 The Center for Spoken Language Research
2 Stanford NLP Group
Stanford University
University of Colorado at Boulder

2. OUTLINE Semantic Role Labeling (SRL) Nature of Shared Task Data
Our Strategy
System Description & Features
Experiments
Concluding Remarks

3. SEMANTIC ROLE LABELING
Based on predicate-argument structure:
First explored by (Gildea & Jurafsky, 2000)
Predicate: pursue
A1 (Theme)
A0 (Agent)
AM-MNR (Manner) we
completion of this transaction
aggressively
PropBank style
Thematic role
[A0We] are prepared to [PREDpursue] [A1aggressively] [AM-MNR completion of this transaction] he says

4. EXAMPLE OF SHARED TASK DATA
words
POS tags
Clause tags
Semantic labels
Sales NNS B-NP (S* - (A1*A1)
declined VBD B-VP * decline (V*V)
CD B-NP * - (A2*
% NN I-NP * *A2)
to TO B-PP * *
$ $ B-NP * - (A4*
CD I-NP * *
million CD I-NP * *A4)
from IN B-PP * *
$ $ B-NP * - (A3*
CD I-NP * *
million CD I-NP * *A3)
O *S) *
BP tags (BOI2)
Predicate Info

5. OUTLINE OF OUR STRATEGY
Change Shared Task Representation
make sure that it is reversible
Engineer additional features
use intuition, experience and data analysis
Optimize system settings
context size
SVM parameters; degree of polynomial, C

6. CHANGE IN REPRESENTATION
Restructure available information
- words collapsed into respective BPs
- only headwords are retained (rightmost words)
- exceptions: VPs with the predicate; Outside (O) chunks
Modify semantic role labeling
- BOI2 scheme instead of bracketing scheme

7. NEW REPRESENTATION BPs POS tags Clause tags Semantic labels (BOI2)
NP Sales NNS B-NP (S* - B-A1
VP declined VBD B-VP * decline B-V
NP % NN I-NP * - B-A2
PP to TO B-PP * - O
NP million CD I-NP * - B-A4
PP from IN B-PP * - O
NP million CD I-NP * - B-A3
O O *S) - O
Headwords
BP tags (BOI2)
Predicate Info

8. DIFFERENCES BETWEEN REPRESENTATIONS
Original
Representation
New
Tokens
words
base phrases
Lexical Info
all words
headwords
#Tagging Steps
larger
fewer
Context span
narrower
wider
# Role Labels
greater
smaller
Info Loss -
yes
Performance
worse
better

9. SYSTEM DESCRIPTION Phrase-by-phrase Left-to-right
Binary feature encoding
Discriminative
Deterministic
SVM based (YamCha toolkit, developed by Taku Kudo)
Simple post-processing (for consistent bracketing)

10. BASE FEATURES Words Predicate lemmas Part of speech tags
Base phrase IOB2 tags
Clause bracketing tags
Named Entities

11. ADDITIONAL FEATURES Token level Sentence level Token position
Path
Clause bracket patterns
Clause Position
Headword suffixes
Distance
Length
Predicate POS tag
Predicate Frequency
Predicate Context (POS, BP)
Predicate Argument Frames
Number of predicates

12. EXPERIMENTAL SET-UP Corpus: Flattened PropBank (2004 release)
Training set: Sections 15-18
Dev set: Section 20
Test set: Section 21
SVMs: 78 OVA classes, polynomial kernel, d=2, C=0.01
Context: sliding +2/-2 tokens window

13. RESULTS Base features, W-by-W & P-by-P approaches, dev set
Method
Precision
Recall F1
W-by-W
68.34%
45.16%
54.39
P-by-P
69.04%
54.68%
61.02
All features, P-by-P approach
Data
Precision
Recall F1
Dev set
74.17%
69.42%
71.72
Test set
72.43%
66.77%
69.49

14. CONCLUSIONS We have done SRL by tagging base phrase chunks
- original representation has been changed
- additional features have been engineered
- SVMs have been used
Improved performance with new representation and
additional features
Compared to W-by-W approach, our method
- classifies larger units
- uses wider context
- runs faster
- performs better

15. THANK YOU! So so… Boring! Cool! Wow!… That’s OK!… Awesome!
Not too bad!
Yawning..

16. CLAUSE FEATURES Clause (CL) markers CL pattern to predicate
One CD B-NP (S* - OUT (S*(S**S) troubling VBG I-NP * - OUT (S**S) (S* aspect NN I-NP * - OUT (S**S) (S* of IN B-PP * - OUT (S**S) (S* DEC NNP B-NP * - OUT (S**S) (S* 's POS B-NP * - OUT (S**S) (S* results NNS I-NP * - OUT (S**S) (S* , , O * - OUT (S**S) (S* analysts NNS B-NP (S* - IN (S**S) (S* said VBD B-VP *S) say IN
, , O * - OUT *S) *S)
was VBD B-VP * - OUT *S) *S) its PRP$ B-NP * - OUT *S) *S)
performance NN I-NP * - OUT *S) *S)
in IN B-PP * - OUT *S) *S)
Europe NNP B-NP * - OUT *S) *S)
. . O *S) - OUT *S)*S)
CL pattern to
sentence begin
predicate
CL pattern to
sentence end

17. SUFFIXES  suffixes of length 2-4 as features for head words are tried
The confusion
B-AM-MNR  B-AM-TMP
single word cases: fetchingly, tacitly, provocatively
 suffixes of length 2-4 as features for head words are tried