Presentation is loading. Please wait.

Presentation is loading. Please wait.

SEMANTIC ROLE LABELING BY TAGGING SYNTACTIC CHUNKS Kadri Hacioglu 1, Sameer Pradhan 1, Wayne Ward 1 James H. Martin 1, Daniel Jurafsky 2 1 The Center for.

Similar presentations


Presentation on theme: "SEMANTIC ROLE LABELING BY TAGGING SYNTACTIC CHUNKS Kadri Hacioglu 1, Sameer Pradhan 1, Wayne Ward 1 James H. Martin 1, Daniel Jurafsky 2 1 The Center for."— Presentation transcript:

1 SEMANTIC ROLE LABELING BY TAGGING SYNTACTIC CHUNKS Kadri Hacioglu 1, Sameer Pradhan 1, Wayne Ward 1 James H. Martin 1, Daniel Jurafsky 2 1 The Center for Spoken Language Research University of Colorado at Boulder 2 Stanford NLP Group Stanford University

2 Semantic Role Labeling (SRL) Nature of Shared Task Data Our Strategy System Description & Features Experiments Concluding Remarks OUTLINE

3 Based on predicate-argument structure: First explored by (Gildea & Jurafsky, 2000) SEMANTIC ROLE LABELING Predicate: pursue A1 (Theme) A0 (Agent) AM-MNR (Manner) we completion of this transaction aggressively [ A0 We] are prepared to [ PRED pursue] [ A1 aggressively] [ AM-MNR completion of this transaction] he says PropBank style Thematic role

4 EXAMPLE OF SHARED TASK DATA Sales NNS B-NP(S*-(A1*A1) declined VBD B-VP* decline (V*V) 10 CD B-NP*-(A2* % NN I-NP*- *A2) to TO B-PP*- * $ $ B-NP*-(A4* CD I-NP*- * million CD I-NP*- *A4) from IN B-PP*- * $ $ B-NP*-(A3* CDI-NP*- * million CDI-NP*- *A3)..O*S)- * wordsPOS tags BP tags (BOI2) Clause tags Predicate Info Semantic labels

5 OUTLINE OF OUR STRATEGY Change Shared Task Representation make sure that it is reversible Engineer additional features use intuition, experience and data analysis Optimize system settings context size SVM parameters; degree of polynomial, C

6 CHANGE IN REPRESENTATION Restructure available information - words collapsed into respective BPs - only headwords are retained (rightmost words) - exceptions: VPs with the predicate; Outside (O) chunks Modify semantic role labeling - BOI2 scheme instead of bracketing scheme

7 NEW REPRESENTATION NPSales NNS B-NP(S*-B-A1 VP declined VBD B-VP* declineB-V NP % NN I-NP*-B-A2 PPto TO B-PP*-O NPmillion CD I-NP*-B-A4 PPfrom IN B-PP*-O NPmillion CDI-NP*-B-A3 O..O*S)-O BPsPOS tags BP tags (BOI2) Clause tags Predicate Info Semantic labels (BOI2) Headwords

8 DIFFERENCES BETWEEN REPRESENTATIONS DIFFERENCES BETWEEN REPRESENTATIONS Original Representation New Representation Tokenswordsbase phrases Lexical Infoall wordsheadwords #Tagging Steps largerfewer Context spannarrowerwider # Role Labelsgreatersmaller Info Loss-yes Performanceworsebetter

9 SYSTEM DESCRIPTION Phrase-by-phrase Left-to-right Binary feature encoding Discriminative Deterministic SVM based (YamCha toolkit, developed by Taku Kudo) Simple post-processing (for consistent bracketing)

10 Words Predicate lemmas Part of speech tags Base phrase IOB2 tags Clause bracketing tags Named Entities BASE FEATURES

11 ADDITIONAL FEATURES Token position Path Clause bracket patterns Clause Position Headword suffixes Distance Length Predicate POS tag Predicate Frequency Predicate Context (POS, BP) Predicate Argument Frames Number of predicates Token level Sentence level

12 EXPERIMENTAL SET-UP Corpus: Flattened PropBank (2004 release) Training set: Sections Dev set: Section 20 Test set: Section 21 SVMs: 78 OVA classes, polynomial kernel, d=2, C=0.01 Context: sliding +2/-2 tokens window

13 RESULTS DataPrecisionRecallF1F1 Dev set74.17%69.42%71.72 Test set72.43%66.77%69.49 MethodPrecisionRecallF1F1 W-by-W68.34%45.16%54.39 P-by-P69.04%54.68%61.02 Base features, W-by-W & P-by-P approaches, dev set All features, P-by-P approach

14 CONCLUSIONS We have done SRL by tagging base phrase chunks - original representation has been changed - additional features have been engineered - SVMs have been used Improved performance with new representation and additional features Compared to W-by-W approach, our method - classifies larger units - uses wider context - runs faster - performs better

15 THANK YOU! So so… That’s OK!… Boring! Awesome! Cool! Wow!… Yawning.. Not too bad!

16 CLAUSE FEATURES One CD B-NP (S* - OUT (S*(S**S) - troubling VBG I-NP * - OUT (S**S) (S* aspect NN I-NP * - OUT (S**S) (S* of IN B-PP * - OUT (S**S) (S* DECNNP B-NP * - OUT (S**S) (S* 's POS B-NP * - OUT (S**S) (S* results NNS I-NP * - OUT (S**S) (S*,, O * - OUT (S**S) (S* analysts NNS B-NP (S* - IN (S**S) (S* said VBD B-VP *S) say IN - -,, O * - OUT *S) *S) was VBD B-VP * - OUT *S) *S) its PRP$ B-NP * - OUT *S) *S) performance NN I-NP * - OUT *S) *S) in IN B-PP * -OUT *S) *S) Europe NNP B-NP * - OUT *S) *S).. O *S) - OUT *S)*S) - CL pattern to predicate CL pattern to sentence end CL pattern to sentence begin predicate Clause (CL) markers

17 SUFFIXES The confusion B-AM-MNR  B-AM-TMP single word cases: fetchingly, tacitly, provocatively  suffixes of length 2-4 as features for head words are tried


Download ppt "SEMANTIC ROLE LABELING BY TAGGING SYNTACTIC CHUNKS Kadri Hacioglu 1, Sameer Pradhan 1, Wayne Ward 1 James H. Martin 1, Daniel Jurafsky 2 1 The Center for."

Similar presentations


Ads by Google