Presentation is loading. Please wait.

Presentation is loading. Please wait.

A Structured Model for Joint Learning of Argument Roles and Predicate Senses Yotaro Watanabe Masayuki Asahara Yuji Matsumoto ACL 2010 Uppsala, Sweden July.

Similar presentations


Presentation on theme: "A Structured Model for Joint Learning of Argument Roles and Predicate Senses Yotaro Watanabe Masayuki Asahara Yuji Matsumoto ACL 2010 Uppsala, Sweden July."— Presentation transcript:

1 A Structured Model for Joint Learning of Argument Roles and Predicate Senses Yotaro Watanabe Masayuki Asahara Yuji Matsumoto ACL 2010 Uppsala, Sweden July 12, 2010 Tohoku University Nara Institute of Science and Technology

2 Page  2 Predicate-Argument Structure Analysis (Semantic Role Labeling)  Task of analyzing predicates and its arguments –A predicate represents a state or an event, and its arguments have relations to the predicate –Each of arguments has a particular semantic role (Agent, Theme, etc)  In recent years, predicate sense disambiguation has been included in predicate-argument structure analysis [Surdeanu+ 08, Hajič+ 09] –‘sell.01’ means that ‘sold’ is an instance of the first sense of ‘sell’  Important for many NLP applications –MT, QA, RTE, etc. Theme Location Temporal luxuryautomakerlastTheyearsold1,214carsintheU.S. maker.01 sell.01 Product Agent

3 Page  3 drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion Two Types of Dependencies of Elements in Predicate- Argument Structures (1)Inter-dependencies between a predicate and its arguments –A1: car => we can infer that the correct sense is drive.01 (2)Non-local dependencies among arguments Two or more arguments do not have the same role Basically, obligatory roles of the predicate should appear in sentences drive.01 A0 A1 SBJ NMOD OBJ Pauldrovehiscar In order to realize robust predicate-argument structure analysis, it is necessary to deal with these types of dependencies

4 Page  4 Previous Work (1)Non-local dependencies among arguments: Re-ranking [Johansson and Nugues 2008, etc.] Generate N-best assignments of argument roles, then obtain global features for each assignment, finally select the argmax using the re-ranker Can not explicitly capture inter-dependencies between a predicate and its arguments (2)Inter-dependencies between a predicate and its arguments: Markov Logic Networks [Meza-Ruiz and Riedel 2009, etc.] Jointly learn and classify pred. senses and arg. roles simultaneously MLN can not deal with particular types of global features Currently, no existing (discriminative) approach sufficiently handles both types of dependencies

5 Page  5 Previous Work (1)Non-local dependencies among arguments: Re-ranking [Johansson and Nugues 2008, etc.] Generate N-best assignments of argument roles, then obtain global features for each assignment, finally select the argmax using the re-ranker Can not explicitly capture inter-dependencies between a predicate and its arguments (2)Inter-dependencies between a predicate and its arguments: Markov Logic Networks [Meza-Ruiz and Riedel 2009, etc.] Jointly learn and classify pred. senses and arg. roles simultaneously MLN can not deal with particular types of global features Currently, no existing (discriminative) approach sufficiently handles both types of dependencies We propose a structured model that can capture both types of dependencies simultaneously

6 Page  6 The proposed model SBJ NMOD OBJ Pauldrovehiscar drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion

7 Page  7 The proposed model A0 drive.01 drive.02 … A1 A0 … Paulcar drove NONE SBJ NMOD OBJ Pauldrovehiscar drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion Expand the possible labels of predicate senses and argument roles

8 Page  8 The proposed model A0 drive.01 drive.02 … A1 A0 … Paulcar drove NONE SBJ NMOD OBJ Pauldrovehiscar drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion Expand the possible labels of predicate senses and argument roles We use four types of factors which score labels of elements in predicate- argument structures

9 Page  9 The proposed model A0 drive.01 drive.02 … A1 A0 … Paulcar drove NONE SBJ NMOD OBJ Pauldrovehiscar drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion Expand the possible labels of predicate senses and argument roles These factors are defined by (linear model) These factors are defined by (linear model) We use four types of factors which score labels of elements in predicate- argument structures

10 Page  10 A0 drive.01 drive.02 … A1 A0 … Paulcar drove NONE The proposed model SBJ NMOD OBJ Pauldrovehiscar FPFP drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion use a factor which scores sense labels of the predicate

11 Page  11 A0 drive.01 drive.02 … A1 A0 … Paulcar drove NONE The proposed model SBJ NMOD OBJ Pauldrovehiscar FAFA FPFP drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion use a factor which scores role labels of each argument

12 Page  12 A0 … A1 A0 … Paulcar drove NONE The proposed model SBJ NMOD OBJ Pauldrovehiscar F PA FAFA FPFP drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion drive.01 drive.02 add a factor which scores label pairs of a predicate sense and a semantic role of an argument

13 Page  13 The proposed model drive.02 … A1 A0 … Paulcar drove NONE A0,drive01,A1 … A0,drive01,A1 … A0 drive.01 A1 SBJ NMOD OBJ Pauldrovehiscar FPFP F PA FAFA FGFG drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion add a factor which captures plausibility of the whole predicate-argument structure (use global features) add a factor which captures plausibility of the whole predicate-argument structure (use global features)

14 Page  14 The proposed model drive.02 … A1 A0 … Paulcar drove NONE A0,drive01,A1 … A0,drive01,A1 … A0 drive.01 A1 SBJ NMOD OBJ Pauldrovehiscar FPFP F PA FAFA FGFG drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion add a factor which captures plausibility of the whole predicate-argument structure (use global features) add a factor which captures plausibility of the whole predicate-argument structure (use global features) The predicate ‘drive’ has all obligatory roles A0 and A1 => F G assigns the higher score to the weight corresponds to this feature The predicate ‘drive’ has all obligatory roles A0 and A1 => F G assigns the higher score to the weight corresponds to this feature

15 Page  15 The proposed model drive.02 … A1 A0 … Paulcar drove NONE A0 drive.01 A1 NONE SBJ NMOD OBJ Pauldrovehiscar A0,drive01,A1 … A0,drive01,A1 … FPFP F PA FAFA FGFG drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion The proposed model combines these types of factors

16 Page  16 The proposed model drive.02 … A1 A0 … Paulcar drove NONE A0 drive.01 A1 NONE drive.01 A0 A SBJ NMOD OBJ Pauldrovehiscar A0,drive01,A1 … A0,drive01,A1 … FPFP F PA FAFA FGFG drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion The proposed model combines these types of factors The highest scoring assignment is returned by the proposed model

17 Page  17 Dealing with global (non-local) features  Introduce the fundamental idea of [Kazama and Torisawa 2007] –Features are divided into local features and global features –Inference: N-best based approach (1) Generate N-best assignments using only local features (2) Obtain global features in the N-best assignments (3) Select the argmax –Learning: train parameters with two margin constraints All: train parameters so as to ensure a sufficient margin using all features (both local features and global features) Local only: when the constraint All is satisfied, train parameters so as to ensure a sufficient margin using only local features K&T proposed a Margin-Perceptron Learning Algorithm

18 Page  18 Inference and Learning Algorithm of the Proposed Model Inference: generate N-best assignments for each predicate sense Learning: the online Passive-Aggressive Algorithm [Crammer 2006] The parameters are trained by solving the optimization problem used in PA with the two margin constraints: All (local + global) and Local only Inference: generate N-best assignments for each predicate sense Learning: the online Passive-Aggressive Algorithm [Crammer 2006] The parameters are trained by solving the optimization problem used in PA with the two margin constraints: All (local + global) and Local only (1) All (local + global) margin (2) Local only margin positive other positive other

19 Page  19 Results on the CoNLL-2009 ST Dataset (average) feature selection Overall (Sem. F1) WSD (Acc.) SRL (Lab. F1) F P +F A no F P +F A +F PA no F P +F A +F G no ALLno Björkelundyes80.80 Zhaoyes80.47 Meza-Ruizno77.46 sense FPFP F PA FGFG FAFA … role 1 role 2 role N  The best performance is obtained by using the all factors  Our model achieved the competitive results with the top system in the CoNLL-2009 Shared Task without any feature selection procedure

20 Page  20 Results on the CoNLL-2009 ST Dataset (average) feature selection Overall (Sem. F1) WSD (Acc.) SRL (Lab. F1) F P +F A no F P +F A +F PA no F P +F A +F G no ALLno Björkelundyes80.80 Zhaoyes80.47 Meza-Ruizno77.46 sense FPFP F PA FGFG FAFA … role 1 role 2 role N  By adding two types of factors F PA and F G, we obtained performance improvements in both tasks (predicate sense disambiguation and argument role labeling) => Succeeded in joint learning

21 Page  21 Summary  We proposed a structured model that can capture two types of dependencies (1)Non-local dependencies among arguments (2)Inter-dependencies between a predicate and its arguments  The proposed model achieved the competitive results with the state-of- the-art SRL systems without any feature selection procedure  By adding two types of factors, we obtained performance improvements on both predicate sense disambiguation and argument role labeling => succeeded in joint learning  Future Work –exploiting unlabeled data (unsupervised or semi-supervised predicate-argument structure analysis)


Download ppt "A Structured Model for Joint Learning of Argument Roles and Predicate Senses Yotaro Watanabe Masayuki Asahara Yuji Matsumoto ACL 2010 Uppsala, Sweden July."

Similar presentations


Ads by Google