Semantic Role Labeling using Maximum Entropy Model Joon-Ho Lim NLP Lab. Korea Univ.

2 Contents  Previous work Review the previously studied ML approaches for SRL  Semantic Role Labeling using ME Explain a probabilistic model for semantic role sequence, and an incremental approach  Feature Sets for Semantic Role Labeling  Experiments  Conclusion

3 Previous work  Machine learning approaches for SRL [Gildea 2002] : A probabilistic discriminative model  It needs a complex interpolation for smoothing because of the data sparseness problem. [Pradhan 2003] : Applied a SVM to SRL.  It requires high computational complexity because of the polynomial kernel function.  Because the SVM is a binary-classifier, one-vs-rest or pairwise method is required. [Thompson 2003] : A probabilistic generative model.  This model assumes that a constituent is generated by a semantic role, so it is called a Generative Model.  Because the constituent depends only on the role that generated it, and the constituent is independent of each other, this model can’t exploit rich features.

4 Semantic Role Labeling using ME (1/4)  An overview of Maximum Entropy Modeling It has advantages as follows:  It enables to exploit rich features.  It can alleviate the data sparseness problem.  It is adequate for the multi-class classification problem.  It has relatively low computational complexity. The conditional probability of predicting an outcome y given a history x :  f i (x,y) : feature function λ i : the weighting parameter of f i (x,y) k : the number of features Z(x) : the normalization factor

5 Semantic Role Labeling using ME (2/4)  Probabilistic model for semantic role sequence  R is the sequence of semantic roles.  c 1n is the sequence of the constituents such as a clause or a chunk.  r i is the i-th semantic role, which is represented by using a BIO- notation O is too frequently occurred than others, so it has somewhat high probability than others Therefore, we use B-A*, I-A*, O- (previous others), O0 (pred), O+ (posterior others). This probability is estimated using a ME.

6 Semantic Role Labeling using ME (3/4)  An incremental approach The motivation of an incremental approach is as follows:  It is much easier and more reliable to identify the arguments in the immediate clause which contains the target predicate. Most of semantic roles are located in the immediate clause. A few of semantic roles are located in the upper clauses which include the immediate clause It is composed of two steps 1) In the first step, we label the constituents in the immediate clause 2) In the second step, we label the constituents in the upper clauses

7 Semantic Role Labeling using ME (4/4)  Rewrite a probabilistic model to reflect an incremental approach  m : the number of constituents in the immediate clause  Φ 1 : a feature set for immediate clause  Φ 2 : a feature set for upper clauses Immediate clause Upper clauses

8 Feature Sets for SRL (1/2)  For the accurate SRL, the following features are important First, as features for the predicate.  We use pred-POS, pred-lex, pred-type, voice The pred-type represents an information such as to-infinitive form (TO), etc. The voice represents an information such as active or passive Second, as features for the constituent  We use constituent tag, head-word, content-word For example, for the PP-chunk, the head-word is a preposition, and the content-word is a noun. Third, as features for the relation between them  We use path, position path is the sequence of constituent tags between the current constituent and the predicate. position is the constituent position with respect to predicate.

9 Feature Sets for SRL (2/2) Fourth, as features for the context of the constituent  We use a previous-label, previous-tag, previous-head-word, next-tag, next-head-word  Using these features, we construct Conjoined Feature Sets (Φ 1, Φ 2 ). Because the ME model assumes the independence of features, we should conjoin the coherent features. The more detailed descriptions of the features and the feature sets are explained in the proceeding.

10 Experiments (1/2)  For our experiments, we use a Zhang le's MaxEnt Toolkit The BFGS parameter estimation algorithm is used with the Gaussian Prior Smoothing.  Experimental result of the proposed method on test set

11 Experiments (2/2)  From the experimental results, we can know that The proposed method has relatively high performance on the labels related to A0 and A1, while it has relatively low performance on the other labels. We think that this may be caused by following two reasons.  Firstly, the instances of A0 and A1 are provided enough for accurate semantic role labeling.  Secondly, the thematic roles of A0 and A1 are more clear than other core semantic roles. For example, agent is labeled as mainly A0 while benefactive can be labeled as A2 or A3. Therefore, the ME model can get a good generalization performance in case of A0 and A1, but can't generalize well in other cases.

12 Conclusion  Semantic role labeling method using a ME model We use an incremental approach  It is a kind of divide-and-conquer strategy. It is much easier and more reliable to identify the arguments in the immediate clause. And then, we label the semantic roles in the upper clauses As feature sets  We use features which utilize the characteristics of a predicate, a constituent, a relationship between them, and a context.

13 Thank you.