# POS Tagging & Chunking Sambhav Jain LTRC, IIIT Hyderabad.

## Presentation on theme: "POS Tagging & Chunking Sambhav Jain LTRC, IIIT Hyderabad."— Presentation transcript:

POS Tagging & Chunking Sambhav Jain LTRC, IIIT Hyderabad

POS Tagging - Introduction Need for POS Tag ? – Assigning category which are < Vocab – Essential for systems – Has grammatical information 2POS Tagging and Chunking

Building a POS Tagger Rule Based Approaches – Come up with rules – Eg. Substitution: The {sad,green,fat,old} one in the garden. Statistical Learning / Machine Learning – Make machine learn automatically from annotated instances – Unsupervised - Baum Welch Pros/Cons of each approach 3POS Tagging and Chunking

Machine Learning ---- ??? How can machine learn from examples (on Board) 4POS Tagging and Chunking

Problem Statement Given W = w0 w1 w2 w3 w4 w5 w6 w7 Find T = t1 t2 t3 t4 t5 t6 t7 OR That T for which P(T/W) should be maximum 5POS Tagging and Chunking

Hidden Markov Models P(T/W) = P(W/T)*P(T)/P(W) T= Argmax T P(W/T)*P(T)/P(W) = Argmax T P(W/T)*P(T) P(W/T) = P(w0 w1 w2 w3 w4 w5 w6 w7 / t1 t2 t3 t4 t5 t6 t7 ) (On Board) – Chain Rule Markov Assumption 6POS Tagging and Chunking

Hidden Markov Models How can we learn these values form annotated corpus. Emission Matrix Transition Matrix Example (on Board) 7POS Tagging and Chunking

Hidden Markov Models Given Emission/Transition Matrix – Tag a new sequence – Complexity Viterbi Algorithm - Decoding Example (On Board) 8POS Tagging and Chunking

Viterbi Algorithm (Decoding) Most probable tag sequence given text: T*= arg max T P λ (T | W) = arg max T P λ (W | T) P λ (T) / P λ (W) (Bayes’ Theorem) = arg max T P λ (W | T) P λ (T) (W is constant for all T) = arg max T  i [ a(t i-1  t i ) b(w i | t i ) ] = arg max T  i log [ a(t i-1  t i ) b(w i | t i ) ] 9POS Tagging and Chunking

t1t1 t2t2 t3t3 w1w1 t1t1 t2t2 t3t3 w2w2 t1t1 t2t2 t3t3 w3w3 t0t0 A(,) t1t1 t2t2 t3t3 t 0  0.0050.020.1 t 1  0.020.10.005 t 2  0.50.0005 t 3  0.05 0.005 B(,) w1w1 w2w2 w3w3 t1t1 0.20.005 t2t2 0.020.20.0005 t3t3 0.02 0.05 10POS Tagging and Chunking

-log A t1t1 t2t2 t3t3 t 0  2.31.71 t 1  1.712.3 t 2  0.33.3 t 3  1.3 2.3 -log B w1w1 w2w2 w3w3 t1t1 0.72.3 t2t2 1.70.73.3 t3t3 1.7 1.3 t1t1 t2t2 t3t3 w1w1 t1t1 t2t2 t3t3 w2w2 t1t1 t2t2 t3t3 w3w3 t0t0 -1.7 -0.3 -1.3 -3 -3.4 -2.7 -2.3 -1.7 -6 -4.7 -6.7 -1.7 -0.3 -1.3 -7.3 -9.3 -10.3 11POS Tagging and Chunking

Tools for POS Tagging It is a Sequence Labeling Task Tools – HMM based – TNT Tagger (http://www.coli.uni-saarland.de/~thorsten/tnt/)http://www.coli.uni-saarland.de/~thorsten/tnt/ – CRF based – CRF++ (http://crfpp.googlecode.com/svn/trunk/doc/index.html)http://crfpp.googlecode.com/svn/trunk/doc/index.html 12POS Tagging and Chunking

CRF for Chunking (On Board) Tools – CRF++ (http://crfpp.googlecode.com/svn/trunk/doc/index.html)http://crfpp.googlecode.com/svn/trunk/doc/index.html 13POS Tagging and Chunking

Understanding CRF Template # Unigram U00:%x[-2,0] U01:%x[-1,0] U02:%x[0,0] U03:%x[1,0] U04:%x[2,0] U05:%x[-1,0]/%x[0,0] U06:%x[0,0]/%x[1,0] U10:%x[-2,1] U11:%x[-1,1] U12:%x[0,1] U13:%x[1,1] U14:%x[2,1] U15:%x[-2,1]/%x[-1,1] U16:%x[-1,1]/%x[0,1] U17:%x[0,1]/%x[1,1] U18:%x[1,1]/%x[2,1] U20:%x[-2,1]/%x[-1,1]/%x[0,1] U21:%x[-1,1]/%x[0,1]/%x[1,1] U22:%x[0,1]/%x[1,1]/%x[2,1] # Bigram B 14POS Tagging and Chunking