Presentation is loading. Please wait.

Presentation is loading. Please wait.

CS460/IT632 Natural Language Processing/Language Technology for the Web Lecture 2 (06/01/06) Prof. Pushpak Bhattacharyya IIT Bombay Part of Speech (PoS)

Similar presentations


Presentation on theme: "CS460/IT632 Natural Language Processing/Language Technology for the Web Lecture 2 (06/01/06) Prof. Pushpak Bhattacharyya IIT Bombay Part of Speech (PoS)"— Presentation transcript:

1 CS460/IT632 Natural Language Processing/Language Technology for the Web Lecture 2 (06/01/06) Prof. Pushpak Bhattacharyya IIT Bombay Part of Speech (PoS) Tagging

2 06/01/06Prof. Pushpak Bhattacharyya, IIT Bombay 2 Tagging or Annotation ● Purpose is Disambiguation ● A word can have a number of labels ● The problem is to give unique label. ● PoS tagging makes use of the “local context”, whereas Sense tagging needs “long distance dependency” and hence difficult too. ● PoS tagging is needed in mainly parsing and also in other applications.

3 06/01/06Prof. Pushpak Bhattacharyya, IIT Bombay 3 Approaches ● Rule Based approach ● Statistical approach – we will mainly focus on the statistical approach

4 06/01/06Prof. Pushpak Bhattacharyya, IIT Bombay 4 Types of Tagging Tasks ● PoS ● Named entity ● Sense ● Parse tree

5 06/01/06Prof. Pushpak Bhattacharyya, IIT Bombay 5 PoS Tagging ● Example – “The Orange ducks clean the bills.” ● Assign tags to each word from the lexicon; multiple possibilities exist

6 06/01/06Prof. Pushpak Bhattacharyya, IIT Bombay 6 Lexicon dictionary ● The: – DT (Determiner) ● Orange: – NN (Noun) – JJ (Adjective) ● Duck: – NN – VB ( Basic verb) ● Clean: – NN – VB ● Bill : – NN – VB JJ, VB, NN are called as Syntactic entities or PoS tags

7 06/01/06Prof. Pushpak Bhattacharyya, IIT Bombay 7 PoS tagging as a sequence labelling task ● Task is to assign the correct PoS tag sequence to the words. ● It can be: – Unigram: Consider one word while deciding the sequence. – Multigram: Consider multiple words. ● 16 (=1*2*2*2*1*2) possible sequences for the “Duck” example. ● It is a classification problem: classify each word’s tag correctly into the right category.

8 06/01/06Prof. Pushpak Bhattacharyya, IIT Bombay 8 Challenges ● Lexical ambiguity: Multiple choices ● Morphology analysis: Find the root word ● Tokenization: Find word boundaries – In Thai language there is no blank space – Non trivial (example: capturing boundaries when the word is continued to the next line with a “-”)

9 06/01/06Prof. Pushpak Bhattacharyya, IIT Bombay 9 Named Entity tagging ● Example 1: – “Mohan went to school in Kolkata” ● Tagged as: – “Mohan_Person went to School_Place in Kolkata_Place”. ● Example 2: – “Kolkata bore the brunt of 1947 riots when 1947 children died at Kolkata. – “Kolkata_? bore the brunt of 1947_year riots when 1947_num children died at Kolkata_Place.

10 06/01/06Prof. Pushpak Bhattacharyya, IIT Bombay 10 Sense tagging ● Detecting the meaning. ● Our example tagged as: – The Orange_{colour} ducks_{bird} clean the bills_{body_part} ● Sense tagging has been done by means of hypernymy. ● Semantic relations like hypernymy are stored in the lexical resource called “WordNet”.

11 06/01/06Prof. Pushpak Bhattacharyya, IIT Bombay 11 Parse Tree tagging ● Example parse tree:

12 06/01/06Prof. Pushpak Bhattacharyya, IIT Bombay 12 Parse Tree tagging (contd.) ● Given a grammar, one can construct the parse tree. ● Annotation will produce following structure: – [ [The_DT [Orange_JJ Ducks_NN] NP ] NP [clean_VB[the_VB [bills_NN] NP ] NP ] VP ] S ● This structure is called the Penn Treebank form ● From the Treebank form, one can arrive at a grammar through learning.

13 06/01/06Prof. Pushpak Bhattacharyya, IIT Bombay 13 Statistical Formulation of the PoS tagging problem ● Input: – W 1,W 2,...W n words – C 1,C 2,....C m Lexical tags reposition (DT,JJ, NN et. al.) ● Output: – “Best” PoS tag sequence C i 1, C i 2, C i 3....C i n for the given words. ● Best means: – P(C i 1, C i 2, C i 3....C i n |W 1,W 2,...W n ) is the maximum of all possible C-sequence.

14 06/01/06Prof. Pushpak Bhattacharyya, IIT Bombay 14 ● Example: – P(DT JJ NN| The Orange duck) > P(DT NN VB| The Orange duck) is required ● Why?: – Because given the phrase “The orange duck”, there is overwhelming evidence in the corpus that “DT JJ NN” is the right tag sequence. Statistical Formation of PoS tagging problem

15 06/01/06Prof. Pushpak Bhattacharyya, IIT Bombay 15 Mathematical machinery

16 06/01/06Prof. Pushpak Bhattacharyya, IIT Bombay 16 Bayes Theorem ● P(A|B) = (P(A).P(B|A)) / P(B) – Where, – P(A): Prior probability – P(A|B): Posterior probability – P(B|A): likelihood ● Why apply Bayes theorem: – This is the Generative Vs Discriminative model question.

17 06/01/06Prof. Pushpak Bhattacharyya, IIT Bombay 17 Apply Bayes theorem P(C i 1, C i 2, C i 3....C i n |W 1,W 2,...W n ) = P(C|W) = where, C = W = P(C). P(W|C) P(W)

18 06/01/06Prof. Pushpak Bhattacharyya, IIT Bombay 18 Best tag sequence C* = *, where * signifies best C- sequence = argmax(P(C|W)) ● As denominator is common in all the tag sequences Therefore, C* = argmax(P(C).P(W|C))

19 06/01/06Prof. Pushpak Bhattacharyya, IIT Bombay 19 Processing the1 st part P(C) = P(C i 1, C i 2, C i 3....C i n ) = P(C i 1 ).P(C i 2 |C i 1 ).P(C i 3 |C i 1. C i 2 )..P(C i n |C i 1 C i 2.. C i n-1 ) (on applying chain rule of probability) Ex: P(DT JJ NN) = P(DT).P(JJ|DT).P(NN|DT JJ)

20 06/01/06Prof. Pushpak Bhattacharyya, IIT Bombay 20 Markov assumption ● Tag depends only on a window, not on everything that the “chain law” of probability demands. ● K th order Markov assumption considers only previous K tags. ● Typical values of K = 3 for English, and (it seems) 5 for Hindi.

21 06/01/06Prof. Pushpak Bhattacharyya, IIT Bombay 21 Apply assumption With K=2, our problem will be: P(C) = P(C i |C i-1 ), i: 1..n C 0 : sentence beginning marker.

22 06/01/06Prof. Pushpak Bhattacharyya, IIT Bombay 22 Exercise given in the lecture ● Contrast PoS tagging with Sense tagging. ● Find an example to show the difference.


Download ppt "CS460/IT632 Natural Language Processing/Language Technology for the Web Lecture 2 (06/01/06) Prof. Pushpak Bhattacharyya IIT Bombay Part of Speech (PoS)"

Similar presentations


Ads by Google