1. Applications of Sequence Learning CMPT 825 Mashaal A. Memon

2. What We Know of Sequence Learning
Part Of Speech (POS) Tagging is a sequence learning problem.
3 approaches to solving the problem:
Noisy-Channel
Classification
Rule-Based

3. What We Know About POS Tagging
A part of speech (POS) explains not what the word is, but how it is used.
Problem: Which POS does each word represent?
Tags: POS tags (i.e. NN = Noun, VB = Verb, etc…)
Training: Words sequences with corresponding POS tags.
Input: Word sequences.

4. What We Know About POS Tagging Continued…
Examples:
Anoop is a great professor .
NN VBZ DT JJ NN .
I am kissing butt right now .
PRP VBP RB NN RB RB .

5. What Is My Point?
Other interesting and important problems can be represented as tagging problems.
The same three approaches can be used.
4 such applications will be briefly introduced:
Chunking
Named Entity Recognition
Cascaded Chunking
Word Segmentation

6. (1) Chunking
A chunk is a syntactically correlated part of a language (i.e. noun phrase, verb phrase, etc.)
Problem: Which type of chunk does each word or group of words belong to?
Note: Chunks of the same type can sometimes kiss each other.

7. (1) Chunking Continued…
Noun-Phrase (NP) Chunking
Only look for noun phrase chunks.
Tags: B = beginning noun phrase
I = in noun phrase
O = other
Training: Word sequences with corresponding POS and NP tags.
Input: Word sequences and POS tags.

8. (1) Chunking Continued…
Noun-Phrase (NP) Chunking
Examples:
The student talked to Anoop .
B I O O B O
The guy he talked to was smelly .
B I B O O O O O

9. (1) Chunking Continued…
General Chunking
Look for other syntactical constructs as well as noun phrases.
Tags: - B or I prefix to each chunk type
- chunk types (NP = noun phrase, VP = verb phrase, PP = prepositional phrase, O = other)
Training: Word sequences with corresponding POS and chunk tags.
Input: Word sequences and POS tags.

10. (1) Chunking Continued…
General Chunking
Examples:
Anoop should give me an A+ .
B-NP B-VP I-VP B-NP B-NP I-NP O
His presentation is boring
me to death .
B-NP I-NP B-PP B-VP
B-NP B-PP B-VP O

11. (2) Named Entity Recognition
A named entity is a phrase that contains names of persons, organizations or locations
Problem: Does a word or group of words represent a named entity or not?
Tags: - B or I prefix to each NE type
- NE types (PER = person, ORG = organization, LOC = location, O = other)
Training: Word sequences with corresponding POS and NE tags. Sometimes lists of NE data are used (Cheating!!)
Input: Word sequences with POS tags.

12. (2) Named Entity Recognition Continued…
Examples:
The United States of America
O B-LOC I-LOC I-LOC I-LOC
has an intelligent leader in D.C.
O O O O O B-LOC
, Dick Cheney of Halliburton .
O B-PER I-PER O B-ORG O

13. (3) Cascaded Chunking
Cascaded chunking gives us the parse tree of the sentence back.
Can think of it as chunker taking initial input and then continues to work on its OWN output until no more changes are made to input.
Difference: Chunks may contain other chunks and POS

14. (3) Cascaded Chunking Continued…
CHUNKER (W = {w1..wn}, T = {t1..tn})
→ T’ = {t’1..t’n};
CASCADE (W = {w1..wn}, T = {t1..tn}) {
OutputBefore = {Ø};
OutputAfter = CHUNKER (W,T);
while (OutputBefore != OutputAfter) do {
OutputBefore = OutputAfter;
OutputAfter = CHUNKER (W, OutputBefore);
/* Output result of current iteration */ }

15. (3) Cascaded Chunking Continued…
Example:
The effort to establish such a conclusion is unnecessary .
DT NN TO VB PDT DT NN VBZ JJ
______ __ ________ __________ ___________
DT NP IP VP PDT DT NP AP
__________ ____________ __________________ ______________
DP CP DP CP
...
___________________________________________________________ S
Chunking is an intermediate step to a full parse

16. (4) Word Segmentation
When written, some languages like Chinese don’t have obvious word boundries.
Problem: Find whether a character or group of characters is a single word?
Tags: B = beginning of word
I = in word
Training: Character sequences with corresponding WS tags.
Input: Character sequences.

17. (4) Word Segmentation Continued…
Example:
參賽者並未參加任何賓大語料之競賽
B I I B I B I B I B B B I B B I

18. Conclusion
All problems are different in their goals, but with the same type of representation, they all can be solved with the same approaches.
We all LOVE sequence learning 
THE END

19. Questions?!

20. References
Manning D., H. Schultze. Foundations of Statistical Natural Language Processing
CoNLL shared task on Chunking Website: (
CoNLL shared task on NER Website: (
CoNLL shared task on NER Website: (
Abney, S.. Parsing By Chunks. In Journal of Psychological Research, 18(1), 1989.
Chinese Word Segmentation Bakeoff Website: (