1. Hidden Markov Models
IP notice: slides from Dan Jurafsky

2. Outline Markov Chains Hidden Markov Models Three Algorithms for HMMs
The Forward Algorithm
The Viterbi Algorithm
The Baum-Welch (EM Algorithm)
Applications:
The Ice Cream Task
Part of Speech Tagging

3. Chomsky Grammars
Distinguish grammatical English from ungrammatical English:
John thinks Sara hit the boy
The hit thinks Sara John boy
John thinks the boy was hit by Sara
Who does John think Sara hit?
John thinks Sara hit the boy and the girl
Who does John think Sara hit the boy and?
John thinks Sara hit the boy with the bat
What does John think Sara hit the boy with?
Colorless green ideas sleep furiously.
Green sleep furiously ideas colorless.

4. Miniature English grammar and lexicon
S → NP VP S → Aux NP VP S → VP NP → Pronoun NP → Proper-Noun NP → Det Nominal Nominal → Noun Nominal → Nominal Noun Nominal → Nominal PP VP → Verb VP → Verb NP VP → Verb NP PP VP → Verb PP VP → VP PP PP → Preposition NP
Det → that | this | a Noun → book | flight | meal | money Verb → book | include | prefer Pronoun → I | she | me Proper-Noun → Houston | TWA Aux → does Preposition → from | to | on | near | through

5. Sentence Generation
S → VP VP → Verb NP Verb → book NP → Det Nominal Det → that Nominal → Noun Noun → flight Book that flight

6. Acceptors and Transformers
Chomsky’s grammars are about which utterances are acceptable
Other research programs are aimed at transforming utterances
Translate a English sentence into Japanese…
Transform a speech waveform into transcribed words…
Compress a sentence, summarize a text…
Transform a syntactic analysis into a semantic analysis…
Generate a text from a semantic representation…

7. Strings and Trees
Trees are a useful representation for grammatical structure
A sentence is a noun phrase (NP) followed by a verb phrase (VP)
A noun phrase is a determiner (DT) followed by a noun (NN)
A noun phrase is a noun phrase (NP) followed by a prepositional phrase (PP)
A PP is a preposition (IN) followed by an NP
A string is acceptable if it has an acceptable tree
Transformations may take place at the tree level S NP VP PP IN

8. Natural Language Processing
1980s: Many tree-based grammatical formalisms
1990s: Regression to sequence-based formalisms
Hidden Markov Models (HMM), Finite-State Acceptors (FSAs) and Transducers (FSTs)
N-gram models for accepting sentences [e.g., Jelinek 90]
Taggers and other statistical transformations [e.g., Church 88]
Machine translation [e.g., Brown et al. 93]
Software toolkits implementing generic weighted FST operations [e.g., Mohri, Pereira, Riley 00]

9. Natural Language Processing
2000s: Emerging interest in tree-based probabilistic models
Machine translation
[Wu 97, Yamada & Knight 02, Melamed 03, Chiang 05, …]
Summarization
[Knight & Marcu 00, …]
Paraphrasing
[Pang et al 03, …]
Question answering
[Echihabi & Marcu 03, …]
Natural language generation
[Bangalore & Rambow 00, …]

10. FSA and FST Acceptors Transducers Unweighted Weighted c c:z a a:x c/.7
e:y
c:z
Unweighted
a/.5
e/.5
c/.7
a:x/.5
e:y/.5
c:z/.7
Weighted

11. Finite-State Transducer (FST)
Original input:
Transformation: q
i : AY q q2
qfinal q3 q4
k : ε
n : N
h : ε
g : ε
t : T k n i g h t

12. Finite-State (String) Transducer
Original input:
Transformation: q q2
qfinal q3 q4
k : ε
n : N
h : ε
g : ε
t : T
i : AY k q2 n i g h t

13. Finite-State (String) Transducer
Original input:
Transformation: k q q2
qfinal q3 q4
k : ε
n : N
h : ε
g : ε
t : T
i : AY N n i g h t

14. Finite-State (String) Transducer
Original input:
Transformation: k q q2
qfinal q3 q4
k : ε
n : N
h : ε
g : ε
t : T
i : AY N n i AY q g h t t

15. Finite-State (String) Transducer
Original input:
Transformation: k q q2
qfinal q3 q4
k : ε
n : N
h : ε
g : ε
t : T
i : AY N n i AY g q3 h t

16. Finite-State (String) Transducer
Original input:
Transformation: k q q2
qfinal q3 q4
k : ε
n : N
h : ε
g : ε
t : T
i : AY N n AY i g h q4 t

17. Finite-State (String) Transducer
Original input:
Transformation: k q q2
qfinal q3 q4
k : ε
n : N
h : ε
g : ε
t : T
i : AY N n AY i g h T
qfinal t

18. Definitions A Weighted Finite-State Automaton (WFSA)
An FSA with probabilities on the arcs
The sum of the probabilities leaving any arc must sum to one
A Markov Chain (or observable Markov Model)
a special case of a WFST in which the input sequence uniquely determines which states the automaton will go through
Markov Chains can’t represent inherently ambiguous problems
Useful for assigning probabilities to unambiguous sequences

19. Weighted Finite State Transducer
FST: FSA whose state transitions are labeled with both input and output symbols.
A weighted transducer puts weights on transitions in addition to the input and output symbols
Weights may encode probabilities, durations, penalties, ...
Used in speech recognition
Tutorial at

20. Markov Chain for weather

21. Markov Chain for words

22. Markov Chain First-order observable Markov Model A set of states Q
q1, q2…qN sequence of states: state at time t is qt
Transition probabilities:
a set of probabilities A = a01a02…an1…ann.
Each aij represents the probability of transitioning from state i to state j
Distinguished start and end states

23. Markov Chain Markov Assumption:
Current state only depends on previous state
P(qi | q1 … qi-1) = P(qi | qi-1)

24. Another representation for start state
Instead of start state
Special initial probability vector p
An initial distribution over probability of start states
Constraints:

25. The weather model using p

26. The weather model: specific example

27. Markov chain for weather
What is the probability of 4 consecutive warm days?
Sequence is warm-warm-warm-warm
i.e., state sequence is
P(3, 3, 3, 3) =
3a33a33a33a33 = 0.2 • (0.6)3 =

28. How about? Hot hot hot hot Cold hot cold hot
What does the difference in these probabilities tell you about the real world weather info encoded in the figure?

29. HMM for Ice Cream You are a climatologist in the year 2799
Studying global warming
You can’t find any records of the weather in Baltimore, MD for summer of 2008
But you find Jason Eisner’s diary
Which lists how many ice-creams Jason ate every date that summer
Our job: figure out how hot it was

30. Hidden Markov Model
For Markov chains, the output symbols are the same as the states.
See hot weather: we are in state hot
But in named-entity or part-of-speech tagging (and speech recognition and other things)
The output symbols are words
But the hidden states are something else
Part-of-speech tags
Named entity tags
So we need an extension!
A Hidden Markov Model is an extension of a Markov chain in which the input symbols are not the same as the states.
This means we don’t know which state we are in.

31. Hidden Markov Models

32. Assumptions Markov assumption: P(qi | q1 … qi-1) = P(qi | qi-1)
Output-independence assumption

33. Eisner task Given Produce:
Ice Cream Observation Sequence: 1,2,3,2,2,2,3…
Produce:
Weather Sequence: H,C,H,H,H,C…

34. HMM for ice cream

35. Different types of HMM structure
Ergodic =
fully-connected
Bakis = left-to-right

36. The Three Basic Problems for HMMs
Jack Ferguson at IDA in the 1960s
Problem 1 (Evaluation): Given the observation sequence O=(o1o2…oT), and an HMM model  = (A,B), how do we efficiently compute P(O| ), the probability of the observation sequence, given the model
Problem 2 (Decoding): Given the observation sequence O=(o1o2…oT), and an HMM model  = (A,B), how do we choose a corresponding state sequence Q=(q1q2…qT) that is optimal in some sense (i.e., best explains the observations)
Problem 3 (Learning): How do we adjust the model parameters  = (A,B) to maximize P(O|  )?

37. Problem 1: computing the observation likelihood
Given the following HMM:
How likely is the sequence 3 1 3?

38. How to compute likelihood
For a Markov chain, we just follow the states and multiply the probabilities
But for an HMM, we don’t know what the states are!
So let’s start with a simpler situation.
Computing the observation likelihood for a given hidden state sequence
Suppose we knew the weather and wanted to predict how much ice cream Jason would eat.
i.e. P( | H H C)

39. Computing likelihood of 3 1 3 given hidden state sequence

40. Computing joint probability of observation and state sequence

41. Computing total likelihood of 3 1 3
We would need to sum over
Hot hot cold
Hot hot hot
Hot cold hot ….
How many possible hidden state sequences are there for this sequence?
How about in general for an HMM with N hidden states and a sequence of T observations? NT
So we can’t just do separate computation for each hidden state sequence.

42. Instead: the Forward algorithm
A kind of dynamic programming algorithm
Just like Minimum Edit Distance
Uses a table to store intermediate values
Idea:
Compute the likelihood of the observation sequence
By summing over all possible hidden state sequences
But doing this efficiently
By folding all the sequences into a single trellis

43. The forward algorithm P(o1, o2 … oT, qT = qF | l)
The goal of the forward algorithm is to compute
P(o1, o2 … oT, qT = qF | l)
We’ll do this by recursion

44. at(j) = P(o1, o2 … ot, qt = j | l)
The forward algorithm
Each cell of the forward algorithm trellis at(j)
Represents the probability of being in state j
After seeing the first t observations
Given the automaton
Each cell thus expresses the following probability
at(j) = P(o1, o2 … ot, qt = j | l)

45. The Forward Recursion

46. The Forward Trellis

47. We update each cell

48. The Forward Algorithm

49. Decoding Given an observation sequence And an HMM
3 1 3
And an HMM
The task of the decoder
To find the best hidden state sequence
Given the observation sequence O=(o1o2…oT), and an HMM model  = (A,B), how do we choose a corresponding state sequence Q=(q1q2…qT) that is optimal in some sense (i.e., best explains the observations)

50. Decoding One possibility: Why not? Instead:
For each hidden state sequence Q
HHH, HHC, HCH,
Compute P(O|Q)
Pick the highest one
Why not? NT
Instead:
The Viterbi algorithm
Is again a dynamic programming algorithm
Uses a similar trellis to the Forward algorithm

51. Viterbi intuition
We want to compute the joint probability of the observation sequence together with the best state sequence

52. Viterbi Recursion

53. The Viterbi trellis

54. Viterbi intuition Process observation sequence left to right
Filling out the trellis
Each cell:

55. Viterbi Algorithm

56. Viterbi backtrace

57. Training a HMM
Forward-backward or Baum-Welch algorithm (Expectation Maximization)
Backward probability (prob. of observations from t+1 to T)
bt(i) = P(ot+1, ot+2…oT | qt = i, l)
bT(i) = ai,F 1  i  N

58. function FORWARD-BACKWARD(observations of len T, output vocabulary V, hidden state set Q) returns HMM=(A,B) initialize A and B iterate until convergence E-step M-step return A, B

59. Hidden Markov Models for Part of Speech Tagging

60. Part of speech tagging 8 (ish) traditional English parts of speech
Noun, verb, adjective, preposition, adverb, article, interjection, pronoun, conjunction, etc.
This idea has been around for over 2000 years (Dionysius Thrax of Alexandria, c. 100 B.C.)
Called: parts-of-speech, lexical category, word classes, morphological classes, lexical tags, POS
We’ll use POS most frequently
Assuming that you know what these are

61. POS examples N noun chair, bandwidth, pacing
V verb study, debate, munch
ADJ adj purple, tall, ridiculous
ADV adverb unfortunately, slowly,
P preposition of, by, to
PRO pronoun I, me, mine
DET determiner the, a, that, those

62. POS Tagging example the DET koala N put V the DET keys N on P table N
WORD tag
the DET
koala N
put V
the DET
keys N
on P
table N

63. POS Tagging Words often have more than one POS: back
The back door = JJ
On my back = NN
Win the voters back = RB
Promised to back the bill = VB
The POS tagging problem is to determine the POS tag for a particular instance of a word.
These examples from Dekang Lin

64. POS tagging as a sequence classification task
We are given a sentence (an “observation” or “sequence of observations”)
Secretariat is expected to race tomorrow
She promised to back the bill
What is the best sequence of tags which corresponds to this sequence of observations?
Probabilistic view:
Consider all possible sequences of tags
Out of this universe of sequences, choose the tag sequence which is most probable given the observation sequence of n words w1…wn.

65. Getting to HMM
We want, out of all sequences of n tags t1…tn the single tag sequence such that P(t1…tn|w1…wn) is highest.
Hat ^ means “our estimate of the best one”
argmaxx f(x) means “the x such that f(x) is maximized”

66. Getting to HMM
This equation is guaranteed to give us the best tag sequence
But how to make it operational? How to compute this value?
Intuition of Bayesian classification:
Use Bayes rule to transform into a set of other probabilities that are easier to compute

67. Using Bayes Rule

68. Likelihood and prior n

69. Two kinds of probabilities (1)
Tag transition probabilities P(ti|ti-1)
Determiners likely to precede adjs and nouns
That/DT flight/NN
The/DT yellow/JJ hat/NN
So we expect P(NN|DT) and P(JJ|DT) to be high
But P(DT|JJ) to be low
Compute P(NN|DT) by counting in a labeled corpus:

70. Two kinds of probabilities (2)
Word likelihood probabilities P(wi|ti)
VBZ (3sg Pres verb) likely to be “is”
Compute P(is|VBZ) by counting in a labeled corpus:

71. An Example: the verb “race”
Secretariat/NNP is/VBZ expected/VBN to/TO race/VB tomorrow/NR
People/NNS continue/VB to/TO inquire/VB the/DT reason/NN for/IN the/DT race/NN for/IN outer/JJ space/NN
How do we pick the right tag?

72. Disambiguating “race”

73. ML Estimation P(NN|TO) = .00047 P(VB|TO) = .83 P(race|NN) = .00057
P(race|VB) =
P(NR|VB) = .0027
P(NR|NN) = .0012
P(VB|TO)P(race|VB)P(NR|VB) =
P(NN|TO)P(race|NN)P(NR|NN) =
So we (correctly) choose the verb reading

74. HMM for PoS tagging
Transitions probabilities A between the hidden states: tags

75. B observation likelihoods for POS HMM
Emission probabilities B: words

76. The A matrix for the POS HMM

77. The B matrix for the POS HMM

78. Viterbi intuition: we are looking for the best ‘path’
Slide from Dekang Lin

79. Viterbi example

80. Outline Markov Chains Hidden Markov Models Three Algorithms for HMMs
The Forward Algorithm
The Viterbi Algorithm
The Baum-Welch (EM Algorithm)
Applications:
The Ice Cream Task
Part of Speech Tagging
Next time: Named Entity Tagging