1. The Viterbi Algorithm
Application of Dynamic Programming-the Principle of Optimality
-Search of Citation Index -213 references since 1998
Applications
Telecommunications
Convolutional codes-Trellis codes
Inter-symbol interference in Digital Transmission
Continuous phase transmission
Magnetic Recording-Partial Response Signaling-
Divers others
Image restoration
Rainfall prediction
Gene sequencing
Character recognition

2. Milestones Viterbi (1967) decoding convolutional codes
Omura (1968) VA optimal
Kobayashi (1971) Magnetic recording
Forney (1973) Classic survey recognizing the generality of the VA
Rabiner (1989) Influential survey paper of hidden Markov chains

3. Example-Principle of Optimality
Professor X chooses an optimum
path on his trip to lunch
Find optimal path to each bridge
Perish
Publish
1.2 N .2 .7
1.2 .5 N
Faculty
Club
1.2 .5 .3
EE Bld
Optimal: 6 adds
Brute force:8 adds
N bridges
Optimal: 4(N+1) adds
Brute force: (N-1)2N adds
In order to carry out the optimization, six additions are necessary; whereas, the brute force approach would require eight additions. If there were N streams to cross, each with two bridges, dynamic programming would require 4(N-1)+2 additions and the brute-force (N-1)2N. .5 S .8
1.0 .8 S .8

4. Digital Transmission with Convolutional Codes
Information
Source
Convolutional
Encoder
BSC p p
Information
Sink
Viterbi
Algorithm

5. Maximum a Posteriori (MAP) Estimate
A brute-force algorithm would lead to exponential growth with the length of the sequence.
Brute force = Exponential Growth with N

6. Convolutional codes-Encoding a sequence
(output,input)
efficiency=input/output
Example(3,1) code
Output
Input
110100 T T
The optimization is carried out by utilizing the structure of the code. This is like all good decoding algorithms.

7. Fig.2.14

8. Trellis Representation
State output Next state
0 input
s1s2
1 input 00 01 10 11
000
111 00 01 10 11
001
110
011
100
010
101

9. Iteration for Optimization

10. Key step!
Redundant
Linear growth in N

11. Deciding Previous State
State i-1
State i 00 00 1 4 4
Search previous states 10 2 2

12. Viterbi Algorithm-shortest path to detect sequence
First step s0 s1 s2 s3
Trace though successive states
shortest path-Hamming distance to s0
Trellis codes-Euclidean distance
Optimum Sequence Detection

13. Inter-symbol Interference
Transmitter
Channel
Equalizer VA
Decisions

14. AWGN Channel-MAP Estimate

15. Viterbi Algorithm for ISI
Magic Iteration
State = number of symbols in memory

16. Magnetic flux passes over heads Differentiation of pulses
Magnetic Recording
Magnetization pattern
Magnetic flux passes over heads
Differentiation of pulses
Controlled ISI
Same model applies to
Partial Response signaling
Output
Sample

17. Continuous Phase FSK
odd number ½ cycles
Whole cycles

18. Merges and State Reduction
Optimal paths through trellis
All paths merge
Force merges to reduce complexity
Computations order of (No states)2
Carry only high probability states

19. Blurring Analogous to ISI
Image Restoration
Input Pixel
Effect of Blurring
C. Miller,et al., J Opt Soc Am., Vol,17, No.2 Feb 2000
Blurring Analogous to ISI

20. Known state transitions Utilized for state reduction
Row Scan
VA for optimal row sequence
Known state transitions
And Decision Feed back
Utilized for state reduction

21. Hidden Markov Chain Data suggests Markovian structure
Estimate initial state probabilities
Estimate transition probabilities
VA used for estimation of Probabilities
Iteration
l. Rabiner, Proc IEEE, Vol 77, N0 2, Feb 1989

22. Rainfall Prediction No rain Rainfall observations Rainy wet Rainy dry
Transition probabilities estimated
from Rainfall data using VA.
No rain
Showery
wet
Showery
dry
J. Sansom, J. of Climate, Vol. 11, Jan 1998
Rainfall observations

23. DNA Sequencing Nucleotide sequence CGGATTC DNA-double helix
Sequences of four nucleotides, A,T,C and G
Pairing between strands
Bonding
Genes
Made up of Cordons, i.e. triplets of adjacent nucleotides
Overlapping of genes
Nucleotide sequence
CGGATTC
M.Y. Borodovskii et al. J of Computer and System Sciences, Vol. 39, No.1, pp1555
Gene 1
Cordon A in three genes
Gene 2
Gene 3

24. Hidden Markov Chain Tracking genesP1
S-start first cordon of gene
P ,…,+4 from start
G- Gene
E-stop
H-gap
M1-4 -1,…,-4 from start M2 P2 M3 P3
Initial and
Transition
Probabilities known M4 P4 . G H E

25. Recognizing Handwritten Chinese Characters
Text-line images
Estimate stroke width
Set up m X n grid
Estimate initial and transition probabilities
Detect possible segmentation paths by VA
The image is scanned to estimate the stroke width.
Next Slide
Results

26. Example Segmenting Handwritten Characters
Eliminating
Redundant
Paths
All possible
segmentation
paths
Removal of
Overlapping
paths
Y-H Tseng and H-J Lee Pattern Recognition Letters, (20) 1999, pp791
Discarding
near paths