The Viterbi Algorithm Application of Dynamic Programming-the Principle of Optimality -Search of Citation Index -213 references since 1998 Applications Telecommunications Convolutional codes-Trellis codes Inter-symbol interference in Digital Transmission Continuous phase transmission Magnetic Recording-Partial Response Signaling- Divers others Image restoration Rainfall prediction Gene sequencing Character recognition

Milestones Viterbi (1967) decoding convolutional codes Omura (1968) VA optimal Kobayashi (1971) Magnetic recording Forney (1973) Classic survey recognizing the generality of the VA Rabiner (1989) Influential survey paper of hidden Markov chains

Example-Principle of Optimality Professor X chooses an optimum path on his trip to lunch Find optimal path to each bridge Perish Publish 1.2 N .2 .7 1.2 .5 N Faculty Club 1.2 .5 .3 EE Bld Optimal: 6 adds Brute force:8 adds N bridges Optimal: 4(N+1) adds Brute force: (N-1)2N adds In order to carry out the optimization, six additions are necessary; whereas, the brute force approach would require eight additions. If there were N streams to cross, each with two bridges, dynamic programming would require 4(N-1)+2 additions and the brute-force (N-1)2N. .5 S .8 1.0 .8 S .8

Digital Transmission with Convolutional Codes Information Source Convolutional Encoder BSC p p Information Sink Viterbi Algorithm

Maximum a Posteriori (MAP) Estimate A brute-force algorithm would lead to exponential growth with the length of the sequence. Brute force = Exponential Growth with N

Convolutional codes-Encoding a sequence (output,input) efficiency=input/output Example(3,1) code Output 111 100 010 110 011 001 000 Input 110100 T T The optimization is carried out by utilizing the structure of the code. This is like all good decoding algorithms.

Fig.2.14

Trellis Representation State output Next state 0 input s1s2 1 input 00 01 10 11 000 111 00 01 10 11 001 110 011 100 010 101

Iteration for Optimization

Key step! Redundant Linear growth in N

Deciding Previous State State i-1 State i 00 00 1 4 4 Search previous states 10 2 2

Viterbi Algorithm-shortest path to detect sequence First step s0 s1 s2 s3 Trace though successive states shortest path-Hamming distance to s0 Trellis codes-Euclidean distance Optimum Sequence Detection

Inter-symbol Interference Transmitter Channel Equalizer VA Decisions

AWGN Channel-MAP Estimate

Viterbi Algorithm for ISI Magic Iteration State = number of symbols in memory

Magnetic flux passes over heads Differentiation of pulses Magnetic Recording Magnetization pattern Magnetic flux passes over heads Differentiation of pulses Controlled ISI Same model applies to Partial Response signaling Output Sample

Continuous Phase FSK odd number ½ cycles Whole cycles

Merges and State Reduction Optimal paths through trellis All paths merge Force merges to reduce complexity Computations order of (No states)2 Carry only high probability states

Blurring Analogous to ISI Image Restoration Input Pixel Effect of Blurring C. Miller,et al., J Opt Soc Am., Vol,17, No.2 Feb 2000 Blurring Analogous to ISI

Known state transitions Utilized for state reduction Row Scan VA for optimal row sequence Known state transitions And Decision Feed back Utilized for state reduction

Hidden Markov Chain Data suggests Markovian structure Estimate initial state probabilities Estimate transition probabilities VA used for estimation of Probabilities Iteration l. Rabiner, Proc IEEE, Vol 77, N0 2, Feb 1989

Rainfall Prediction No rain Rainfall observations Rainy wet Rainy dry Transition probabilities estimated from Rainfall data using VA. No rain Showery wet Showery dry J. Sansom, J. of Climate, Vol. 11, Jan 1998 Rainfall observations

DNA Sequencing Nucleotide sequence CGGATTC DNA-double helix Sequences of four nucleotides, A,T,C and G Pairing between strands Bonding Genes Made up of Cordons, i.e. triplets of adjacent nucleotides Overlapping of genes Nucleotide sequence CGGATTC M.Y. Borodovskii et al. J of Computer and System Sciences, Vol. 39, No.1, pp1555 Gene 1 Cordon A in three genes Gene 2 Gene 3

Hidden Markov Chain Tracking genes P1 S-start first cordon of gene P1-4- +1,…,+4 from start G- Gene E-stop H-gap M1-4 -1,…,-4 from start M2 P2 M3 P3 Initial and Transition Probabilities known M4 P4 . G H E

Recognizing Handwritten Chinese Characters Text-line images Estimate stroke width Set up m X n grid Estimate initial and transition probabilities Detect possible segmentation paths by VA The image is scanned to estimate the stroke width. Next Slide Results

Example Segmenting Handwritten Characters Eliminating Redundant Paths All possible segmentation paths Removal of Overlapping paths Y-H Tseng and H-J Lee Pattern Recognition Letters, (20) 1999, pp791 Discarding near paths