2. Communications & Multimedia Signal Processing Formant Track Restoration in Train Noisy Speech Qin Yan Communication & Multimedia Signal Processing Group Dept of Electronic & Computer Engineering, Brunel University 25 May, 2004

3. Communications & Multimedia Signal Processing Main Progress Restore the formant tracks from the noisy speech. Initial progress of the speech enhancement system

4. Communications & Multimedia Signal Processing Formant Tracking by 2D HMM in Noise Conditions SNRF1F2F3F4F5 051.312.56.33.72.6 5429.74.62.71.8 1032.37.43.421.4 1523.15.82.61.51.1 2015.64.62.11.21 Table : Average errors (%) of formant tracks in train noisy speech by 2D HMM at different SNR conditions 2D HMM is not robust to formant tracking in noise conditions

5. Communications & Multimedia Signal Processing LP Based Formant Tracking Noise Model LP-based Spectral Subtraction Formant Candidates Selection LP Pole Analysis Kalman Filter based Formant Tracker Noisy Speech Formant tracks VAD Figure : Procedure of LP formant Tracking High LP order is to over-model the LP spectrum to split the poles from formants and noise. Formant candidate selection rejects spurious candidates. Kalman filter smoothes formant tracks. Formant tracks are fed back to reclassification according to the distance to the initial tracks Reclassifier

6. Communications & Multimedia Signal Processing LP Spectral Subtraction Noise is modelled by a low LP order but speech is modelled by a high order. Computation efficiency Disadvantage : Noise variance absence. A hard-decision needs to be employed to avoid the subtracted values going below a noise-floor. The spectral trajectory across time is not modeled and used in the denoising process. If> other

7. Communications & Multimedia Signal Processing Performance of LP Spectra Subtraction Figure : Improvement by LP spectra subtraction Note : Improvement is calculated between average frame SNRs as:

8. Communications & Multimedia Signal Processing LPC Spectrogram of speech in noisy train (SNR= 0) LPC Spectrogram of Speech in noisy train after spectral subtraction Performance I

9. Communications & Multimedia Signal Processing R is the measurement covariance matrix, updated by variance of differences between noisy observation and estimated tracks. The process matrix Q is set to 0.16 experimentally. Kalman Filter Time Update Equations Measurement Update Equations “CORRECT” “PREDICT”

10. Communications & Multimedia Signal Processing Performance II Figure : Comparison of clean formant tracks (solid) and cleaned formant tracks (dash dot) and noisy formant tracks (dot). SNR=0Cleaned F151.318.1 F212.511.8 F36.36.2 F43.72.7 F52.62.5 Table : Average errors (%) of formant tracks in train noisy speech and cleaned speech.

11. Communications & Multimedia Signal Processing Noise Model LP-based Spectral Subtraction Formant Candidates Selection LP Pole Analysis Kalman Filter based Formant Tracker Noisy Speech Formant tracks VAD Reclassifier Wiener Filter Speech Reconstruction Enhanced Speech Initial Speech Enhancement system Initial Speech Enhancement System

12. Communications & Multimedia Signal Processing Speech enhancement with restored formant trajectories Future Work Noise Model LP-based Spectral Subtraction Formant Candidates Selection LP Pole Analysis Kalman Filter based Formant Tracker Noisy Speech Formant tracks VAD Reclassifier Wiener Filter Speech Reconstruction Enhanced Speech Initial Speech Enhancement system Pitch Track Restoration Residual

13. Communications & Multimedia Signal Processing Speech enhancement with restored formant trajectories Future Work Noise Model LP-based Spectral Subtraction Formant Candidates Selection LP Pole Analysis Kalman Filter based Formant Tracker Noisy Speech Formant tracks VAD Reclassifier Wiener Filter Speech Reconstruction Enhanced Speech Speech Enhancement System Pitch Track Restoration Residual Formant Tracks Restoration System

14. Communications & Multimedia Signal Processing The End