1 MAXENT 2007 R. F. Astudillo, D. Kolossa and R. Orglmeister

2 MAXENT 2007 R. F. Astudillo, D. Kolossa and R. Orglmeister PROPAGATION OF STATISTICAL INFORMATION THROUGH NON-LINEAR FEATURE EXTRACTIONS FOR ROBUST SPEECH RECOGNITION Overview: 1.Introduction: Automatic speech recognition. 2.Problem: Imperfect noise suppression. 3.Proposed solution: Uncertainty propagation. 4.Tests & results. 5.Conclusions. R. F. Astudillo, D. Kolossa and R. Orglmeister - TU-Berlin

3 MAXENT 2007 R. F. Astudillo, D. Kolossa and R. Orglmeister Automatic Speech Recognizer (ASR) Feature extraction transforms signal into a domain more suitable for recognition. Speech recognizer models abstract speech components like phonemes or triphones, generates transcription. Most of speech recognition applications need noise suppression preprocessing.

4 MAXENT 2007 R. F. Astudillo, D. Kolossa and R. Orglmeister Non-linear transformations that imitate the way humans process speech. Robust against inter-speaker and intra-speaker variability. Mel-cepstral and RASTA-PLP transformations. Feature Extraction

5 MAXENT 2007 R. F. Astudillo, D. Kolossa and R. Orglmeister Speech Recognition Statistical models are used to model speech. Hidden Markov models with mixture of Gaussians (multivariable) for the emitting states. Example: Mel-cepstral features

6 MAXENT 2007 R. F. Astudillo, D. Kolossa and R. Orglmeister Noise Suppression MMSE-LSA bayesian estimation [Ephraim1985] is one of the most used. Leaves residual noise. Introduces artifacts in speech. Most methods obtain an estimation of the short-time spectrum (STFT) of the clean signal. Problem: Imperfect estimation.

7 MAXENT 2007 R. F. Astudillo, D. Kolossa and R. Orglmeister Solution: Modeling Uncertainty of Estimation We model each element of the STFT as a complex Gaussian random distribution. Mean set equal to estimated clean value. Parameter controls the uncertainty.

8 MAXENT 2007 R. F. Astudillo, D. Kolossa and R. Orglmeister Propagation of Uncertainty We propagate first and second order moments of the distributions. Correlation between feature appears (covariance). Resulting uncertainty is combined with statistical model parameters for robust speech recognition

9 MAXENT 2007 R. F. Astudillo, D. Kolossa and R. Orglmeister Propagation of Uncertainty We propagate first and second order moments of the distributions. Correlation between feature appears (covariance). Resulting uncertainty is combined with statistical model parameters for robust speech recognition

10 MAXENT 2007 R. F. Astudillo, D. Kolossa and R. Orglmeister Approaches to Uncertainty Propagation Analytic solutions Imply complex calculations. Specific for each transformation. Pseudo-Montecarlo Unscented Transform [Julier1996]. Inefficient for high number of dimensions (i.e. STFT 256 dim./frame). ► Piecewise Propagation Efficient combination of both methods. Valid for many feature extractions (i.e. MELSPEC, MFCC, RASTA-PLP).

11 MAXENT 2007 R. F. Astudillo, D. Kolossa and R. Orglmeister Piecewise Uncertainty Propagation Exemplified with Mel-Ceptral transformation: 1.Modulus extraction (non-linear). 2.Filterbank (linear). 3.Logarithm (non-linear). 4.Discrete-cosine-transform (linear). 5.Delta and acceleration coefficients (linear).

12 MAXENT 2007 R. F. Astudillo, D. Kolossa and R. Orglmeister Propagation through Modulus By integrating the phase of a complex Gaussian distribution we obtain the Rice distribution. Mean and variance can be calculated as: were L is the Legendre polynom.

13 MAXENT 2007 R. F. Astudillo, D. Kolossa and R. Orglmeister Propagation through filterbank Each filter output m is a weighted sum of frequency moduli. It can be expressed as a matrix multiplication. Mean and variance can be calculated as:

14 MAXENT 2007 R. F. Astudillo, D. Kolossa and R. Orglmeister Full Covariance and other linear transformations DCT, delta and acceleration can be computed similarly. Covariance after filterbank is no longer diagonal. Additional computation costs.

15 MAXENT 2007 R. F. Astudillo, D. Kolossa and R. Orglmeister Propagation through Logarithm Non-linear transformation Distribution after filterbank difficult to model not diagonal Dimesionality of the Mel features much smaller than the STFT features ► Unscented transform can be applied efficiently

16 MAXENT 2007 R. F. Astudillo, D. Kolossa and R. Orglmeister Unscented Transform Only points must be propagated. Points on the th covariace contour and the mean. = feature dimension Example for =2

17 MAXENT 2007 R. F. Astudillo, D. Kolossa and R. Orglmeister Unscented Transform II Mean and covariances are calculated by using weighted averages: Parameter allows higher moments of the distribution to be considered.

18 MAXENT 2007 R. F. Astudillo, D. Kolossa and R. Orglmeister Use of Uncertainty After propagation of uncertainty, missing feature techniques or uncertainty decoding may be applied. These techniques combine uncertainty and model information to ignore or reestimate noisy features. Parameters of state f1

19 MAXENT 2007 R. F. Astudillo, D. Kolossa and R. Orglmeister Use of Uncertainty II Modified imputation [Kolossa2005] showed the best performance. It reestimates features for state q by maximizing the probability: Assuming multivariate Gaussian distribution for uncertainty and model:

20 MAXENT 2007 R. F. Astudillo, D. Kolossa and R. Orglmeister Recognition Tests TI-DIGITS database % correct identified words WindnoiseStreetnoise Test TypeUncertainty-15dB5dB-15dB5dB Clean Speech ( )98.76 Noisy ( ) MMSE-LSA ( ) Aprox. uncertainty Ideal uncertainty files (20 different speakers). Best, second best results.

21 MAXENT 2007 R. F. Astudillo, D. Kolossa and R. Orglmeister Conclusions The use of uncertainty in Mel-cepstral domain is useful to compensate imperfect estimation during noise suppression. Piecewise uncertainty propagation is valid for multiple feature extractions. Better estimation of uncertainty should improve the results.

22 MAXENT 2007 R. F. Astudillo, D. Kolossa and R. Orglmeister Thank You! Some literature: [Ephraim1985] Y. Ephraim, and D. Malah, Acoustics, Speech, and Signal Processing, IEEE Transactions on 33, 443–445 (1985). [Julier1996] S. Julier, and J. Uhlmann, A general method for approximating nonlinear transformations of probability distributions, Tech. rep., University of Oxford, UK (1996). [Kolossa2005] D. Kolossa, A. Klimas, and R. Orglmeister, “Separation and robust recognition of noisy, convolutive speech mixtures using time-frequency masking and missing data techniques,” Applications of Signal Processing to Audio and Acoustics, IEEE Workshop on, 2005, pp