1. 1 Introduction1 Introduction 2 Spectral subtraction 3 QBNE 4 Results 5 Conclusion, & future work2 Spectral subtraction 3 QBNE4 Results5 Conclusion, & future work IIT Bombay P.C. Pandey / EE Dept / IIT Bombay Enhancement of Electrolaryngeal Speech by Reducing Leakage Noise Using Spectral Subtraction by Prem C. Pandey EE Dept, IIT Bombay Electro Info Com’2007 / St Francis Inst. of Technology, Mumbai / 4-6 Jan’07

2. 1 Introduction1 Introduction 2 Spectral subtraction 3 QBNE 4 Results 5 Conclusion, & future work2 Spectral subtraction 3 QBNE4 Results5 Conclusion, & future work IIT Bombay P.C. Pandey / EE Dept / IIT Bombay Abstract Transcervical electrolarynx is a vibrator held against the neck tissue in order to provide excitation to the vocal tract, as a substitute to that provided by a natural larynx. It is of great help in verbal communication to a large number of laryngectomee patients. Its intelligibility suffers from the presence of a background noise, caused by leakage of the acoustic energy from the vibrator. Pitch synchronous application of spectral subtraction method, normally used for enhancement of speech corrupted by uncorrelated random noise, can be used for reduction of the self leakage noise for enhancement of electrolaryngeal speech. Average magnitude spectrum of leakage noise, obtained with lips closed, is subtracted from the magnitude spectrum of the noisy speech and the signal is reconstructed using the original phase spectrum. However, the spectrum of the leakage noise varies because of variation in the application pressure and movement of the throat tissue. A quantile based dynamic estimation of the magnitude spectrum without the need for silence/voice detection was found to be effective in noise reduction. 2

3. 1 Introduction1 Introduction 2 Spectral subtraction 3 QBNE 4 Results 5 Conclusion, & future work2 Spectral subtraction 3 QBNE4 Results5 Conclusion, & future work IIT Bombay P.C. Pandey / EE Dept / IIT Bombay Overview ● Introduction ● Spectral subtraction for enhancement of electrolaryngeal speech ● Quantile-based noise estimation ● Results, summary, & ongoing work 3

4. 1 Introduction1 Introduction 2 Spectral subtraction 3 QBNE 4 Results 5 Conclusion, & future work2 Spectral subtraction 3 QBNE4 Results5 Conclusion, & future work IIT Bombay P.C. Pandey / EE Dept / IIT Bombay Natural speech production Introduction 1/5 Glottal excitation to vocal tract 4

5. 1 Introduction1 Introduction 2 Spectral subtraction 3 QBNE 4 Results 5 Conclusion, & future work2 Spectral subtraction 3 QBNE4 Results5 Conclusion, & future work IIT Bombay P.C. Pandey / EE Dept / IIT Bombay External electronic larynx (Barney et al 1959) Excitation to vocal tract from external vibrator Introduction 2/5 5

6. 1 Introduction1 Introduction 2 Spectral subtraction 3 QBNE 4 Results 5 Conclusion, & future work2 Spectral subtraction 3 QBNE4 Results5 Conclusion, & future work IIT Bombay P.C. Pandey / EE Dept / IIT Bombay Problems with artificial larynx ● Difficulty in coordinating controls ● Spectrally deficit ● Unvoiced segments substituted by voiced segments ● Background noise due to leakage of acoustic energy Introduction 3/5 6

7. 1 Introduction1 Introduction 2 Spectral subtraction 3 QBNE 4 Results 5 Conclusion, & future work2 Spectral subtraction 3 QBNE4 Results5 Conclusion, & future work IIT Bombay P.C. Pandey / EE Dept / IIT Bombay Model of noise generation Causes of noise generation : Leakage of vibrations produced by vibrator membrane Improper coupling of vibrator to neck tissue Introduction 4/5 7

8. 1 Introduction1 Introduction 2 Spectral subtraction 3 QBNE 4 Results 5 Conclusion, & future work2 Spectral subtraction 3 QBNE4 Results5 Conclusion, & future work IIT Bombay P.C. Pandey / EE Dept / IIT Bombay Methods of noise reduction Vibrator design  Acoustic shielding of vibrator ( Epsy-Wilson et al 1996)  Piezoelectric vibrators (Katsutoshi et al 1999) Signal processing  2-input noise cancellation based on LMS algorithm ( Epsy-Wilson et al 1996)  Single input noise cancellation ( Pandey et al 2002) based on spectral subtraction algorithm (Boll 1979 & Berouti et al 1979) Introduction 5/5 8

9. 1 Introduction1 Introduction 2 Spectral subtraction 3 QBNE 4 Results 5 Conclusion, & future work2 Spectral subtraction 3 QBNE4 Results5 Conclusion, & future work IIT Bombay P.C. Pandey / EE Dept / IIT Bombay Spectral subtraction for enhancement of electrolayngeal speech (Pandey et al 2000) s(n) = e(n)*h v (n), l(n) = e(n)*h l (n) x(n) = s(n) + l(n) X n (e j  ) = E n (e j  )[H v n (e j  ) + H l n (e j  )] Assumption: h v (n) and h l (n) uncorrelated   X n (e j  )  2 =  E n (e j  )  2 [  H v n (e j  )  2 +  H l n (e j  )  2 ] Noise estimation mode: s(n) = 0  X n (e j  )  2 =  L n (e j  )  2 =  E n (e j  )  2  H l n (e j  )  2  L(e j  )  2 : averaged over many segments Speech enhancement mode:  Y n (e j  )  2 =  X n (e j  )  2 -  L(e j  )  2 contd… Spect. subtrn. 1/4 9

10. 1 Introduction1 Introduction 2 Spectral subtraction 3 QBNE 4 Results 5 Conclusion, & future work2 Spectral subtraction 3 QBNE4 Results5 Conclusion, & future work IIT Bombay P.C. Pandey / EE Dept / IIT Bombay Implementation using DFT  Y n (k)  2 =  X n (k)  2 -  L(k)  2 y n (m) = IDFT [  Y n (k)  e j  X n (k)] Modified spectral subtraction (Berouti et al 1979)  Y n (k)   =  X n (k)   -  L(k)    Y n (k)   =  Y n (k)   if  Y n (k)     L(k)   =  L(k)   otherwise (  : subtraction,  : spectral floor,  : exp. factors) Output normalization factor for  < 1 (Berouti et al 1979) G = {(  X n (k)  2 -  L(k)  2 )/  Y n (k)  2 }/  Spect. subtrn. 2/4 10

11. 1 Introduction1 Introduction 2 Spectral subtraction 3 QBNE 4 Results 5 Conclusion, & future work2 Spectral subtraction 3 QBNE4 Results5 Conclusion, & future work IIT Bombay P.C. Pandey / EE Dept / IIT Bombay Spectral subtraction method with ABNE (Pandey et al 2002) Spect. subtrn. 3/4 11

12. 1 Introduction1 Introduction 2 Spectral subtraction 3 QBNE 4 Results 5 Conclusion, & future work2 Spectral subtraction 3 QBNE4 Results5 Conclusion, & future work IIT Bombay P.C. Pandey / EE Dept / IIT Bombay Drawback of averaged noise estimation during silence ● Two modes: noise estimation & speech enhancement ● Estimated noise considered stationary over entire speech enhancement mode ● Some musical & broadband noise in the output Investigations for continuous noise estimation & signal enhancement ● System with voice activity detector (Berouti et al 1979) ● Without involving speech vs non-speech detection (Stahl et al 2000, Evans et al 2002, Houwu et al 2002) Spect. subtrn. 4/4 12

13. 1 Introduction1 Introduction 2 Spectral subtraction 3 QBNE 4 Results 5 Conclusion, & future work2 Spectral subtraction 3 QBNE4 Results5 Conclusion, & future work IIT Bombay P.C. Pandey / EE Dept / IIT Bombay Quantile-based noise estimation Basis for the technique ● During speech segments, frequency bins tend not to be permanently occupied by speech ● Speech / non-speech boundaries detected implicitly on per frequency basis ● Noise estimates updated throughout non- speech and speech periods QBNE 1/6 13

14. 1 Introduction1 Introduction 2 Spectral subtraction 3 QBNE 4 Results 5 Conclusion, & future work2 Spectral subtraction 3 QBNE4 Results5 Conclusion, & future work IIT Bombay P.C. Pandey / EE Dept / IIT Bombay Implementation of QBNE ● DFT of windowed speech segments ● FIFO array of past spectral values for each freq. sample is formed ● An efficient indexing algorithm used to sort the arrays to obtain particular quantile value: – A sorted value buffer and an index buffer, for each frequency sample – New data placed at locations of oldest data in sorted buffer by referring index buffer – In all sorted buffers only one value needs to be placed at correct position QBNE 2/6 14

15. 1 Introduction1 Introduction 2 Spectral subtraction 3 QBNE 4 Results 5 Conclusion, & future work2 Spectral subtraction 3 QBNE4 Results5 Conclusion, & future work IIT Bombay P.C. Pandey / EE Dept / IIT Bombay QBNE 3/4 Spectral subtraction with QBNE QBNE 3/6 10

16. 1 Introduction1 Introduction 2 Spectral subtraction 3 QBNE 4 Results 5 Conclusion, & future work2 Spectral subtraction 3 QBNE4 Results5 Conclusion, & future work IIT Bombay P.C. Pandey / EE Dept / IIT Bombay Investigations with QBNE ● Single quantile value - Quantile value which gives best visual match between quantile derived spect. & avg. spect. of noise is selected ● Two quantile values - Two quantiles for two frequency bands, which estimates noise close to avg. spect. of noise, were selected ● Frequency dependent quantile values - Estimated spectrum from noisy speech will be close match to the avg. spectrum of noise QBNE 4/6 16

17. 1 Introduction1 Introduction 2 Spectral subtraction 3 QBNE 4 Results 5 Conclusion, & future work2 Spectral subtraction 3 QBNE4 Results5 Conclusion, & future work IIT Bombay P.C. Pandey / EE Dept / IIT Bombay QBNE 5/6 Investigations with QBNE (Contd..) ● Smoothened quantile values - Matched quantiles were averaged using 9 frequency values ● SNR based dynamic quantiles - Dynamic selection of quantiles depending on signal strength q(k) = [(q 1 (k) - q 0 (k)) SNR (k) / SNR 1 (k)] + q 0 (k) q 0 (k) if q (k) < 0 q 1 (k) if q (k) > q 1 (k) 17

18. 1 Introduction1 Introduction 2 Spectral subtraction 3 QBNE 4 Results 5 Conclusion, & future work2 Spectral subtraction 3 QBNE4 Results5 Conclusion, & future work IIT Bombay P.C. Pandey / EE Dept / IIT Bombay Plot of SNR and frequency dependent quantiles for three different applications of vibrator Frequency sample 18 QBNE 5/6

19. 1 Introduction1 Introduction 2 Spectral subtraction 3 QBNE 4 Results 5 Conclusion, & future work2 Spectral subtraction 3 QBNE4 Results5 Conclusion, & future work IIT Bombay P.C. Pandey / EE Dept / IIT Bombay Recorded and enhanced speech with (α=2,β=0.001,γ=1,N=16 ms), speaker: SP, material: /a/, /i/,and /u/ using electrolarynx Servox Noise segment /a/ /u/ /i/ UnprocessedProcessed Enhancement results 19 Results 1/3

20. 1 Introduction1 Introduction 2 Spectral subtraction 3 QBNE 4 Results 5 Conclusion, & future work2 Spectral subtraction 3 QBNE4 Results5 Conclusion, & future work IIT Bombay P.C. Pandey / EE Dept / IIT Bombay Enhancement results Results 2/3 Recorded and enhanced speech with (α=2,β=0.001,γ=1, Widow length=16 ms), speaker: SP, material: question-answer pair in English “ What is your name? My name is santosh” using electrolarynx Servox 20

21. 1 Introduction1 Introduction 2 Spectral subtraction 3 QBNE 4 Results 5 Conclusion, & future work2 Spectral subtraction 3 QBNE4 Results5 Conclusion, & future work IIT Bombay P.C. Pandey / EE Dept / IIT Bombay Recorded and enhanced speech with (α=2,β=0.001,γ=1), speaker: SP, material: question-answer pair in English “ What is your name? My name is santosh” using electrolarynx NP-1, Servox, and Solatone Results 3/3 Enhancement results 21

22. 1 Introduction1 Introduction 2 Spectral subtraction 3 QBNE 4 Results 5 Conclusion, & future work2 Spectral subtraction 3 QBNE4 Results5 Conclusion, & future work IIT Bombay P.C. Pandey / EE Dept / IIT Bombay Conclusion 1/2 Conclusion ● QBNE technique implemented for cont. updating of noise spectrum & different methods for selection of quantile values for noise estimation investigated ● Results with QBNE during non-speech segment are comparable with results using ABNE ● Smoothened quantiles and SNR based quantiles resulted in better quality speech ● QBNE is effective for longer duration ● QBNE using SNR based dynamic quantiles is effective during long pauses 22

23. 1 Introduction1 Introduction 2 Spectral subtraction 3 QBNE 4 Results 5 Conclusion, & future work2 Spectral subtraction 3 QBNE4 Results5 Conclusion, & future work IIT Bombay P.C. Pandey / EE Dept / IIT Bombay Ongoing work ● Evaluation of intelligibility and quality improvement ● Selection of optimum quantile values for different models of electrolarynx and users ● Phase resynthesis from magnitude spectrum using cepstral method ● Real-time implementation of noise reduction, using ADSP- BF533 board ● Analysis-synthesis for introducing small amount of jitter to improve naturalness Conclusion 2/2 23

24. 1 Introduction1 Introduction 2 Spectral subtraction 3 QBNE 4 Results 5 Conclusion, & future work2 Spectral subtraction 3 QBNE4 Results5 Conclusion, & future work IIT Bombay P.C. Pandey / EE Dept / IIT Bombay