2. SOME SIMPLE MANIPULATIONS OF SOUND USING DIGITAL SIGNAL PROCESSING Richard M. Stern 18-491 demo January 15, 2015 Department of Electrical and Computer Engineering and School of Computer Science Carnegie Mellon University Pittsburgh, Pennsylvania 15213

3. Carnegie Mellon Slide 218-791 Digital Signal Processing I The original sound and its spectrogram

4. Carnegie Mellon Slide 318-791 Digital Signal Processing I Downsampling the waveform Downsampling the waveform by factor of 2:

5. Carnegie Mellon Slide 418-791 Digital Signal Processing I Consequences of downsampling Original: Downsample Downsampled:

6. Carnegie Mellon Slide 518-791 Digital Signal Processing I Upsampling the waveform Upsampling by a factor of 2:

7. Carnegie Mellon Slide 618-791 Digital Signal Processing I Consequences of upsampling Original: Upsampled:

8. Carnegie Mellon Slide 718-791 Digital Signal Processing I Linear filtering the waveform x[n] y[n] Filter 1: y[n] = 3.6y[n–1]+5.0y[n–2]–3.2y[n–3]+.82y[n–4] +.013x[n]–.032x[n–1]+.044x[n–2]–.033x[n–3]+.013x[n–4] Filter 2: y[n] = 2.7y[n–1]–3.3y[n–2]+2.0y[n–3–.57y[n–4] +.35x[n]–1.3x[n–1]+2.0x[n–2]–1.3x[n–3]+.35x[n–4]

9. Carnegie Mellon Slide 818-791 Digital Signal Processing I Filter 1 in the time domain

10. Carnegie Mellon Slide 918-791 Digital Signal Processing I Output of Filter 1 in the frequency domain Original: Lowpass:

11. Carnegie Mellon Slide 1018-791 Digital Signal Processing I Filter 2 in the time domain

12. Carnegie Mellon Slide 1118-791 Digital Signal Processing I Output of Filter 2 in the frequency domain Original: Highpass:

13. Carnegie Mellon Slide 1218-791 Digital Signal Processing I The source-filter model of speech A useful model for representing the generation of speech sounds: Pitch Pulse train source Noise source Vocal tract model Amplitude p[n]

14. Carnegie Mellon Slide 1318-791 Digital Signal Processing I Original speech: Speech with 75-Hz excitation: Speech with 150-Hz excitation: Speech with noise excitation: Separating the vocal-tract excitation from the filter

15. Carnegie Mellon Slide 1418-791 Digital Signal Processing I Some Research Foci in ECE Processing, analysis, and compression of static and moving video Optical signal processing techniques Signal processing for digital data storage Speech recognition and understanding Multimedia fusion of video and audio information Architecture and protocols of telecommunications and computer networks

16. Carnegie Mellon Slide 1518-791 Digital Signal Processing I Approach of Acero, Liu, Moreno, et al. (1990-1997)… Compensation achieved by estimating parameters of noise and filter and applying inverse operations “Clean” speech x[m] h[m] n[m] z[m] Linear filtering Degraded speech Additive noise Classical signal enhancement: compensation of speech for noise and filtering

17. Carnegie Mellon Slide 1618-791 Digital Signal Processing I “Classical” combined compensation improves accuracy in stationary environments Threshold shifts by ~7 dB Accuracy still poor for low SNRs CMN (baseline) Complete retraining VTS (1997) CDCN (1990) –7 dB 13 dB Clean Original “Recovered”

18. Carnegie Mellon Slide 1718-791 Digital Signal Processing I Another type of signal enhancement: adaptive noise cancellation Speech + noise enters primary channel, correlated noise enters reference channel Adaptive filter attempts to convert noise in secondary channel to best resemble noise in primary channel and subtracts Performance degrades when speech leaks into reference channel and in reverberation

19. Carnegie Mellon Slide 1818-791 Digital Signal Processing I Simulation of noise cancellation for a PDA using two mics in “endfire” configuration Speech in cafeteria noise, no noise cancellation Speech with noise cancellation But …. simulation assumed no reverb

20. Carnegie Mellon Slide 1918-791 Digital Signal Processing I Signal separation: speech is quite intelligible, even when presented only in fragments Procedure: –Determine which time-frequency time- frequency components appear to be dominated by the desired signal –Reconstruct signal based on “good” components A Monaural example: –Mixed signals - –Separated signals -

21. Carnegie Mellon Slide 2018-791 Digital Signal Processing I Practical signal separation: Audio samples using selective reconstruction based on ITD RT60 (ms) 0 300 No Proc Delay-sum ZCAE-bin ZCAE-cont

22. Carnegie Mellon Slide 2118-791 Digital Signal Processing I Summary Lots of interesting topics that extend core material from DSP Greater emphasis on implementation and applications Greater emphasis on statistically-optimal signal processing I hope that you have as much fun with this material as I have had!

23. Carnegie Mellon Slide 2218-791 Digital Signal Processing I Academic integrity (i.e. cheating and plagiarism) CMU’s take on academic integrity: –http://www.cmu.edu/policies/documents/Cheating.htmlhttp://www.cmu.edu/policies/documents/Cheating.html Most important rule: Don’t cheat! But what do we mean by that? –Discussing general strategies on homework with other students is OK –Solving homework together is NOT OK –Accessing material from previous years is NOT OK –“Collaborating” on exams is REALLY REALLY NOT OK!