2. 1 A Novel Approach to Speech Coding After Time Scale Modification Presented by, H. Gokhan Ilk, Ph.D

3. Speech CodingFIT, Brno University of Technology2 Something about the presenter B.Sc, Ankara University Electronics Eng. Dept. M.Sc. Instrument Design & Applications UMIST, University of Manchester, Institute of Science and Technology, UK

4. Speech CodingFIT, Brno University of Technology3 Ph.D DCT Based Prototype Interpolation Speech Coding University of Manchester, UK Something about the presenter

5. Speech CodingFIT, Brno University of Technology4 Where is the Department?

6. Speech CodingFIT, Brno University of Technology5 Contact Details Address : Ankara University, Faculty of Engineering Electronics Engineering Department Beşevler 06100 Ankara, Turkey ilk@ieee.org

7. Speech CodingFIT, Brno University of Technology6 Medical Doctors are more interested in this figure Speech Production LUNGS

8. Speech CodingFIT, Brno University of Technology7 How does it look like? This figure is more interesting for a DSP course/seminar Long term correlation Short term correlation

9. Speech CodingFIT, Brno University of Technology8 How does it look like? Speech can be generally classified as Voiced or Unvoiced. Voiced part is a quasi- periodic (almost periodic) signal with higher energy and less zero crossing. Unvoiced part is a noise like signal

10. Speech CodingFIT, Brno University of Technology9 a)Voiced PSD: Power Spectrum Density b) Unvoiced How does it look like in the freqency domain

11. Speech CodingFIT, Brno University of Technology10 Now is a good time for maths Anyone heard of Wiener Filter Theory, Optimal Filtering Convolution sum Wiener filter turns out to be an FIR filter with N coefficients

12. Speech CodingFIT, Brno University of Technology11 Optimal Filtering Error is the difference between our signal and optimal estimate

13. Speech CodingFIT, Brno University of Technology12 Prediction as an Optimum Filtering Problem

14. Speech CodingFIT, Brno University of Technology13 LPC Analysis Filter + Linear Prediction Filter -

15. Speech CodingFIT, Brno University of Technology14 The AR (Auto Regressive) Model Considering optimum filter theory and regression analysis, since both independent and dependent variables belong to the same random process, x, x[n] is called an autoregressive or AR process. That is the process is regressed upon itself. Thanks to the people from Statistics, who called this analysis regression analysis of time series, long long time ago.

16. Speech CodingFIT, Brno University of Technology15 Innovations representation H(z)=A(z)H -1 (z)=1/A(z) From Linear System Theory The inverse system has many advantages. 1.In communications (left and right systems are apart) 2.The system on the right does not need any input ???

17. Speech CodingFIT, Brno University of Technology16 Innovations representation Innovations representation is basically an inverse system. Why called innovations?? Assume that x, our discrete random signal is speech. It can be either voiced, which means it is quasi-periodic or unvoiced, then it is noise. If x is voiced, then LPC analysis works very well and e[n] is close to zero If x is unvoiced, then LPC analysis works well again because e[n] is white noise In any case we do not need e[n] and thus the filters themselves present the information. That is why the representation is called INNOVATIONS.

18. Speech CodingFIT, Brno University of Technology17 What are these filters? Linear Prediction Synthesis Filter A -1 (z) E(z)X(z)

19. Speech CodingFIT, Brno University of Technology18 What are these filters? Finally LPC Analysis and Synthesis Filters A(z) LPC analyses filter, 1/A(z) LPC synthesis filter The filter in the AR model is therefore an IIR filter and AR model is therefore said to be an “all pole” model Useful information for statisticians

20. Speech CodingFIT, Brno University of Technology19 What is the deal with these filters??? Since 1/A(z) is a causal filter (does everybody see that???), this implies that it is minimum phase (It is causal stable (???) with a causal stable inverse) Since A(z) is an FIR filter, it is always stable and we know that it is causal. We also know that 1/A(z) is also causal. BUT IS IT ALWAYS STABLE??? We will now see that the a i (LPC coefficients) are found by solving Normal equations with a positive definite correlation function. Since they are found by solving a positive definite matrix inverse, the poles always lie within the unit circle...

21. Speech CodingFIT, Brno University of Technology20 How do we calculate LPC coefficients ? The problem is to determine the parameters a j, j=1,2,....p If  j : represents the estimates of a j then the error (or residual) is given by

22. Speech CodingFIT, Brno University of Technology21 It is now possible to determine the estimates by minimising the mean squared error, i.e. Setting the partial derivatives of Error with respect to  j to zero for j = 1,2,...,p, we get where E{.} is the expectation operator Derivatives again ?

23. Speech CodingFIT, Brno University of Technology22 That is, e(n) is orthogonal to s(n-i) for i = 1,2,...p. Equation can be rearranged to give Signal assumed stationary Solving the linear equation This is auto correlation? Or is it not?

24. Speech CodingFIT, Brno University of Technology23 Are we good with linear algabra? That is, e(n) is orthogonal to s(n-i) for i = 1,2,...p A x = b Obtained from University of Chicago web site

25. Speech CodingFIT, Brno University of Technology24 Auto-Correlation Method N : length of the sample sequence s n (m) = 0 outside the interval 0  m  N-1 Method I

26. Speech CodingFIT, Brno University of Technology25 Short time auto correlation Levinson- Durbin recursion

27. Speech CodingFIT, Brno University of Technology26 Covariance Method It requires the use of the samples in the interval -p  m  N-1 Method II

28. Speech CodingFIT, Brno University of Technology27 Covariance Method Symmetric covariance matrix, Cheolesky decomposition

29. Speech CodingFIT, Brno University of Technology28 What is next??? Now that we have the LPC a i coefficients, we can present speech with a compact representation This further requires an efficient representation of the excitation (residual, error) signal. In fact for example optimum magnitude calculation of regularly spaced pulses for the excitation constitutes GSM (Global System for Mobile Communications)

30. Speech CodingFIT, Brno University of Technology29 State of Art Efficient quantization of LPC parameters (called LSP or LSF (line spectral frequencies or pairs) together with the efficient representation and quantization of the excitation results in today’s state of art voice coding. Examples: GSM, CELP (code excited linear prediction), MELP (mixed excited linear prediction) etc.

31. Speech CodingFIT, Brno University of Technology30 Anything novel and interesting? Linear predictive coding and efficient representation of the excitation signal attracted so much interest that these poor subjects had been beaten to death. Therefore one has to do A LOT in order to gain A LITTLE Or merge two different disciplines in a clever way. It turns out that Prof. Verhelst has already developed one of the most important tools in one of these disciplines.

32. Speech CodingFIT, Brno University of Technology31 What is the novelty? Since speech signal exhibit both short and long term correlation and LPC analysis removes most of the short term correlation, we can remove the long term correlation, i.e. get rid of long term redundancy. The key is not to disturb pitch and formant frequencies. A detailed investigation of these parameters could be found in: W. Verhelst, “Overlap-add methods for time-scaling of speech”, Speech Commun. 30 (2000) 207–221.

33. Speech CodingFIT, Brno University of Technology32 How does it work? If pitch and formant frequencies are not disturbed by the WSOLA algorithm then one can compress speech (before coding) with a compression rate of beta and then expand the decoded speech at the receiver side with an expansion factor of 1/beta. If for example beta=0.5, then one can have a full duplex channel at a half duplex bandwidth. Why? Because the same signal is represented at half duration with minimum distortion.

34. Speech CodingFIT, Brno University of Technology33 Waveform Similarity Overlap and ADD

35. Speech CodingFIT, Brno University of Technology34 How does it work? U=N/2 No rate change (WSOLA  =1) U =1) U>N/2 Speech speeds up, compression (WSOLA  <=1) This is for 50% overlapping frames. A good way to test the algorithm::: Compress with  =1 and expand with  =1

36. Speech CodingFIT, Brno University of Technology35 Is that it ??? We have tried this approach with many different algorithms operating in time and frequency domains. Our experiments with the new NATO standard, Stanag 4591, MELP (mixed excitation linear predictive vocoder) indeed proved that WSOLA produces high quality output and it is computationally efficient. Details can be found H.G. Ilk, S. Tugac, “Channel and source considerations of a bit rate reduction technique for a possible wireless communications system’s performance enhancement”, IEEE Trans. Wireless Commun. vol. 4(1), January 2005, pp. 93–99 Not the other way around !!! But what if we would like to make most of our bandwidth? Then the system should be adaptive. It means WSOLA should operate at different time compression factors. This is an engineer’s dream come true. You dont operate at constant or multi-rate bit rates but you operate at flexible bit rates. That is YOU tell me how much bandwidth you got and I give tou the best quality possible. Not the other way around !!!

37. Speech CodingFIT, Brno University of Technology36 We are more clever than that Up to this point we are only using Werner’s WSOLA algorithm, that has been developed for hearing disabled. What is we want to change beta seamlessly. How do we do that? To change beta, you can either change U or N. Restrictions::: Frame size (N) should not change at the transmitter, during compression. That is determined by your codec and it is standard.

38. Speech CodingFIT, Brno University of Technology37 What is the extension then?? Different beta as we proceed, Compression As you can see from solid black lines N is constant. As you can see from dashed blue lines U changes for each frame

39. Speech CodingFIT, Brno University of Technology38 Half symmetric windows in order to go back to the original time scale Expansion During synthesis at the receiver, N has to change for synchronous output speech

40. Speech CodingFIT, Brno University of Technology39 What is the originality? This approach is particularly useful in packet switching network applications like VoIP (Voice Over IP) in dynamic networks because the load may change abruptly and it is not symmetric at each direction. That is novel is it NOT???) It is also equally valuable in circuit switching congested voice networks because today’s networks either allow multi-rates (2.4, 4.8 or 8.0 kb/s) or simply drops your call. This will allow priority in phone calls or cheaper tariffs leading to QoS in a circuit switching network (That is novel is it NOT???) Details can be found in Hakkı Gökhan İlk and Saadettin Güler, “Adaptive time scale modification of speech for graceful degrading voice quality in congested networks for VoIP applications”, Signal Processing, Volume 86, pp 127-139, 2006

41. Speech CodingFIT, Brno University of Technology40 Samples Male “Steve wore a bright red cashmere sweater” Female “Before Thursday’s exam review every formula” 2.4 kb/s 1.0 kb/s 128 kb/s PCM 2.4 kb/s 1.0 kb/s 128 kb/s PCM

42. Speech CodingFIT, Brno University of Technology41 Reward! Our algorithm has been selected as one of the two finalists in a competition by TURKCELL (a GSM giant in Turkey). We hope to win the competition by our presentation and demo on 28 September.

43. Speech CodingFIT, Brno University of Technology42 I would like to thank Honza and FIT for making this exchange possible