1. 1 Chapter 6 Basics of Digital Audio 6.1 Digitization of Sound 6.2 MIDI: Musical Instrument Digital Interface 6.3 Quantization and Transmission of Audio 6.4 Further Exploration

2. 2 Recall: Frequency Modulation Frequency (channels) f1f1 f2f2 f3f3 多媒體 傳輸 儲存 處理 Time domain Frequency domain Pixel domain Time domain 無直接關係無直接關係 訊號用調頻傳送 研究訊號本身的頻譜

3. 3 What is Sound A wave phenomenon like light Molecules of air being compressed and expanded under the action of some physical device pressure wave continuous values (before digitized) reflection ( 反射 ) refraction ( 折射 ) diffraction ( 繞射 )

4. 4 Interesting Titbits Typical Sampling Rates = 8k / 48k Hz Human voice  4K Hz. Human ear can hear  20 ~ 20K Hz. Nyquist Sampling Rate discussed later Musicology/ Octave/ Harmonics: note “A” (La) within middle C is 440 Hz. Octave above is another A note doubling the frequency, i.e., 880 Hz. any series of musical tones whose frequencies are integral multiples of the frequency of a fundamental tone.

5. 5 Ch 6: Basics of Digital Audio 6.1 Digitization of Sound 6.2 MIDI: Musical Instrument Digital Interface 6.3 Quantization and Transmission of Audio 6.4 Further Exploration

6. 6 Digitization Sampling Quantization

7. 7 Issues for Digital Audio Data What is the sampling rate? How finely is the data to be quantized, and is quantization uniform? How is audio data formatted? (file format)

8. 8 If a signal is band-limited, i.e., there is a lower limit f1 and an upper limit f2 of frequency components in the signal Sampling rate should be at least 2(f2 – f1). Usually, f1 is referred to as “ 0 ”. Nyquist Theorem

9. 9 Signal to Noise Ratio (SNR) A measure of the quality of the signal. In units of dB (decibel), 10dB= 1 bel Base-10 logarithms of the Ratio of (the power of the correct signal) and (the power of the noise) The higher the better Note: P=V 2 /R

10. 10 dB Applied to Common Sounds A ratio to the quietest sound The quietest sound capable of hearing i.e. the just audible sound with frequency 1KHz Def. 10 -5 N/m 2 The lower the better

11. 11 Signal to Quantization Noise Ratio SQNR, Quantization noise = round-off error Let quantization accuracy = N bits per sample The worst case SQNR = 6.02 N (dB) input signal is sinusoidal, the quantization error is statistically independent, SQNR = 6.02 N + 1.76 (dB) SNR (SQNR) > 70 Can be acceptable in general, i.e., We need N > 12

12. 12 Linear and Non-linear Quantization Linear format: samples are typically stored as uniformly quantized values. Non-uniform quantization: set up more finely- spaced levels where humans hear with the most acuity. Weber's Law stated formally says that equally perceived differences have values proportional to absolute levels : ΔResponse  ΔStimulus / Stimulus (6.5)

13. 13 Nonlinear Quantization Transforming an analog signal from the raw s space into the theoretical r space, and then uniformly quantizing the resulting values quantization of r giving finer resolution in s at the quiet end Called  -law encoding, (or u-law). A very similar rule, called A-law used in telephony in Europe.

14. 14 (6.9) (6.10) Equations of u-law and A-law

15. 15 Nonlinear Transform for audio signals Fig 6.6 s/s p r

16. 16 Audio Filtering Band-pass filter Prior to sampling and AD conversion Filter to remove unwanted frequencies (a) For speech, 50 ~ 10kHz is retained (b) For music signal, 20 ~ 20k Hz. Low-pass filter At the DA converter end, because high frequencies may reappear in the output sampling and quantization  smooth input signal is replaced by a series of step functions containing all possible frequencies. Involving Electronic Circuit, Not digital signal processor.

17. 17 Data rate and bandwidth in sample audio applications Table 6.2 Bytes 1/8[1,2,6] x 1/2, “ >= ”

18. 18 Recall: Digitization Sampling Quantization

19. 19 Pulse Code Modulation: PCM The basic coding method Producing quantized sampled output for audio The differences version: DPCM ( 差值脈碼調變 ) A crude but efficient variant (delta): DM. The adaptive version: ADPCM. Coding of Audio Example: WAV - PCM Skype - ADPCM, 32kbps

20. 20 Fig 6.13 Pulse Code Modulation: PCM (a)Original analog signal & corresponding PCM signals. (b) Decoded staircase signal. (c) Reconstructed signal after low-pass filtering.

21. 21 PCM in Telephony System 8-bit, 8 kHz  64 kbps (Compression) Nonlinear Quantization

22. 22 Recall: Nonlinear Quantization Fig 6.6 s/s p r

23. 23 Recall: SQNR SQNR, Quantization noise = round-off error Let quantization accuracy = N bits per sample The worst case SQNR = 6.02 N (dB) input signal is sinusoidal, the quantization error is statistically independent, SQNR = 6.02 N + 1.76 (dB) SNR (SQNR) > 70 Can be acceptable in general, i.e., We need N > 12 Nonlinear Q[] N: 12  8

24. 24 Pulse Code Modulation: PCM The basic coding method Producing quantized sampled output for audio The differences version: DPCM A crude but efficient variant (delta): DM. The adaptive version: ADPCM. Coding of Audio Example: WAV 是一種 PCM 編碼 Skype 採用 ADPCM, 32kbps

25. 25 Three-Stages Compression Every compression scheme has three stages: (A) The input data is transformed to a new representation that is easier or more efficient to compress. (B) We may introduce loss of information. Quantization is the main lossy step  we use a limited number of reconstruction levels, fewer than in the original signal. (C) Coding. Assign a codeword (thus forming a binary bitstream) to each output level or symbol. This could be a fixed-length code, or a variable length code such as Human coding (Chap. 7). DPCM (next pp preview) e.g. Hoffman code

26. 26 Example: DPCM codec module

27. 27 Break A Begin {Huffman Code}

28. 28 Huffman Code (Lossless Compression) Expected length Original  1/8  2 + 1/4  2 + 1/2  2 + 1/8  2 = 2 bits / symbol Huffman  1/8  3 + 1/4  2 + 1/2  1 + 1/8  3 = 1.75 bits / symbol Symbol@#$& Frequency1/81/41/21/8 Original Encoding 00011011 2 bits Huffman Encoding 110100111 3 bits2 bits1 bit3 bits

29. 29 Huffman Tree Construction 1 3 2587 A BCDE

30. 30 Huffman Tree Construction 2 3 58 2 7 5 A CD B E

31. 31 Huffman Tree Construction 3 3 5 8 2 7 5 10 A C D B E

32. 32 Huffman Tree Construction 4 3 5 8 2 7 5 10 15 A C D B E

33. 33 Huffman Tree Construction 5 3 58 2 7 5 10 15 25 1 1 1 1 0 0 0 0 A CD B E E = 00 D = 01 C = 10 B = 110 A = 111 010001110101110001 =DEDBCAED Average Length: 3x3/25 +3x2/25 +2x5/25 + 2x8/25 +2x7/27 = 2.2 (bits)

34. 34 Break B End {Huffman Code} Begin {Differential Coding Method}

35. 35 Differential Coding of Audio Audio is often stored not in simple PCM Instead in a form that exploits differences – which are generally smaller numbers, so offer the possibility of using fewer bits to store. (6.12)

36. 36 Fig 6.15 Histogram of digital speech signal Signal Values v.s. Signal Differences

37. 37 Coding of Signal Differences Suppose : sample values in [0, 255]  Differences could be in [-255, 255] codewords for differences in [-15, 16] define two new codes, SU and SD standing for Shift-Up and Shift-Down "32" For example, 100 is transmitted as: SU, SU, SU, 4 Recall: Huffman Code

38. 38 Lossless Predictive Coding (6.13) (6.14)

39. 39 Example (Predictive Coding) f 1,f 2, f 3, f 4, f 5 = 21, 22, 27, 25, 22. Invent an extra signal value f0 = f1 = 21 (6.15)

40. 40 f0=f1, e0=0 Schematic Diagram (Pred. Coding)

41. 41 Break C End {Differential Coding Method}

42. 42 Pulse Code Modulation: PCM The basic coding method Producing quantized sampled output for audio The differences version: DPCM A crude but efficient variant (delta): DM. The adaptive version: ADPCM. Coding of Audio Example: WAV 是一種 PCM 編碼 Skype 採用 ADPCM, 32kbps

43. 43 Fig 6.15 Recall: Histogram of Signal Signal Values v.s. Signal Differences

44. 44 f0=f1, e0=0 Recall: Predictive Coding ?!

45. 45 DPCM codec module

46. 46 (6.16) DPCM Formulae "^" hat “ ~" tilde

47. 47 Example (DPCM, formulae)

48. 48 Example (DPCM, results) 130 (1) (2) (3) Encoder: (1) (2) (3) Decoder: (1) (3)

49. 49 (6.21) DM (Delta Modulation) Formulae

50. 50 Example (DM, results) k=4, f 1 =f 1 =10 ~

51. 51 ADPCM codec module

52. 52 End of Chap #6