1. SAMPLING & QUANTIZATION Dhany Arifianto

2. How might you measure waves on a beach? Or, even, sample the wave height

3. Is sampling okay? Question: Is a sampled representation the same thing? –Yes, provided…

4. Digital means discrete (like whole numbers) and Analog means continuous (like physical properties such as temperature, volume, etc.).Digital means discrete (like whole numbers) and Analog means continuous (like physical properties such as temperature, volume, etc.). The term analog comes from early computers (circa WWII) used to solve differential equations with continuous variables,The term analog comes from early computers (circa WWII) used to solve differential equations with continuous variables, as contrasted with discrete state machines (like an elevator controller) built from open- or-closed switches or on-off digital circuitsas contrasted with discrete state machines (like an elevator controller) built from open- or-closed switches or on-off digital circuits

5. Definitions (from whatis.com) Analog Using physical representationUsing physical representation Relating to a system, device that represents data variation by a measurable physical quality such as temperature, volume, distance, weight, pressure …Relating to a system, device that represents data variation by a measurable physical quality such as temperature, volume, distance, weight, pressure … Which is continuous in time or space and valueWhich is continuous in time or space and value

6. Definitions Digital Representing data as numbersRepresenting data as numbers –Processing –Operating on –Storing –Transmitting –Displaying Data in the form of numerical digits, as in a digital computerData in the form of numerical digits, as in a digital computer

7. Representing a physical quantityRepresenting a physical quantity –such as sound, light, or electricity by means of samplesby means of samples –taken at discrete times (or places) –and given numerical values usually in the binary systemusually in the binary system –as in a digital audio recording –or in digital television –or in digital photography

8. Analog vs. Digital Analog –Media represented using real values –Sound pressure in the air (example) –Electronic representation of sound from a microphone Digital –Media represented using discrete values (integers, quantized numbers, floating point representations) May be infinite, but for a fixed range finite. In a mathematical sense

9. Electronic capture of sounds NS Analog pressure variations Analog voltage variations

10. In Communications Analog is used to refer to systems with signals that are continuous in value and timeAnalog is used to refer to systems with signals that are continuous in value and time –such as AM and FM, where the electrical signals are representations of the information signals.

11. Amplitude Modulation (AM)

12. Phase or Frequency Modulation (FM)

13. In Communications Digital is used to refer to discrete-state, discrete-time signals that can take on only specific values at specific times;Digital is used to refer to discrete-state, discrete-time signals that can take on only specific values at specific times; such assuch as –sampled/quantized signals, –pulse modulated signals, and to data communication signals in general.and to data communication signals in general.

14. Digital Modulation: Discrete in Time and Value

15. Parameters of Information Sources & Systems Analog (continuous functions of time, space, weight, …)Analog (continuous functions of time, space, weight, …) –voice, audio, image, video, temperature Bandwidth – frequency (harmonics) rangeBandwidth – frequency (harmonics) range Statistics – amplitude distribution, power, spectrum (frequency content, harmonics)Statistics – amplitude distribution, power, spectrum (frequency content, harmonics) Digital (sets of numbers):Digital (sets of numbers): –ASCII characters, computer words, … Bit Rate – bps, kbps, Mbps, Gbps, Tbps, Ebps, …Bit Rate – bps, kbps, Mbps, Gbps, Tbps, Ebps, …

16. How does Information Become “Digital”?

17. Digital Representation Information that is naturally discrete, such as state of a light switch (on-off), integers, or text can be represented by binary numbers in obvious ways.Information that is naturally discrete, such as state of a light switch (on-off), integers, or text can be represented by binary numbers in obvious ways. Text (as generated on a keyboard) is often represented by 8-bit binary numbers.Text (as generated on a keyboard) is often represented by 8-bit binary numbers. Speech may be represented by a pressure wave, which is continuous – in time and value – and has to be sampled and quantized to be represented digitally.Speech may be represented by a pressure wave, which is continuous – in time and value – and has to be sampled and quantized to be represented digitally.

18. Discrete Information Some information, such as numerals and characters is discrete and can be represented “digitally” easilySome information, such as numerals and characters is discrete and can be represented “digitally” easily Take characters of the English Language for exampleTake characters of the English Language for example The American Standard Code for Information Interchange (ASCII) is the binary representation used in teletype messaging and adopted as a universal computer character representation.The American Standard Code for Information Interchange (ASCII) is the binary representation used in teletype messaging and adopted as a universal computer character representation.

19. “A” = 11000001 “a” = 11100001 10001101 = CR 10001010 = LF 10000001 = SOH 10000010 = STX 10000100 = EOT 10000011 = ETX “%” = 10100101 Formatting Messaging

20. Common Sense Digitization of Analog Information All continuous signals can be represented by a collection of numbers to any degree of accuracy byAll continuous signals can be represented by a collection of numbers to any degree of accuracy by –sampling often enough and –using enough quantization levels* to represent the signal value at the sampling instants. –* determined by the number of digits in the representation

21. Analog-to-Digital Conversion Two stage processTwo stage process SampleSample –Sampling Theorem –Nyquist Rate QuantizeQuantize –Precision, SNR (% average error) –Note: a digital representation of an analog value always has error

22. The Sampling Theorem Shannon’s Sampling Theorem states thatShannon’s Sampling Theorem states that –any bandlimited signal may be represented by samples taken at a rate of twice its highest frequency*, and –may be reconstructed without error if the appropriate interpolation functions are used**. * Twice the highest frequency is called the Nyquist Rate. ** Physically unrealizable sinx/x or (sinc) functions. This theorem is also known as Nyquist–Shannon–Kotelnikov Whittaker–Shannon–Kotelnikov Whittaker–Nyquist–Kotelnikov–Shannon WKS Cardinal Theorem of Interpolation Theory

23. The Sampling Theorem Harry Nyquist, working at Bell Labs developed what has become known as the Nyquist Reconstruction Theorem: Given a uniform sample rate of f s, the highest frequency that can be unambiguously represented is f s /2 –f s /2 is the Nyquist Frequency CD Players use f s =44,100 samples/sec –The Nyquist Frequency is 22,050 Hz –Why don’t CD’s sample at 40KHz?

24. Sampling Lets consider a sign wave to be sampled (e.g. voice signal) If you sample at 1 time per cycle, you can think it is a constant Sampling 1.5 times each cycle appears as a low frequency sine signal Nyquist Theorem: For Lossless digitization, the sampling rate should be at least twice the maximum frequency responses

25. Representation of a CT Signal Using Impulse Functions The goal of this lecture is to convince you that bandlimited CT signals, when sampled properly, can be represented as discrete-time signals with NO loss of information. This remarkable result is known as the Sampling Theorem. Recall our expression for a pulse train: A sampled version of a CT signal, x(t), is: This is known as idealized sampling. We can derive the complex Fourier series of a pulse train: t x(t)x(t) -2T-T-T0T2T2T … …

26. Sampling Sampling - simply taking readings at fixed points in time Uniform sampling – sampling at regular intervals

27. Impulse Sampling

28. The Fourier series of our sampled signal, x s (t) is: Recalling the Fourier transform properties of linearity (the transform of a sum is the sum of the transforms) and modulation (multiplication by a complex exponential produces a shift in the frequency domain), we can write an expression for the Fourier transform of our sampled signal: If our original signal, x(t), is bandlimited: Fourier Transform of a Sampled Signal

29. Sampling of Narrowband Signals What is the lowest sample frequency we can use for the narrowband signal shown to the right? Recalling that the process of sampling shifts the spectrum of the signal, we can derive a generalization of the Sampling Theorem in terms of the physical bandwidth occupied by the signal. A general guideline is, where B = B 2 – B 1. A more rigorous equation depends on B 1 and B 2 : Sampling can also be thought of as a modulation operation, since it shifts a signal’s spectrum in frequency.

30. Signal Reconstruction Note that if, the replicas of do not overlap in the frequency domain. We can recover the original signal exactly. The sampling frequency,, is referred to as the Nyquist sampling frequency. There are two practical problems associated with this approach:  The lowpass filter is not physically realizable. Why?  The input signal is typically not bandlimited. Explain.

31. Signal Interpolation The frequency response of the lowpass, or interpolation, filter is: The impulse response of this filter is given by: The output of the interpolating filter is given by the convolution integral: Using the sifting property of the impulse:

32. Signal Interpolation (Cont.) Inserting our expression for the impulse response: This has an interesting graphical interpretation shown to the right. This formula describes a way to perfectly reconstruct a signal from its samples. Applications include digital to analog conversion, and changing the sample frequency (or period) from one value to another, a process we call resampling (up/down). But remember that this is still a noncausal system so in practical systems we must approximate this equation. Such implementations are studied more extensively in an introductory DSP class.

33. Reconstruction

34. Sampling Analog Devices AD585 Sample and Hold Amplifier Real values at fixed points in time Uses a “sample-and-hold” Values are called “samples” Analog signal Sample and Hold Sampled signal Sample

35. Sound in the Real World Recall: All sounds can be expressed as a combination of sinusoids String instruments generate harmonics C8 on piano = 4186 Hz 6 th Harmonic is 25,116 Hz Above the Nyquist Frequency!! Many instruments generate content above 20 KHz

36. A short section of a speech waveform (highest frequency component is 3KHz) Reconstructed speech waveform with 1 KHz sampling rate (note the resulting waveform does not resemble original waveform)

37. Undersampling Sampling at an inadequate frequency rate Aliased into new form - Aliasing Loses information in the original signal Oversampling Sampling at a rate higher than minimum rate More values to digitize and process Increases the amount of storage and transmission COST $$ Undersampling & Oversampling

38. Aliasing Recall that a time-limited signal cannot be bandlimited. Since all signals are more or less time-limited, they cannot be bandlimited. Therefore, we must lowpass filter most signals before sampling. This is called an anti-aliasing filter and are typically built into an analog to digital (A/D) converter. If the signal is not bandlimited distortion will occur when the signal is sampled. We refer to this distortion as aliasing: How was the sample frequency for CDs and MP3s selected?

39. Antialiasing Filters We have to filter (remove) any content above the Nyquist Frequency Analog signal Sample and Hold Sampled signal Antialiasing Filter Not digital devices!!!

40. Analog filtering Filtering is the relative modification of amplitudes of different frequencies 0 1 010k20k30k 2 Flat response High (treble) boost Low (bass) cut

41. Antialiasing filters What we want is a brick wall filter, that passes everything below 20 KHz and cuts everything above 0 1 010k20k30k 2

42. Reality vs. Ideal But, we can’t build a brick wall filter We can specify: Maximum ripple Pass band Cut band Minimum cut Transition band

43. Real analog filters We pay for transition/pass ratio trpass Costs: Component count Drift Noise Non-linearity Phase distortion Other junk…

44. Undersampling and Oversampling of a Signal

45. Sampling is a Universal Engineering Concept Note that the concept of sampling is applied to many electronic systems:  electronics: CD players, switched capacitor filters, power systems  biological systems: EKG, EEG, blood pressure  information systems: the stock market. Sampling can be applied in space (e.g., images) as well as time, as shown to the right. Full-motion video signals are sampled spatially (e.g., 1280x1024 pixels at 100 pixels/inch), temporally (e.g., 30 frames/sec), and with respect to color (e.g., RGB at 8 bits/color). How were these settings arrived at?

46. Downsampling and Upsampling Simple sample rate conversions, such as converting from 16 kHz to 8 kHz, can be achieved using digital filters and zero-stuffing:

47. Oversampling Sampling and digital signal processing can be combined to create higher performance samplers For example, CD players use an oversampling approach that involves sampling the signal at a very high rate and then downsampling it to avoid the need to build high precision converter and filters.

48. Summary All signals can be represented by a collection of numbers to any degree of accuracy by sampling often enough and using enough quantization levels to represent the signal value at the sampling instant.All signals can be represented by a collection of numbers to any degree of accuracy by sampling often enough and using enough quantization levels to represent the signal value at the sampling instant.

49. Summary Shannon’s Sampling Theorem states that any strictly bandlimited function may be presented by sampling at a rate that is at least twice as fast as the highest frequency in the signal, and that it may be recovered without distortion by passing the (impulse) samples through an ideal low-pass filter with a bandwidth equal to that of the signal.Shannon’s Sampling Theorem states that any strictly bandlimited function may be presented by sampling at a rate that is at least twice as fast as the highest frequency in the signal, and that it may be recovered without distortion by passing the (impulse) samples through an ideal low-pass filter with a bandwidth equal to that of the signal.

50. Quantization For processing, storage or communication, samples with infinite precision must be quantized Such that a range, or interval, of values is represented by a single, finite precision, number For example, by a finite binary number.

51. Quantization 7 -3 1 2 7 5 5 -2 -4 -3 -2 7 7 3 2 6 7 4 3 1 1 time

52. Reconstitution time -2 -3 -4 Quantum Boundary Reconstruction Value Actual Value ERROR -3 7 1 2 7 5 5 -2 -4 -3 -2 7 7 3 2 6 7 4 3 1 1

53. Quantization (linear vs. logarithmic) Linear Quantization Logarithmic Quantization  The 127 quantization levels are spread evenly over the voice signal’s dynamic range  This gives loud voice signals the same degree of resolution (same step size) as soft voice signals  Encoding an analog signal in this manner, while conceptually simplistic, does not give optimized fidelity in the reconstruction of human voice  Most of the energy in human voice is concentrated in the lower end of voice’s dynamic range (no shouting – just from boss)  Quantization levels distributed according to a logarithmic, instead of linear, function gives finer resolution, or smaller quantization steps, at lower signal amplitudes  PCM in North America uses a logarithmic function called μ-law

54. Quantization Error (for nerds and audiophiles) The quantization error depends on the number of distinct quantization intervals used. If N binary digits are used, the number of distinct intervals is 2 N. The signal-to-quantization-error ratio is about (6N + 1.8) dB.

55. Binary Representation Once information is discretized, or sampled, a number can be assigned to represent the value of each sample. The number can be expressed as a binary number, e.g., 2009 is 1024 + 512 + 256 + 128 +64 + 32 + 8 + 4 + 1 1x 2 10 + 1x 2 9 + 1x 2 8 + 1x 2 7 + 1x 2 6 +1x 2 5 +1x 2 3 + 1x 2 2 + 1x 2 0 11111101101

56. Example: CD Audio Human hearing band: 20-20000 Hz Passband: 0-20000 Hz Transition band: 0.10 * 20000 = 2000 10% transition band Band is 20000-22000 Hz Minimum possible sample rate: 2 * 22000 = 44000 samples/second They actually use 44,100 samples/second Any ideas why? Example: Professional Audio Human hearing band: 20-20000 Hz Passband: 0-20000 Hz Transition band: 0.20 * 20000 = 4000 20% transition band Band is 20000-24000 Hz Minimum possible sample rate: 2 * 24000 = 48000 samples/second Most modern equipment has moved to 96,000 samples/second and a passband of 0- 40000Hz

57. Why 44,100 for CDs? 44,100 is evenly divisible by: –60 (American/Japanese television field rate) –50 (European television field rate) –30 (American/Japanese television frame rate) –25 (European television frame rate) The exact frame rate in the US/Japan is actually 29.97, which is not divisible, but there are lots of funky numbers in the television signal…

58. Choosing a sample rate for analog sampling 1.Select a passband 2.Choose a transition band ratio 3.Design an antialiasing filter 4.Select a sample rate such that f s /2 is above the transition band Generally you’ll use standard sample rates Remember, each sample is a data point ! Those rules are applicable in DAC/ADC design(for signal processing, control, etc.)

59. Converting Digital to Analog Assume: Unsigned 8 bit value, analog range 1 volt 1/8 V 1/16 V 1/32 V 1/64 V 1/128 V 1/256 V 1/4 V 1/2 V Digital Signal Analog Voltage Adder

60. Converting Analog to Digital Binary search for correct value Successive Approximation (Binary search) Successive Approximation (Binary search) a>b? a b D/A Conversion D/A Conversion Result