1. Lecture 2 Signals & Systems Review Marc Moonen & Toon van Waterschoot Dept. E.E./ESAT, K.U.Leuven marc.moonen@esat.kuleuven.be www.esat.kuleuven.be/scd/

2. DSP-II p. 2 Version 2009-2010 Lecture-2: Signals & Systems Review Signals & Systems Review Discrete-Time/Digital Signals sampling, quantization, reconstruction Discrete-Time Systems LTI, impulse response, convolution, z-transform, frequency response, frequency spectrum, IIR/FIR Discrete/Fast Fourier Transform (I)DFT, FFT

3. DSP-II p. 3 Version 2009-2010 Lecture-2: Signals & Systems Review Introduction: Digital Signal Processing? Signal = a physical quantity that varies as a function of some independent variable(s), e.g. time, position, frequency, … –1-dimensional: speech signal, audio signal, electromagnetic/radio signal, … –2-dimensional: image, … –N-dimensional Processing = `filtering’ (mostly) = noise reduction, equalization, signal separation, … Digital = …in `digital domain’ here…

4. DSP-II p. 4 Version 2009-2010 Lecture-2: Signals & Systems Review Introduction: Digital Signal Processing? Analog Signal Processing Circuit Analog Domain (Continuous-Time Domain) Analog signal processing Analog INAnalog OUT (=Spectrum/Fourier Transform) Joseph Fourier (1768-1830)

5. DSP-II p. 5 Version 2009-2010 Lecture-2: Signals & Systems Review Introduction: Digital Signal Processing? Analog-to- Digital Conversion DSP Digital-to- Analog Conversion Analog INAnalog OUTDigital IN Digital OUT 011010 0101 100110 0010 Digital signal processing in an analog world Analog domain Digital domain Analog domain

6. DSP-II p. 6 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete-Time/Digital Signals 1/10 Analog-to- Digital Conversion DSP Digital-to- Analog Conversion Analog INAnalog OUTDigital IN Digital OUT 011010 0101 100110 0010 sampling & quantization reconstruction

7. DSP-II p. 7 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete-Time/Digital Signal 2/10 : Sampling Time-domain sampling amplitude discrete-time [k] continuous-time (t) impulse train It will turn out (page 27) that a spectrum can be computed from x[k], which (remarkably) will be equal to the spectrum (Fourier transform) of the (continuous-time) sequence of impulses = discrete-time signal continuous-time signal 0 1 2 3 4

8. DSP-II p. 8 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete-Time/Digital Signals 3/10 : Sampling Spectrum replication –time domain: –frequency domain: magnitude frequency (f) magnitude frequency (f)

9. DSP-II p. 9 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete-Time/Digital Signals 4/10 : Sampling Sampling theorem –analog signal spectrum X(f) has a bandwidth of f max Hz –spectrum replicas are separated by f s =1/T s Hz –no spectral overlap if and only if magnitude frequency

10. DSP-II p. 10 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete-Time/Digital Signals 5/10 : Sampling Sampling theorem –terminology: sampling frequency/rate f s Nyquist frequency f s /2 sampling interval/period T s –e.g. CD audio: f s = 44,1 kHz Anti-aliasing prefilters –ifthen frequencies above the Nyquist frequency are ‘folded’ into lower frequencies (=aliasing) –to avoid aliasing, sampling is usually preceded by a low- pass (=anti-aliasing) filtering Harry Nyquist (7 februari 1889 – 4 april 1976)

11. DSP-II p. 11 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete-Time/Digital Signals 6/10 : Quantization B-bit quantization amplitude discrete time [k] 0 Q 2Q2Q 3Q3Q -Q-Q -2Q -3Q R amplitude discrete time [k] quantized discrete-time signal =digital signal discrete-time signal

12. DSP-II p. 12 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete-Time/Digital Signals 7/10 : Quantization B-bit quantization: –the quantization error can only take on values between and –hence can be considered as a random noise signal (see below) with range –the signal-to-noise ratio (SNR) of the B-bit quantizer can then be defined as the ratio of the signal range and the quantization noise range : = the “6dB per bit” rule

13. DSP-II p. 13 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete-Time/Digital Signals 8/10 : Reconstruction Reconstructor = –‘fill the gaps’ between adjacent samples –e.g. staircase reconstructor (with `hold’ circuit): amplitudediscrete time [k]amplitudecontinuous time (t) reconstructed analog signal discrete-time/digital signal

14. DSP-II p. 14 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete-Time/Digital Signals 9/10 : Reconstruction Ideal reconstructor = –Ideal (rectangular) low-pass filter –no distortion magnitude frequency (f) magnitude frequency Staircase reconstructor = –sinc-like low-pass filter with sidelobes –distortion due to spurious high frequencies magnitude frequency (f)

15. DSP-II p. 15 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete-Time/Digital Signals 10/10 : Reconstruction Anti-image post-filter –low-pass filter succeeds reconstructor, to remove spurious high frequency components due to non-ideal reconstruction Complete scheme is… Digital OUT Analog IN DSP Digital IN samplerquantizer anti- aliasing prefilter anti- image postfilter reconstructor Analog OUT

16. DSP-II p. 16 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete-Time Systems 1/15 Discrete-time (DT) system is `sampled data’ system Input signal u[k] is a sequence of samples (=numbers)..,u[-2],u[-1],u[0], u[1],u[2],… System then produces a sequence of output samples y[k]..,y[-2],y[-1],y[0], y[1],y[2],… Example: `DSP’ block in previous slide u[k]y[k]

17. DSP-II p. 17 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete-Time Systems 2/15 Will consider linear time-invariant (LTI) DT systems Linear : input u1[k] -> output y1[k] input u2[k] -> output y2[k] hence a.u1[k]+b.u2[k]-> a.y1[k]+b.y2[k] Time-invariant (shift-invariant) input u[k] -> output y[k], hence input u[k-T] -> output y[k-T] u[k]y[k]

18. DSP-II p. 18 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete-Time Systems 3/15 Causal systems iff for all input signals with u[k]=0,k output y[k]=0,k<0 Impulse response input …,0,0, 1,0,0,0,...-> output …,0,0, h[0],h[1],h[2],h[3],... General input u[0],u[1],u[2],u[3] (cfr. linearity & shift-invariance!) this is called a `Toeplitz’ matrix

19. DSP-II p. 19 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete-Time Systems 4/15 Convolution u[0],u[1],u[2],u[3]y[0],y[1],... h[0],h[1],h[2],0,0,... = `convolution sum’

20. DSP-II p. 20 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete-Time Systems 5/15 Z-Transform of system h[k] and signals u[k],y[k] H(z) is `transfer function’

21. DSP-II p. 21 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete-Time Systems 6/15 Z-Transform easy input-output relation: may be viewed as `shorthand’ notation (for convolution operation/Toeplitz-vector product) stability = bounded input u[k] leads to bounded output y[k] --iff --iff poles of H(z) inside the unit circle (now z=complex variable) (for causal,rational systems, see below)

22. DSP-II p. 22 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete-Time Systems 7/15 Example-1 : `Delay operator’ Impulse response is …,0,0,0, 1,0,0,0,… Transfer function is Pole at z=0 Example-2 : Delay + feedback Impulse response is …,0,0,0, 1,a,a^2,a^3… Transfer function is Pole at z=a u[k]y[k]=u[k-1] x + a u[k]y[k]

23. DSP-II p. 23 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete-Time Systems 8/15 Will consider only rational transfer functions: In general, these represent `infinitely long impulse response’ (`IIR’) systems N poles (zeros of A(z)), N zeros (zeros of B(z)) corresponds to difference equation Hence rational H(z) can be realized with finite number of delay elements, multipliers and adders

24. DSP-II p. 24 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete-Time Systems 9/15 Special case is N poles at the origin z=0 (hence guaranteed stability) N zeros (zeros of B(z)) = `all zero’ filters corresponds to difference equation =`moving average’ (MA) filters impulse response is = `finite impulse response’ (`FIR’) filters

25. DSP-II p. 25 Version 2009-2010 Lecture-2: Signals & Systems Review H(z) & frequency response: given a system H(z) given an input signal = complex sinusoid output signal : = `frequency response’ = complex function of radial frequency ω = H(z) evaluated on the unit circle Discrete-Time Systems 10/15 Re u[0]=1 u[2] u[1] Im Re

26. DSP-II p. 26 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete-Time Systems 11/15 H(z) & frequency response: periodic : period = for a real impulse response h[k] Magnitude response is even function Phase response is odd function example (`low pass filter’): Nyquist frequency (=2 samples/period)

27. DSP-II p. 27 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete-Time Systems 12/15 H(z) & Fourier transform –the frequency response of an LTI system is equal to the Fourier transform of the continuous-time impulse sequence (see p.7) constructed with h[k] : –similarly, the frequency spectrum of a discrete-time signal (=its z-transform evaluated at the unit circle) is equal to the Fourier transform of the continuous-time impulse sequence constructed with u[k], y[k] : Input/output relation:

28. DSP-II p. 28 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete-Time Systems 13/15 Example: All-pass filter –a (unity-gain) all-pass filter is a filter that passes all input signal frequencies without gain or attenuation –hence a (unity-gain) all-pass filter preserves signal energy –an all-pass filter may have any phase response

29. DSP-II p. 29 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete-Time Systems 14/15 Example: Biquadratic (=2 nd order) all-pass filter –it can be shown that for the unity-gain constraint to hold, the denominator coefficients must equal the numerator coefficients in reverse order, i.e., –the poles and zeros are then related as follows

30. DSP-II p. 30 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete-Time Systems 15/15 PS: –So far have only considered single-input/single-output (SISO) systems –Similar equations for multiple-input/multiple-output (MIMO) systems –Example : 2-inputs, 3 outputs

31. DSP-II p. 31 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete/Fast Fourier Transform 1/5 DFT definition: –the frequency spectrum/response of a discrete-time signal/system x[k] is a (periodic) continuous function of the radial frequency ω –The `Discrete Fourier Transform’ (DFT) is a discretized version of this, obtained by sampling ω at N uniformly spaced frequencies (n=0,1,..,N-1) and by truncating x[k] to N samples (k=0,1,..,N-1) = DFT

32. DSP-II p. 32 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete/Fast Fourier Transform 2/5 Inverse discrete Fourier transform (IDFT): –an -point DFT can be calculated from an -point time sequence: –vice versa, an -point time sequence can be calculated from an -point DFT: = IDFT = DFT

33. DSP-II p. 33 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete/Fast Fourier Transform 3/5 DFT/IDFT in matrix form –Using shorthand notation.. –..the DFT can be rewritten as –an -point DFT requires complex multiplications

34. DSP-II p. 34 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete/Fast Fourier Transform 4/5 DFT/IDFT in matrix form –Using shorthand notation.. –..the IDFT can be rewritten as –an -point IDFT requires complex multiplications

35. DSP-II p. 35 Version 2009-2010 Lecture-2: Signals & Systems Review Discrete/Fast Fourier Transform 5/5 Fast Fourier Transform (FFT) (1805/1965) –divide-and-conquer approach: split up N-point DFT in two N/2-point DFT’s split up two N/2-point DFT’s in four N/4-point DFT’s … split up N/2 2-point DFT’s in N 1-point DFT’s calculate N 1-point DFT’s rebuild N/2 2-point DFT’s from N 1-point DFT’s … rebuild two N/2-point DFT’s from four N/4-point DFT’s rebuild N-point DFT from two N/2-point DFT’s –DFT complexity of multiplications is reduced to FFT complexity of multiplications James W. Cooley John W.Tukey Carl Friedrich Gauss (1777-1855

36. DSP-II p. 36 Version 2009-2010 Lecture-2: Signals & Systems Review Need more? Introductory books –S. J. Orfanidis, “Introduction to Signal Processing”, Prentice-Hall Signal Processing Series, 798 p., 1996 –J. H. McClellan, R. W. Schafer, and M. A. Yoder, “DSP First: A Multimedia Approach”, Prentice-Hall, 1998 –P. S. R. Diniz, E. A. B. da Silva and S. L. Netto, “Digital Signal Processing: System Analysis and Design”, Cambridge University Press, 612 p., 2002 Online books –Smith, J.O. Mathematics of the Discrete Fourier Transform (DFT), http://ccrma.stanford.edu/~jos/mdft/, 2003, ISBN 0-9745607-0-7. –Smith, J.O. Introduction to Digital Filters, August 2006 Edition, http://ccrma.stanford.edu/~jos/filters06/.