ALARI/DSP INTRODUCTION-2 Toon van Waterschoot & Marc Moonen Dept. E.E./ESAT, K.U.Leuven toon.vanwaterschoot@esat.kuleuven.be http://homes.esat.kuleuven.be/~tvanwate

INTRODUCTION-1 : Overview Discrete-time signals sampling, quantization, reconstruction Stochastic signal theory deterministic & random signals, (auto-)correlation functions, power spectra, … Discrete-time systems LTI, impulse response, FIR/IIR, causality & stability, convolution & filtering, … Complex number theory complex numbers, complex plane, complex sinusoids, circular motion, sinusoidal motion, … Toon van Waterschoot & Marc Moonen INTRODUCTION-2

INTRODUCTION-2 : Overview z-transform and Fourier transform region of convergence, causality & stability, properties, frequency spectrum, transfer function, pole-zero representation, … Elementary digital filters shelving filters, presence filters, all-pass filters Discrete transforms DFT, FFT, properties, fast convolution, overlap-add/overlap-save, … Toon van Waterschoot & Marc Moonen INTRODUCTION-2

z- and Fourier-transform: overview z-transform: definition & properties complex variables region of convergence Fourier transform: frequency response Fourier transform Transfer functions: difference equations rational transfer functions poles & zeros stability in the z-domain Toon van Waterschoot & Marc Moonen INTRODUCTION-2

z- and Fourier-transform: z-transform definition: discrete-time sequence in integer variable z-transform discrete-time series in complex variable Toon van Waterschoot & Marc Moonen INTRODUCTION-2

z- and Fourier-transform: z-transform definition: z-transform of a discrete-time signal: z-transform Toon van Waterschoot & Marc Moonen INTRODUCTION-2

z- and Fourier-transform: z-transform definition: z-transform of a discrete-time system impulse response: z-transform Toon van Waterschoot & Marc Moonen INTRODUCTION-2

z- and Fourier-transform: z-transform properties: linearity property: time-shift theorem: convolution theorem: Toon van Waterschoot & Marc Moonen INTRODUCTION-2

z- and Fourier-transform: z-transform region of convergence: the z-transform of an infinitely long sequence is a series with an infinite number of terms for some values of the series may not converge the z-transform is only defined within the region of convergence (ROC): Toon van Waterschoot & Marc Moonen INTRODUCTION-2

z- and Fourier-transform: Fourier transf. Frequency response: for an LTI system a sinusoidal input signal produces a sinusoidal output signal at the same frequency the output can be calculated from the convolution: Toon van Waterschoot & Marc Moonen INTRODUCTION-2

z- and Fourier-transform: Fourier transf. Frequency response: the sinusoidal I/O relation is the system’s frequency response is a complex function of the radial frequency : denotes the magnitude response denotes the phase response Toon van Waterschoot & Marc Moonen INTRODUCTION-2

z- and Fourier-transform: Fourier transf. Frequency response: the frequency response is equal to the z-transform of the system’s impulse response, evaluated at for , is a complex function describing the unit circle in the z-plane Im z-plane Re Toon van Waterschoot & Marc Moonen INTRODUCTION-2

z- and Fourier-transform: Fourier transf. Frequency response & Fourier transform the frequency response of an LTI system is equal to the Fourier transform of the continuous-time impulse sequence constructed with h[k] : similarly, the frequency spectrum of a discrete-time signal (=its z-transform evaluated at the unit circle) is equal to the Fourier transform of the continuous-time impulse sequence constructed with u[k], y[k] : Input/output relation: Toon van Waterschoot & Marc Moonen INTRODUCTION-2

z- and Fourier-transform: Transfer func. Difference equations: the I/O behaviour of an LTI system using an FIR model, can be described by a difference equation: the I/O behaviour of an LTI system using an IIR model, can be described by a difference equation with an autoregressive part in the left-hand side: Toon van Waterschoot & Marc Moonen INTRODUCTION-2

z- and Fourier-transform: Transfer func. Rational transfer functions: transforming the FIR difference equation to the z-domain and using the convolution theorem, leads to: the z-transform of the impulse response is called the transfer function of the system: Toon van Waterschoot & Marc Moonen INTRODUCTION-2

z- and Fourier-transform: Transfer func. Rational transfer functions: transforming the IIR difference equation to the z-domain and using the convolution theorem, leads to: the ratio of and is equal to the z-transform of the impulse response and is called the transfer function of the system: Toon van Waterschoot & Marc Moonen INTRODUCTION-2

z- and Fourier-transform: Transfer func. Poles and zeros: the zeros of a rational transfer function are defined as the roots of the nominator polynomial the poles of a rational transfer function are defined as the roots of the denominator polynomial e.g. Im Re z-plane Toon van Waterschoot & Marc Moonen INTRODUCTION-2

z- and Fourier-transform: Transfer func. Stability in the z-domain: the pole-zero representation of a rational transfer function allows for an easy stability check an LTI system is stable if all of its poles lie inside the unit circle in the complex z-plane Im Im Re unstable stable Re Toon van Waterschoot & Marc Moonen INTRODUCTION-2

INTRODUCTION-2 : Overview z-transform and Fourier transform region of convergence, causality & stability, properties, frequency spectrum, transfer function, pole-zero representation, … Elementary digital filters shelving filters, presence filters, all-pass filters Discrete transforms DFT, FFT, properties, fast convolution, overlap-add/overlap-save, … Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Elementary digital filters: overview Shelving filters: definition one-zero one-pole Presence filters: two-zero two-pole biquadratic All-pass filters: Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Elementary digital filters: shelving filters Definition: a shelving filter is a filter that amplifies a signal in the frequency range Hz (boost), while attenuating it in the range Hz (cut), or vice versa Low-pass filter: low-frequency boost, high-frequency cut High-pass filter: low-frequency cut, high-frequency boost Cut-off frequency: the cut-off frequency is usually defined as the frequency at which the filter gain is 3dB less than the gain at Hz (low-pass) or Hz (high-pass) Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Elementary digital filters: shelving filters One-zero shelving filter: difference equation: transfer function: signal flow graph:  Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Elementary digital filters: shelving filters One-zero shelving filter: 1 real zero: highpass if lowpass if Im Im highpass lowpass Re Re Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Elementary digital filters: shelving filters One-zero shelving filter: frequency response frequency magnitude response: frequency phase response: Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Elementary digital filters: shelving filters One-zero shelving filter: Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Elementary digital filters: shelving filters One-pole shelving filter: difference equation: transfer function: signal flow graph:  Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Elementary digital filters: shelving filters One-pole shelving filter: 1 real pole: highpass if lowpass if Im Im highpass lowpass Re Re Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Elementary digital filters: shelving filters One-pole shelving filter: frequency response frequency magnitude response: frequency phase response: Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Elementary digital filters: shelving filters One-pole shelving filter: Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Elementary digital filters: presence filters Definition: a presence filter is a filter that amplifies a signal in the frequency range around a center frequency Hz (boost), while attenuating elsewhere (cut), or vice versa Resonance filter: boost at center frequency (band-pass) Notch filter: cut at center frequency (band-stop) Bandwidth: the bandwidth is defined as the frequency difference between the frequencies at which the filter gain is 3dB lower/higher than the resonance/notch gain Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Elementary digital filters: presence filters Two-zero presence filter: diff. eq.: transfer function: signal flow graph:   Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Elementary digital filters: presence filters Two-zero presence filter: 2 zeros: if : real zeros  cascade shelving filters if : complex conj. zero pair  notch filter Im Im cascade shelving filters notch filter Re Re Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Elementary digital filters: presence filters Two-zero notch filter: transfer function in radial representation: radial center frequency zero radius Im notch filter Re Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Elementary digital filters: presence filters Two-zero notch filter: Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Elementary digital filters: presence filters Two-pole presence filter: diff. eq.: transfer function: signal flow graph:   Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Elementary digital filters: presence filters Two-pole presence filter: 2 poles: if : real poles  cascade shelving filters if : comp. conj. pole pair  resonance filter Im Im cascade shelving filters resonance filter Re Re Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Elementary digital filters: presence filters Two-pole resonance filter: transfer function in radial representation: radial center frequency pole radius Im resonance filter Re Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Elementary digital filters: presence filters Two-pole resonance filter: Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Elementary digital filters: presence filters Biquadratic presence filter: difference equation: transfer function: 2 poles: 2 zeros: Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Elementary digital filters: presence filters Biquadratic presence filter: signal flow graph:   Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Elementary digital filters: presence filters Constrained biquadratic presence filter: Elementary digital filters: presence filters constrained biquadratic resonance filter constrained biquadratic notch filter Im Im Re Re Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Elementary digital filters: all-pass filters Definition: a (unity-gain) all-pass filter is a filter that passes all input signal frequencies without gain or attenuation hence a (unity-gain) all-pass filter preserves signal energy an all-pass filter may have any phase response Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Elementary digital filters: all-pass filters Biquadratic all-pass filter: it can be shown that for the unity-gain constraint to hold, the denominator coefficients must equal the numerator coefficients in reverse order, e.g., the poles and zeros are moreover related as follows Toon van Waterschoot & Marc Moonen INTRODUCTION-2

INTRODUCTION-2 : Overview z-transform and Fourier transform region of convergence, causality & stability, properties, frequency spectrum, transfer function, pole-zero representation, … Elementary digital filters shelving filters, presence filters, all-pass filters Discrete transforms DFT, FFT, properties, fast convolution, overlap-add/overlap-save, … Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Discrete transforms: overview Discrete Fourier Transform (DFT): definition inverse DFT matrix form properties Fast Fourier Transform (FFT): Digital filtering using the DFT/FFT: linear & circular convolution overlap-add method overlap-save method fast convolution Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Discrete transforms: DFT DFT definition: the Fourier transform of a signal or system is a continuous function of the radial frequency : the Fourier transform can be discretized by sampling it at discrete frequencies , uniformly spaced between and : = DFT Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Discrete transforms: DFT Inverse discrete Fourier transform (IDFT): an -point DFT can be calculated from an -point time sequence: vice versa, an -point time sequence can be calculated from an -point DFT: = IDFT Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Discrete transforms: DFT matrix form using the shorthand notations the DFT and IDFT definitions can be rewritten as: DFT: IDFT: Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Discrete transforms: DFT matrix form the DFT coefficients can then be calculated as an -point DFT requires complex multiplications Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Discrete transforms: DFT matrix form the IDFT coefficients can then be calculated as an -point IDFT requires complex multiplications Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Discrete transforms: DFT properties: linearity & time-shift theorem (cf. z-transform) frequency-shift theorem (modulation theorem): circular convolution theorem: if and are periodic with period , then (see also ‘Digital filtering using the DFT/FFT’) Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Discrete transforms: FFT Fast Fourier Transform (FFT) divide-and-conquer approach: split up N-point DFT in two N/2-point DFT’s split up two N/2-point DFT’s in four N/4-point DFT’s … split up N/2 2-point DFT’s in N 1-point DFT’s calculate N 1-point DFT’s rebuild N/2 2-point DFT’s from N 1-point DFT’s rebuild two N/2-point DFT’s from four N/4-point DFT’s rebuild N-point DFT from two N/2-point DFT’s DFT complexity of multiplications is reduced to FFT complexity of multiplications James W. Cooley Carl Friedrich Gauss (1777-1855 John W.Tukey Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Discrete transforms: Digital filtering Linear and circular convolution: circular convolution theorem: due to the sampling of the frequency axis, the IDFT of the product of two -point DFT’s corresponds to the circular convolution of two length- periodic signals LTI system: the output sequence is the linear convolution of the impulse response with the input signal Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Discrete transforms: Digital filtering Linear and circular convolution: the linear convolution of a length- impulse response with a length- input signal is equivalent to their -point circular convolution if both sequences are zero-padded to length : zero padding Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Discrete transforms: Digital filtering Overlap-add/overlap-save: in many applications the input sequence length is much larger than the impulse response length computing the DFT of a very long or even infinitely long sequence is not feasible a block-based convolution method is more appropriate: the input sequence is divided in relatively short blocks each input block is circularly convolved with the impulse response using the DFT approach the output signal is reconstructed using the overlap-add method or the overlap-save method Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Discrete transforms: Digital filtering Overlap-add method: … + + + … Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Discrete transforms: Digital filtering Overlap-save method: = = … discard discard … discard Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Discrete transforms: Digital filtering Fast convolution: if in the above digital filtering methods the DFT is implemented using an FFT algorithm, then so-called fast convolution methods are obtained Toon van Waterschoot & Marc Moonen INTRODUCTION-2

Hungry for more? Some nice introductory books: S. J. Orfanidis, “Introduction to Signal Processing”, Prentice-Hall Signal Processing Series, 798 p., 1996 J. H. McClellan, R. W. Schafer, and M. A. Yoder, “DSP First: A Multimedia Approach”, Prentice-Hall, 1998 P. S. R. Diniz, E. A. B. da Silva and S. L. Netto, “Digital Signal Processing: System Analysis and Design”, Cambridge University Press, 612 p., 2002 Some interesting online books: Smith, J.O. Mathematics of the Discrete Fourier Transform (DFT), http://ccrma.stanford.edu/~jos/mdft/, 2003, ISBN 0-9745607-0-7. Smith, J.O. Introduction to Digital Filters, August 2006 Edition, http://ccrma.stanford.edu/~jos/filters06/. Toon van Waterschoot & Marc Moonen INTRODUCTION-2