1. Signal Processing COS 323 COS 323

2. Digital “Signals” 1D: functions of space or time (e.g., sound)1D: functions of space or time (e.g., sound) 2D: often functions of 2 spatial dimensions (e.g. images)2D: often functions of 2 spatial dimensions (e.g. images) 3D: functions of 3 spatial dimensions (CAT, MRI scans) or 2 space, 1 time (video)3D: functions of 3 spatial dimensions (CAT, MRI scans) or 2 space, 1 time (video)

3. Digital Signal Processing 1.Understand analogues of filters 2.Understand nature of sampling

4. Filtering Consider a noisy 1D signal f(x)Consider a noisy 1D signal f(x) Basic operation: smooth the signalBasic operation: smooth the signal – Output = new function h(x) – Want properties: linearity, shift invariance Linear Shift-Invariant FiltersLinear Shift-Invariant Filters – If you double input, double output – If you shift input, shift output

5. Convolution Output signal at each point = weighted average of local region of input signalOutput signal at each point = weighted average of local region of input signal – Depends on input signal, pattern of weights – “Filter” g(x) = function of weights for linear combination – Basic operation = move filter to some position x, add up f times g

6. Convolution f(x) g(x)

7. Convolution f is called “signal” and g is “filter” or “kernel”, but the operation is symmetricf is called “signal” and g is “filter” or “kernel”, but the operation is symmetric Usually desirable to leave a constant signal unchanged: choose g such thatUsually desirable to leave a constant signal unchanged: choose g such that

8. Filter Choices Simple filters: box, triangleSimple filters: box, triangle

9. Gaussian Filter Very commonly used filterVery commonly used filter

10. Gaussian Filters Gaussians are used because:Gaussians are used because: – Smooth (infinitely differentiable) – Decay to zero rapidly – Simple analytic formula – Separable: multidimensional Gaussian = product of Gaussians in each dimension – Convolution of 2 Gaussians = Gaussian – Limit of applying multiple filters is Gaussian (Central limit theorem)

11. 2D Gaussian Filter

12. Sampled Signals Can’t store continuous signal: instead store “samples”Can’t store continuous signal: instead store “samples” – Usually evenly sampled: f 0 =f(x 0 ), f 1 =f(x 0 +  x), f 2 =f(x 0 +2  x), f 3 =f(x 0 +3  x), … Instantaneous measurements of continuous signalInstantaneous measurements of continuous signal – This can lead to problems 

13. Aliasing Reconstructed signal might be very different from original: “aliasing”Reconstructed signal might be very different from original: “aliasing” Solution: smooth the signal before samplingSolution: smooth the signal before sampling 

14. Discrete Convolution Integral becomes sum over samplesIntegral becomes sum over samples Normalization condition isNormalization condition is

15. Computing Discrete Convolutions What happens near edges of signal?What happens near edges of signal? – Ignore (Output is smaller than input) – Pad with zeros (edges get dark) – Replicate edge samples – Wrap around – Reflect – Change filter

16. Computing Discrete Convolutions If f has n samples and g has m nonzero samples, straightforward computation takes time O( nm )If f has n samples and g has m nonzero samples, straightforward computation takes time O( nm ) OK for small filter kernels, bad for large onesOK for small filter kernels, bad for large ones

17. Example: Smoothing Original image Smoothed with 2D Gaussian kernel

18. Example: Smoothed Derivative Derivative of noisy signal = more noisyDerivative of noisy signal = more noisy Solution: smooth with a Gaussian before taking derivativeSolution: smooth with a Gaussian before taking derivative Differentiation and convolution both linear operators: they “commute”Differentiation and convolution both linear operators: they “commute”

19. Example: Smoothed Derivative Result: good way of finding derivative = convolution with derivative of GaussianResult: good way of finding derivative = convolution with derivative of Gaussian

20. Smoothed Derivative in 2D What is “derivative” in 2D? Gradient:What is “derivative” in 2D? Gradient: Gaussian is separable!Gaussian is separable! Combine smoothing, differentiation:Combine smoothing, differentiation:

21. Smoothed Derivative in 2D

22. Original Image Smoothed Gradient Magnitude

23. Canny Edge Detector SmoothSmooth Find derivativeFind derivative Find maximaFind maxima ThresholdThreshold

24. Canny Edge Detector Original Image Edges

25. Fourier Transform Transform applied to function to analyze its “frequency” contentTransform applied to function to analyze its “frequency” content Several versionsSeveral versions – Fourier series: input = continuous, bounded; output = discrete, unboundedinput = continuous, bounded; output = discrete, unbounded – Fourier transform: input = continuous, unbounded; output = continuous, unboundedinput = continuous, unbounded; output = continuous, unbounded – Discrete Fourier transform (DFT): input = discrete, bounded; output = discrete, boundedinput = discrete, bounded; output = discrete, bounded

26. Fourier Series Periodic function f(x) defined over [– ..  ] wherePeriodic function f(x) defined over [– ..  ] where

27. Fourier Series This works because sines, cosines are orthonormal over [– ..  ]:This works because sines, cosines are orthonormal over [– ..  ]: Kronecker delta:Kronecker delta:

28. Fourier Transform Continuous Fourier transform:Continuous Fourier transform: Discrete Fourier transform:Discrete Fourier transform: F is a function of frequency – describes how much of each frequency f containsF is a function of frequency – describes how much of each frequency f contains Fourier transform is invertibleFourier transform is invertible

29. Fourier Transform and Convolution Fourier transform turns convolution into multiplication: F ( f ( x ) * g ( x ) ) = F ( f ( x ) ) F ( g ( x ) ) (and vice versa): F ( f ( x ) g ( x ) ) = F ( f ( x ) ) * F ( g ( x ) )Fourier transform turns convolution into multiplication: F ( f ( x ) * g ( x ) ) = F ( f ( x ) ) F ( g ( x ) ) (and vice versa): F ( f ( x ) g ( x ) ) = F ( f ( x ) ) * F ( g ( x ) )

30. Fourier Transform and Convolution Useful application #1: Use frequency space to understand effects of filtersUseful application #1: Use frequency space to understand effects of filters – Example: Fourier transform of a Gaussian is a Gaussian – Thus: attenuates high frequencies = Frequency Amplitude Frequency Amplitude Frequency Amplitude

31. Fourier Transform and Convolution Box function?Box function? In frequency space: sinc functionIn frequency space: sinc function – sinc(x) = sin(x) / x – Not as good at attenuating high frequencies

32. Fourier Transform and Convolution Fourier transform of derivative:Fourier transform of derivative: Blows up for high frequencies!Blows up for high frequencies! – After Gaussian smoothing, doesn’t blow up

33. Fourier Transform and Convolution Useful application #2: Efficient computationUseful application #2: Efficient computation – Fast Fourier Transform (FFT) takes time O( n log n ) – Thus, convolution can be performed in time O( n log n + m log m ) – Greatest efficiency gains for large filters