 # CSCE 641 Computer Graphics: Image Sampling and Reconstruction Jinxiang Chai.

## Presentation on theme: "CSCE 641 Computer Graphics: Image Sampling and Reconstruction Jinxiang Chai."— Presentation transcript:

CSCE 641 Computer Graphics: Image Sampling and Reconstruction Jinxiang Chai

Review: 1D Fourier Transform A function f(x) can be represented as a weighted combination of phase-shifted sine waves How to compute F(u)? Inverse Fourier Transform Fourier Transform

Review: Box Function x u f(x) |F(u)| If f(x) is bounded, F(u) is unbounded

Review: Cosine 1  If f(x) is even, so is F(u)

Review: Gaussian If f(x) is gaussian, F(u) is also guassian.

Review: Properties Linearity: Time shift: Derivative: Integration: Convolution:

Review: Properties Linearity: Time shift: Derivative: Integration: Convolution:

Review: Properties Linearity: Time shift: Derivative: Integration: Convolution:

Outline 2D Fourier Transform Nyquist sampling theory Antialiasing Gaussian pyramid

Extension to 2D Fourier Transform: Inverse Fourier transform:

Building Block for 2D Transform Building block: Frequency: Orientation: Oriented wave fields

Building Block for 2D Transform Higher frequency Building block: Frequency: Orientation: Oriented wave fields A function f(x,y) can be represented as a weighted combination of phase-shifted oriented wave fields.

Some 2D Transforms From Lehar

Some 2D Transforms From Lehar

Some 2D Transforms From Lehar

Some 2D Transforms From Lehar

Some 2D Transforms From Lehar

Some 2D Transforms From Lehar Why we have a DC component?

Some 2D Transforms Why we have a DC component? - the sum of all pixel values From Lehar

Some 2D Transforms Why we have a DC component? - the sum of all pixel values Oriented stripe in spatial domain = an oriented line in spatial domain From Lehar

2D Fourier Transform Why? - Any relationship between two slopes?

2D Fourier Transform Linearity Why? - Any relationship between two slopes?

2D Fourier Transform The spectrum is bounded by two slopes. Linearity Why? - Any relationship between two slopes?

Online Java Applet http://www.brainflux.org/java/classes/FFT2D Applet.html

2D Fourier Transform Pairs Gaussian

2D Image Filtering Inverse transform Fourier transform From Lehar

2D Image Filtering Inverse transform Fourier transform Low-pass filter From Lehar

2D Image Filtering Inverse transform Fourier transform Low-pass filter high-pass filter From Lehar

2D Image Filtering Inverse transform Fourier transform Low-pass filter high-pass filter band-pass filter From Lehar

Aliasing Why does this happen?

Aliasing How to reduce it?

Sampling Analysis f(x) x T2T … -2T-T … 0 f s (x) x Sampling

Sampling Analysis f(x) x T2T … -2T-T … 0 f s (x) x Sampling Reconstruction

Sampling Analysis What sampling rate (T) is sufficient to reconstruct the continuous version of the sampled signal? f(x) x T2T … -2T-T … 0 f s (x) x Sampling Reconstruction

Sampling Theory How many samples are required to represent a given signal without loss of information? What signals can be reconstructed without loss for a given sampling rate?

Sampling Analysis: Spatial Domain f(x) T2T … -2T-T … 0 f s (x) x xT2T … -2T-T … 0x X ? continuous signal discrete signal (samples) comb function

Sampling Analysis: Spatial Domain f(x) T2T … -2T-T … 0 f s (x) x xT2T … -2T-T … 0x X ?What happens in Frequency domain? continuous signal discrete signal (samples) comb function

Sampling Analysis: Spatial Domain f(x) T2T … -2T-T … 0 f s (x) x xT2T … -2T-T … 0x X ?What happens in Frequency domain? continuous signal discrete signal (samples) comb function

Fourier Transform of Dirac Comb T

Review: Dirac Delta and its Transform x 1 u f(x) |F(u)| Fourier transform and inverse Fourier transform are qualitatively the same, so knowing one direction gives you the other

Review: Fourier Transform Properties Linearity: Time shift: Derivative: Integration: Convolution:

Fourier Transform of Dirac Comb T

T1/T1/T - Fourier transform of a Dirac comb is a Dirac comb as well. - Moving the spikes closer together in the spatial domain moves them farther apart in the frequency domain!

Sampling Analysis: Spatial Domain f(x) T2T … -2T-T … 0 f s (x) x xT2T … -2T-T … 0x X ? What happens in Frequency domain? continuous signal discrete signal (samples) comb function

Review: Properties Linearity: Time shift: Derivative: Integration: Convolution:

Sampling Analysis: Freq. Domain 1/T…-1/T…0u F(u) u -f max f max

Sampling Analysis: Freq. Domain 1/T…-1/T…0u F(u) u -f max f max How does the convolution result look like?

Sampling Analysis: Freq. Domain 1/T…-1/T…0u F(u) u -f max f max

Sampling Analysis: Freq. Domain 1/T…-1/T…0u F(u) u -f max f max

Sampling Analysis: Freq. Domain 1/T…-1/T…0u F(u) u -f max f max G(0)? G(f max )? G(u)?

Sampling Analysis: Freq. Domain 1/T…-1/T…0u F(u) u -f max f max G(0) = F(0) G(f max ) = F(f max ) G(u) = F(u)

Sampling Analysis: Freq. Domain 1/T…-1/T…0u F(u) u -f max f max F s (u) u -f max f max -1/T1/T How about

Sampling Analysis: Freq. Domain 1/T…-1/T…0u F(u) u -f max f max F s (u) u -f max f max -1/T1/T How about

Sampling Analysis: Freq. Domain 1/T…-1/T…0u F(u) u -f max f max F s (u) u -f max f max -1/T1/T

Sampling Theory How many samples are required to represent a given signal without loss of information? What signals can be reconstructed without loss for a given sampling rate?

Sampling Analysis: Freq. Domain 1/T…-1/T…0u F(u) u -f max f max F s (u) u -f max f max -1/T1/T How can we reconstruct the original signal?

Reconstruction in Freq. Domain u -f max f max F s (u) u -f max f max box(u) u -f max f max

Signal Reconstruction in Freq. Domain T2T … -2T-T … 0 x f s (x) f(x) x F s (u) u -f max f max F(u) u -f max f max Inverse Fourier transform Fourier transform

Signal Reconstruction in Spatial Domain T2T … -2T-T … 0 x x f s (x) sinc(x)

Aliasing Why does this happen?

Sampling Analysis T … -T … 0 f(x)F(u) f s (x) x ux -f max f max F s (u) u -f max f max When does aliasing happen?

Sampling Analysis T … -T … 0 f(x)F(u) f s (x) x ux -f max f max F s (u) u -f max f max When does aliasing happen?

Sampling Analysis T … -T … 0 f(x)F(u) f s (x) x ux -f max f max F s (u) u -f max f max When does aliasing happen?

Sampling Analysis T … -T … 0 f(x)F(u) f s (x) x ux -f max f max F s (u) u -f max f max When does aliasing happen?

Sampling Analysis What sampling rate (T) is sufficient to reconstruct the continuous version of the sampled signal? f(x) x T2T … -2T-T … 0 f s (x) x Sampling Reconstruction

Sampling Analysis What sampling rate (T) is sufficient to reconstruct the continuous version of the sampled signal? f(x) x T2T … -2T-T … 0 f s (x) x Sampling Reconstruction Sampling Rate ≥ 2 * max frequency in the signal this is known as the Nyquist Rate

Antialiasing

Increase the sampling rate to above twice the highest frequency u -f max f max

Antialiasing Increase the sampling rate to above twice the highest frequency Introduce an anti-aliasing filter to reduce f max u -f max f max F s (u) u -f max f max

Antialiasing Increase the sampling rate to above twice the highest frequency Introduce an anti-aliasing filter to reduce f max u -f max f max F s (u) u -f max f max

Sinc Filter F(u) u -f max f max Is this a good filter?

Sinc Filter F(u) u -f max f max x Inverse Fourier transform Is this a good filter?

Sinc Filter Is this a good filter? F(u) u -f max f max x Multiplying with a box function in frequency domain Convolution with a sinc function in spatial domain Inverse Fourier transform

Sinc Filter Is this a good filter? F(u) u -f max f max x Multiplying with a box function in frequency domain Convolution with a sinc function in spatial domain Good: removes all frequency components above a given bandwidth Bad: - an infinite support in spatial domain and hard to do in the spatial domain - negative weights (not good for filtering pixels) - fluctuation causes ripples Inverse Fourier transform

Good Pre-filtering filters Finite support in frequency domain - cut-off frequency Finite support in spatial domain - implementation in spatial domain Positive weights - good for filtering pixels

Other Filter Choices Mean filter Triangular filter Gaussian filter x f(x) u |F(u)| sinc(u) Sin 2 c(u) Cannot have a filter with finite support in both spatial and frequency domain (box, mean, triangular) - box filter: finite in freq. and infinite in spatial domain - mean and triangular: infinite in freq. and finite in spatial domain Gaussian filter provides a good tradeoff between two domains

Other Filter Choices Mean filter Triangular filter Gaussian filter x f(x) u |F(u)| sinc(u) Sin 2 c(u) Cannot have a filter with finite support in both spatial and frequency domain (box, mean, triangular) - box filter: finite in freq. and infinite in spatial domain - mean and triangular: infinite in freq. and finite in spatial domain Gaussian filter provides a good tradeoff between two domains

Gaussian Filters Cannot have a filter with finite support in both spatial and frequency domain (box, mean, triangular) - box filter: finite in freq. and infinite in spatial domain - mean and triangular: infinite in freq. and finite in spatial domain Gaussian filter provides a good tradeoff between two domains

Gaussian Filters Cannot have a filter with finite support in both spatial and frequency domain (box, mean, triangular) - box filter: finite in freq. and infinite in spatial domain - mean and triangular: infinite in freq. and finite in spatial domain Gaussian filter provides a good tradeoff between two domains

Gaussian Filter F s (u) u -f max f max

Gaussian Filter F s (u) u -f max f max -0.02 0.03 0.08 0.13 0.18 Inverse Fourier transform

Gaussian Filter F s (u) u -f max f max Multiplying with gaussian function in frequency domain Convolution with a gaussian function in spatial domain -0.02 0.03 0.08 0.13 0.18 Inverse Fourier transform

Gaussian Filter F s (u) u -f max f max Multiplying with gaussian function in frequency domain Convolution with a gaussian function in spatial domain - Since the Gaussian function decays rapidly, it is reasonable to truncate the filter window -Easy to implement it in the spatial domain - Its standard deviation controls the smoothness of filtered signal: large σ in spatial domain = lower cutoff frequency -0.02 0.03 0.08 0.13 0.18 Inverse Fourier transform

Review: Gaussian If f(x) is gaussian, F(u) is also guassian.

Filtering in Spatial Domain = Filter functionInput imageFiltered image

Gaussian Filtering in Spatial Domain Discrete approximation to Gaussian function with σ =1.0 filtering window: 5X5 ? Step 1: evaluate the filtering function at each grid Step 2: Normalize each weight to ensure the sum of all weights is one

Gaussian Filtering in Spatial Domain Discrete approximation to Gaussian function with σ =1.0 filtering window: 5X5 G(0,0) Step 1: evaluate the filtering function at each grid Step 2: Normalize each weight to ensure the sum of all weights is one

Gaussian Filtering in Spatial Domain Discrete approximation to Gaussian function with σ =1.0 filtering window: 5X5 G(0,0) ? Step 1: evaluate the filtering function at each grid Step 2: Normalize each weight to ensure the sum of all weights is one

Gaussian Filtering in Spatial Domain Discrete approximation to Gaussian function with σ =1.0 filtering window: 5X5 G(0,0) Step 1: evaluate the filtering function at each grid Step 2: Normalize each weight to ensure the sum of all weights is one G(-2,-2)

Gaussian Filtering in Spatial Domain Discrete approximation to Gaussian function with σ =1.0 Step 1: evaluate the filtering function at each grid Step 2: Normalize each weight to ensure the sum of all weights is one

Filtering in Spatial Domain Discrete approximation to Gaussian function with σ =1.0

Filtering in Spatial Domain Discrete approximation to Gaussian function with σ =1.0

Filtering in Spatial Domain Discrete approximation to Gaussian function with σ =1.0

Filtering in Spatial Domain Discrete approximation to Gaussian function with σ =1.0 Filtered_I 45 = X

Gaussian Pre-filtering G 1/4 G 1/8 Gaussian 1/2 Solution: filter the image, then subsample –Filter size should double for each ½ size reduction. Why?

Gaussian Prefiltering G 1/4G 1/8Gaussian 1/2 Solution: filter the image, then subsample –Filter size should double for each ½ size reduction. Why?

Without Prefiltering 1/4 (2x zoom) 1/8 (4x zoom) 1/2

Known as a Gaussian Pyramid –MipMap (Williams, 1983) Image Pyramids

MipMap Mip (latin phase): many things in a small place Used for texture mapping Small-scale texture used for rendering distant objects

Gaussian Pyramid filter mask Repeat –Filter –Subsample Until minimum resolution reached –can specify desired number of levels (e.g., 3-level pyramid ) The whole pyramid is only 4/3 the size of the original image!

Gaussian Pyramid

Laplacian Pyramid Laplacian Pyramid (subband images) - Created from Gaussian pyramid by subtraction - Encode fine to course structure in a different level

Laplacian Pyramid What happens in frequency domain?

Where Are They Used? Texture mapping - Mipmap Image processing - Image composition/blending - Image compression Computer vision - Image matching - Motion tracking - Optical flow estimation

Summary 2D Fourier Transform Nyquist sampling theory - Sampling Rate ≥ 2 * max frequency in the signal Antialiasing - prefiltering Gaussian pyramid - Mipmap

Next Lecture Image Processing - Filtering noise - Bilateral filter - Feature extraction

Download ppt "CSCE 641 Computer Graphics: Image Sampling and Reconstruction Jinxiang Chai."

Similar presentations