1. ELE 488 Fall 2006 Image Processing and Transmission (10-17-06)
10/17/06
ELE 488 Fall 2006 Image Processing and Transmission ( )
Geometric Transformation
translation
rotation
scaling
Affine
Image Registration
Goodness of fit
There are many other examples not touched here.
But with these examples, we can say something about why image processing.
We saw some examples of enhancement, which makes the image looks better, or show more details. We saw some examples of extracting information from the images, both still images and video. We saw the same image have vastly different file sizes. This has to do with image coding and data compression. We also saw video encoding can be used to make it less vulnerable to transmission error.
Image is information. How to store this kind of information, distribute it are important issued. A related topic is the digital library, which presents many challenges. Book, index, search, etc.

2. Mosaicking and Registration
Reference Image
Floating Image

3. Identify Features and Match to Determine Geometric Transformation
Image I (reference)
h(x,y)
Image II (floating)
h–1(x’,y’)
Two cartesian coordinate systems: (x,y) and (x’,y’)
Forward: (x’,y’) = h(x,y) Reverse: (x,y) = h-1(x’,y’)

4. Four Basic Transformations
translation
rotation
(1, 1) y x
scaling
shearing
(1, 0) y’ x’
(2, 1)
(3, 1)

5. Combining Transformations
Rotation and translation
Successive application of rotation, translation and scaling.
Involves matrix multiplications and additions
Homogeneous Coordinates
Matrix multiplication, no addition

6. Translation – Rotation – Scaling
Note: c=0

7. Combining Transformations
1) Translate
2) Rotate
3) Translate

8. Homogeneous Transforms
Rigid-body: Rotation and Translation
2x2 rotation submatrix is orthonormal
Translation/Rotation/Scaling
are determined by rotation and scaling parameters

9. Affine and Projective Transformations
Affine: 6 free parameters
Projective: 8 free parameters
More general geometric transformations between two planes
Widely used in modeling cameras
s scaling factor.

10. Inverse Transform
Examples:

11. Reflections
Reflect in x-axis
Reflect in y-axis
Reflect in line y=x

12. Reflect about a general line y=ax+b

13. Pinhole Camera Model

14. Pinhole Camera Model (shifted image plane)

15. Image Formation: pin hole model
Projection of point (X,Y,Z) in 3D space to (x,y) in image plane

16. Mosaicking and Registration
Reference Image
Floating Image
Two cameras Or one camera at two positions

17. Two Cameras
Point in 3-D has two sets of coordinates

18. Rotation of Camera About One Axis

19. General Rotation of Camera
Two 3-D coordinates of a point are related by a rotation matrix:
rotation matrix R
Decompose R into 3 rotations about the first camera axes:
Counter clockwise rotation about Z axis
Counter clockwise rotation about X axis
Counter clockwise rotation about Y axis

20. Translation
location of 2nd camera in 1st camera coordinates

21. 3-D Coordinate Transformation
The two 3-D coordinates of a point are related by a 3-D rotation and a translation:
Counter clockwise rotation about Z axis
Counter clockwise rotation about X axis
Counter clockwise rotation about Y axis
from pinhole camera

22. 3-D Coordinate Transformation

23. 3-D Coordinate Transformation
Independently scale each side of the equation

24. No Translation
Homogeneous transformation
Knowing establishes the correspondence between the images.

25. Examples

26. Flat Scene Knowing establishes the correspondence between the images.
See slide 9
Homogeneous transformation
Knowing establishes the correspondence between the images.

27. Planar Scene
Homogeneous transformation
Knowing establishes the correspondence between the images.

28. Mosaic: Firestone Library
Photos: P. Ramadge, Mosaic: Y.P. Tan/R. Radke
The zero translation approximation was used to register these images

29. Affine and Projective Transformations
Projective Transformations are closely connected to image formation models and camera rotation
Affine is an approximation:

30. Small Rotations
Can be approximated by affine

31. Three Questions of Image Registration
How is the spatial transformation (= warping function) applied and what family of warping functions are allowed?
How do we measure how well the warped floating image aligns with the reference image?
How to we find the best warp in the allowed class to best match the images (optimization)

32. Warping Function Examples
Translation
Rotation
Scaling
Affine
Projective

33. Goodness of Fit: How well does warped image align with reference
1) Use all pixel values:
2) Only use pixel values around selected “features”: x y
What are features?

34. Features and Feature Selection
Sharp  corners v u
GOOD
BAD v u

35. Registration As An Optimization Problem
Consider transformations parameterized by a vector of parameters θ
Find the transformation that minimizes the registration metric
(e.g., above)
In some special cases this can be solved analytically (in closed form)
In other cases it can be approximated and solved analytically
In the most complex cases it has to be solved iteratively by a gradient-like descent method.

36. Example: Translation + SSD Feature Metric
1 feature - can do this by brute force search by finding the best match
More features?

37. Simplification
Find the best translation for each feature individually by a brute force search (can be refined)
This gives (called a correspondence)
Then select the translation for the entire image to minimize :
This yields

38. Image Registration 756x504 pixels 16 bit color
Reference Image
Floating Image
756x504 pixels
16 bit color
Allowed warpings is translations
Matching metric is how well a selected subset of “features” match using SAD
best translation match. average pixels where images overlap

39. Registration Using Affine Transformation
translation, rotation, scaling, shearing

40. Registration Using Projective Transformation
Add: tilting