1. Computing With Images: Outlook and applications
C306, fall 2001
Martin Jagersand
Zach Dodds, Greg Hager

2. Commercial application areas
Industrial
Surveillance
Medical

3. Academic Related areas
Computer Vision: Interpret images
Graphics: Renders images from 3D models
AI: Somehow “intelligently” reason about images
Robotics: Use vision to guide motion.
Optics/physics: The detailed study of the physical penomena of light transport
Human/biological vision: How we see.
Medical image processing: Often using image-like data from e.g. 3D: NMRI, CAT scan, 2D Xray

4. Computer Vision vs Image processing
Mostly concerned with image-to-image transformations
Filtering
Enhancement
Compression
Computer Vision
Concerned with how images reflect the 3D world
Filtering for feature extraction
Enhancement for recognition/detection
Compression that preserves geometric information in images

5. Tasks in Vision Perception:
What: Label what is in a scene: What people, animals, objects…
Action:
Where: Determine the coordinates of where something is. (Mishkin, Ungleider)
How: Use vision to perform some physical manipulation task. (Goodale etc.)

6. Compare in Human vision: Dorsal and Ventral Pathways

7. The “Vision Problem” Input Output Vision Algorithm

8. The “Vision Problem” Input Output Vision Algorithm

9. Allies and Inspiration
Input
Output
Vision
Algorithm
Image Processing
Cognitive Science
Physics, Optics
Engineering
Biology, Psychology
Computer Science
Graphics, AI, NNets, ...

10. What is an image? What is an object?
Recognizing objects
What is an image? What is an object?

11. What: Recognizing people and objects
How do we determine that these are the same objects?
Two approaches:
Shape
Texture

12. Model based recognition: Find and match the shape
Example:
Find the outlines of objects.
Try to generate 3D or some pseudo 3D geometric description
Check database for a similar geometric object

13. Model based recognition: Techniques.
RIGHT IMAGE
PLANE
LEFT IMAGE
RIGHT
FOCAL
POINT
LEFT
BASELINE d
LENGTH f
Stereo:
From two (or more) images, determine the geometry of the scene by matching corresponding areas of the images

14. Apperance based recognition Match the 2D visual “texture”
Methods:
Spatial: Determine “likeness” by convolution.
“Frequency”: Transform and measure in fourier or KL space.

15. Where Use stereo to determine 3D location with respect to camera.
If we know where the camera is we can determine 3D world position
RIGHT IMAGE
PLANE
LEFT IMAGE
RIGHT
FOCAL
POINT
LEFT
BASELINE d
LENGTH f

16. How (to act in the world)?

17. Motion estimation and tracking
Detecting motion: 50
Candidate areas for
motion

18. Moving the camera Like stereo! The change in spatial location
between the two cameras (the “motion”)
Locations of
points on the object
(the “structure”)

19. Tracking Goal: Stabilizing motion.
Method:Find same region in consecutive images

20. Applications of Computer Vision: Medical Imaging

21. Applications of Computer Vision: Image Databases
(Courtesy D. Forsyth & J. Ponce)
From a search
for horse pix
in 100 horse
images and
1086 non-horse
images

22. Applications of Computer Vision: Data Acquisition

23. Computer Vision Courses
New undergraduate course next academic year:
CMPUT497: Computer vision
Text: Shapiro and Stockman
Connects to graduate courses
CMPUT 615: 3D computer vision, 609 pattern recognition, 613 geom modeling, 616: the brain
See

24. Computer Graphics

25. Computer graphics v.s. vision
shape estimation
motion estimation
recognition
2D modeling
modeling
shape
light
motion
optics
images IP
Computer Vision
rendering
modeling
shape
light
motion
optics
images IP
surface design
animation
user-interfaces
Computer Graphics

26. Modeling: Two Complementary Approaches
Conventional graphics
Image-based modeling and rendering
real
images
geometry, physics computer algorithms
geometry, physics computer algorithms
synthetic
images
synthetic
images

27. Object & Environment Modeling
Basic techniques from the conventional (hand) modeling perspective:
Declarative: write it down (e.g. typical graphics course)
Interactive: sculpt it (Maya, Blender …)
Programmatic: let it grow (L-systems for plants, Fish motion control)
Basic techniques from the image-based perspective:
Collect many pictures of a real object/environment; rely on image analysis to unfold the picture formation process (principled)
Collect one or more pictures of a real object/environment; manipulate them to achieve the desired effect (heuristic)

28. Rendering: Regular and Image-Based
Basic techniques from the forward modeling perspective (traditional rendering)
1. Input: 3D description of 3D scene & camera
2. Solve light transport through environment
3. Project to camera’s viewpoint
4. Perform ray-tracing
Basic technique from the inverse modeling perspective (image-based rendering)
1. Collect one or more images of a real scene
2. Warp, morph, or interpolate between these images to obtain new views

29. 3D sensors: Laser scanner

30. 3D: Direct Replica of Large Museum Objects
STL model
3-D scan of the sculpture
Replicas produced
by a 3-D Laser Sintering
Machine (Kaiplast Inc.)

31. Image-based modeling and rendering
Training
Model
New view I1 It
Structure P
New pose
(R a b) … = =
(R1 a1 b1) …(Rt at bt)
Motion params + +
Texture basis
(Cobzas, Yerex Jagersand 2002)
Warped texture
y1 … yt
Texture coeff

32. Pure graphics application: Rendering speed-up
Post-warping images
Blending a light basis

33. Image-based modeling and rendering
Sample scene with image-based and regular graphics objects:
See:

34. Visual Computing Summary:
Previous: Separate fields: Image-processing, computer vision, graphics.
But all are about computing with images and other spatial representations
Now: Coming to a confluence in the area of visual computing
See: