1. Computer Vision Dan Witzner Hansen Course web page:

2. What is Vision?

3. Today Introduction to the course
Crash course in 2D and 3D geometry (Brush-up from High school)
(Solving linear equations and least squares solution to linear equations)

4. The Vision Problem How to infer salient properties
of 3-D world from time-varying
2-D image projection
¤ What is salient?
¤ How to deal with loss of information going
from 3-D to 2-D?
signal may be noisy, occluded, etc.

5. Computer Vision: Stages
Image formation
Low-level
Single image processing
Multiple views
Mid-level
Estimation, segmentation (main topic of Image Analysis and Foundations of Image Analysis and will only be covered briefly here)
High-level
Recognition
Classification
sort of in order from light being emitted to the brain understanding

6. Image Formation 3-D geometry Physics of light Camera properties
Focal length
Distortion
Sampling issues
Spatial
Temporal

7. Low-level: Single Image Processing
Filtering
Edge
Color
Local pattern similarity
Texture
Appearance characterization from the statistics of applying multiple filters
3-D structure estimation from…
Shading

8. Low-level: Multiple Views
Stereo
Structure from two views
Structure from motion
What can we learn in general from many views, whether they were taken simultaneously or sequentially?

9. Mid-Level: Estimation, Segmentation
Estimation: Fitting parameters to data
Static (e.g., shape)
Dynamic (e.g., tracking)
Segmentation/clustering
Breaking an image or image sequence into a few meaningful pieces with internal similarity

10. High-level: Recognition, Classification
Recognition: Finding and parametrizing a known object
Classification
Assignment to known categories using statistics/probability to make best choice

11. APPLICATIONS Course Overview Image formation and cameras
Projective geometry
Relating points
between images
Motion
Motion analysis
Object Tracking
Shape and recognition
Shape anaysis
Object recognition
APPLICATIONS

12. Applications: Factory Inspection
Cognex’s “CapInspect” system:
Low-level image analysis: Identify edges, regions
Mid-level: Distinguish “cap” from “no cap”
Estimation: What are orientation of cap, height of liquid?

13. Applications: Face Detection
courtesy of H. Rowley
How is this like the bottle problem on the previous slide?
From

14. Applications: Text Detection & Recognition
from J. Zhang et al.
Similar to face finding: Where is the text and what does it say?
Viewing at an angle complicates things...
From

15. Detection and Recognition: How?
Build models of the appearance characteristics (color, texture, etc.) of all objects of interest
Detection: Look for areas of image with sufficiently similar appearance to a particular object
Recognition: Decide which of several objects is most similar to what we see
Segmentation: “Recognize” every pixel

16. Applications: Virtual Advertising
courtesy of Princeton Video Image

17. First-Down Line, Virtual Advertising: How?
Where should message go?
Sensors that measure pan, tilt, zoom and focus are attached to calibrated cameras at surveyed positions
Knowledge of the 3-D position of the line, advertising rectangle, etc. can be directly translated into where in the image it should appear for a given camera
What pixels get painted?
Occluding image objects like the ball, players, etc. where the graphic is to be put must be segmented out. These are recognized by being a sufficiently different color from the background at that point. This allows pixel-by-pixel compositing.

18. Applications: Inserting Computer Graphics with a Moving Camera
See this page:

19. CG Insertion with a Moving Camera: How?
This technique is often called matchmove
Once again, we need camera calibration, but also information on how the camera is moving—its egomotion. This allows the CG object to correctly move with the real scene, even if we don’t know the 3-D parameters of that scene.
Estimating camera motion:
Much simpler if we know camera is moving sideways because then the problem is only 2-D
For general motions: By identifying and following scene features over the entire length of the shot, we can solve retrospectively for what 3-D camera motion would be consistent with their 2-D image tracks. Must also make sure to ignore independently moving objects like cars and people.

20. Applications: Motion Capture
Vicon software:
12 cameras, 41 markers for body capture;
6 zoom cameras, 30 markers for face

21. Applications: Motion Capture without Markers
courtesy of C. Bregler
What’s the difference between these two problems?

22. Motion Capture: How?
Similar to matchmove in that we follow features and estimate underlying motion that explains their tracks
Difference is that the motion is not of the camera but rather of the subject (though camera could be moving, too)
Face/arm/person has more degrees of freedom than camera flying through space, but still constrained
Special markers make feature identification and tracking considerably easier
Multiple cameras gather more information

23. Applications: Image-Based Modeling
courtesy of P. Debevec
Façade project: UC Berkeley Campanile

24. Image-Based Modeling: How?
3-D model constructed from manually-selected line correspondences in images from multiple calibrated cameras
Novel views generated by texture-mapping selected images onto model

25. A Movie
Movie

26. Applications: Robotics
Autonomous driving: Lane & vehicle tracking (with radar)

27. Human Computer Interaction

28. What is the relationship between many of these applications?
Knowledge of
Cameras
Motion and Tracking
Shapes and object recognition
Mathematics and Statistics

29. Course Prerequisites Background in/comfort with:
Linear algebra
Multi-variable calculus
Statistics, probability
Homeworks will use Matlab but you are also welcome to use C/C++ (harder though)
An ability to program in C/C++, Java, or equivalent should be sufficient preparation, but knowing Matlab is better (no introduction given, but you can come see me if needed)

30. Grading
100 % on mandatory assignments
Submission ON TIME

31. More specifically…..

32. Single View Examples

33. Mosaicing

34. Stereo

35. Stereo reconstruction

36. Tracking, Shape and HCI

37. After the course
Understand, choose between, and apply various computer vision algorithms.
Understand the relations between objects in the 3D world and those obtained from cameras.
Understand the principles on how to make 3D models (reconstruction) from images.
Write programs which are able to follow objects in pre-recorded movies or live images obtained from cameras in either Matlab or C++.
Understand principles for making computer vision systems that aim towards enabling humans to interact with a computer through cameras.

38. Reading Material Textbooks:
“Multiple View Geometry” Hartley and Zisserman
”Introductory Techniques for 3D Computer Vision”(less important)
Supplemental readings will be available online as PDF files and a few as photocopies from books.
Complete assigned reading before corresponding lecture and re-read difficult parts after the lecture.
This is NOT an easy course so, expect at least 15 hrs WORK each week.
Show up for ALL lectures.

39. Details
Homework
Submission at to me in by the end of the exercises.
Expect to have it ready before the exercises, though!
NO Lateness policy – Add-on’s will be exprected if late
Exam
Submission of mandatory assignments by the end of the semeste.

40. More Details Instructor E-mail: witzner@itu.dk
Office hours (by appointment):
Friday, 10:00-12:00 pm
Remember that semester projects in connection with
the course are possible.

41. Your First Assignment Try to get Matlab running
Take a look at a Matlab primer
Unfortunately most of the tools (mathematics) have to be developed in the beginning of the course and it may therefore seem quite mathematical.
DON’T LET THAT DISCURAGE YOU

42. First try the web page: www.itu.dk/courses/MCV
More questions?
First try the web page:
Feel free to me at any time

43. What is needed here?