Presentation on theme: "1 Visual Perception in Humans and Machines Kostas Daniilidis Assistant Professor GRASP Lab University of Pennsylvania."— Presentation transcript:
1 Visual Perception in Humans and Machines Kostas Daniilidis Assistant Professor GRASP Lab University of Pennsylvania
2 Examples How do we (humans) recognize faces ? Make a machine find President Clintons face in the web
4 An interdisciplinary definition Computer Vision is devoted to the discovery of algorithms, representations, and architectures that embody the principles of visual capabilities. What are visual capabilities? –Recognizing objects and faces –Estimating shapes and distances –Moving, grasping
5 Relation to other fields Computer Vision is inspired from Biological Vision (Phenomenology and Models in Psychophysics and in Neurobiology) but does not try to imitate the nature's architecture or algorithms. Biological Vision and Psychophysics may find computational models discovered in Computer Vision useful for explaining nature.
6 Target problem in computer vision Compute properties of the 3D world from one or more digital images These properties may be –dynamic (observer and object motion) –geometric (distances, object shape) –enabling recognition The result may be an action (grasp an object, avoid an obstacle)
7 What is an image ? A gray-value image is just a set of numbers (usually from 0 to 255)
8 An image is a set of numbers
9 An image is a surface I(x,y)
10 Basic image processing operations Smoothing and Noise Removal
11 Blur removal
12 Edge detection X derivativeX derivative Y derivativeY derivative Gradient magnitudeGradient magnitude After thresholdingAfter thresholding
13 (Sub) sampling Shannon Theorem: Sampling frequency must be greater than the maximal frequency in the image (therefore smooth before subsample)
14 Brightness perception
15 How do we perceive distances? Perspective distortion in texture, contour, shading, and a-priori knowledge Stereopsis (what most people believe) Motion
16 The Eye as a Pinhole Camera: Perspective Projection u = X/Z Z X
19 Quiz From which points in space is a rectangle viewed as a square (more difficult: an ellipse viewed as a circle ? Be careful: The center ofBe careful: The center of the ellipse in the image is not the projection of the center of the ellipse on the floor!
20 The power of vanishing points Perspective projection preserves cross-ratio = AC/AD : BC/BD is the same on the street and in the image. If A is a vanishing point AC/AD = 1. A B C D We measure A,B,C,D in pixels in the image and form cross ratio for image and for the street. BC is computed from the equality of the two ratios.
21 Stereopsis Infer depth from the disparity between the positions of the same feature in left and right image
22 Stereo Reconstruction
23 Stereo-disparity estimation Search at every pixel for candidates of maximum correlation between left and right Estimate 3D-coordinates of point
24 The power of visual motion Most of the animals have monocular vision (left and right visual fields do not overlap) 8% of the population can not see stereo Stereopsis is limited to a very short depth of field (10m).
26 Kinetic depth effect (moving dots)
27 Self- and object-motion
28 Motion artifacts
29 Structure from Motion Given a sequence of images find 1. Ego-motion 2. 3D-structure 3. Independent motions applying only the assumption of rigidity.
30 Motion Field and Heading Direction
31 Depth map
32 Temporal aliasing Wagon Wheel Illusion: A wheel with a periodic radial pattern is perceived to move backwards depending on the relation between the speed, the radius of the wheel, and the period of the pattern (www.cstr.ed.ac.uk/~rjc/wagonWheel)
33 Aperture problem Inside a small aperture displaying a small line we can estimate only the motion direction perpendicular to the line.
34 My wife and my mother-in-law The role of the focus of attention