Why is computer vision difficult?

Slides:



Advertisements
Similar presentations
Lecture 2: Convolution and edge detection CS4670: Computer Vision Noah Snavely From Sandlot ScienceSandlot Science.
Advertisements

Spatial Filtering (Chapter 3)
Motion illusion, rotating snakes. Slide credit Fei Fei Li.
Computational Photography CSE 590 Tamara Berg Filtering & Pyramids.
Computational Photography Prof. Feng Liu Spring /06/2015.
Lecture 2: Filtering CS4670/5670: Computer Vision Kavita Bala.
1 Image filtering Hybrid Images, Oliva et al.,
Computer Vision Introduction to Image formats, reading and writing images, and image environments Image filtering.
Lecture 1: Images and image filtering
1 Image Filtering Readings: Ch 5: 5.4, 5.5, 5.6,5.7.3, 5.8 (This lecture does not follow the book.) Images by Pawan SinhaPawan Sinha formal terminology.
Linear filtering. Overview: Linear filtering Linear filters Definition and properties Examples Gaussian smoothing Separability Applications Denoising.
Announcements Kevin Matzen office hours – Tuesday 4-5pm, Thursday 2-3pm, Upson 317 TA: Yin Lou Course lab: Upson 317 – Card access will be setup soon Course.
1 Image filtering Images by Pawan SinhaPawan Sinha.
1 Image filtering
Lecture 2: Image filtering
Matlab Tutorial Continued Files, functions and images.
Linear filtering.
1 Image filtering Hybrid Images, Oliva et al.,
Linear Filtering About modifying pixels based on neighborhood. Local methods simplest. Linear means linear combination of neighbors. Linear methods simplest.
Most slides from Steve Seitz
Chapter 3 (cont).  In this section several basic concepts are introduced underlying the use of spatial filters for image processing.  Mainly spatial.
Lecture 1: Images and image filtering CS4670/5670: Intro to Computer Vision Kavita Bala Hybrid Images, Oliva et al.,
Filtering Course web page: vision.cis.udel.edu/cv March 5, 2003  Lecture 9.
Applications of Image Filters Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/04/10.
CS559: Computer Graphics Lecture 3: Digital Image Representation Li Zhang Spring 2008.
CSC 589 Introduction to Computer Vision Lecture 2 linear filtering Instructor: Bei Xiao Thursday, January 15th Salvador Dali “Gala Contemplating the Mediterranean.
Lecture 03 Area Based Image Processing Lecture 03 Area Based Image Processing Mata kuliah: T Computer Vision Tahun: 2010.
Linear Filters Monday, Jan 24 Prof. Kristen Grauman UT-Austin …
Linear Filters August 27 th 2015 Devi Parikh Virginia Tech 1 Slide credit: Devi Parikh Disclaimer: Many slides have been borrowed from Kristen Grauman,
Image Filtering Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/02/10.
Linear filtering. Motivation: Noise reduction Given a camera and a still scene, how can you reduce noise? Take lots of images and average them! What’s.
CSE 473/573 Computer Vision and Image Processing (CVIP) Ifeoma Nwogu Lecture 7 – Linear Filters 1.
Visual Computing Computer Vision 2 INFO410 & INFO350 S2 2015
CS 691B Computational Photography
CSE 6367 Computer Vision Image Operations and Filtering “You cannot teach a man anything, you can only help him find it within himself.” ― Galileo GalileiGalileo.
Pixels and Image Filtering Computational Photography Derek Hoiem 08/26/10 Graphic:
Lecture 3: Filtering and Edge detection
Lecture 5: Fourier and Pyramids
Reconnaissance d’objets et vision artificielle
Linear filtering. Motivation: Image denoising How can we reduce noise in a photograph?
Instructor: Mircea Nicolescu Lecture 5 CS 485 / 685 Computer Vision.
Lecture 8: Edges and Feature Detection
CSE 185 Introduction to Computer Vision Image Filtering: Spatial Domain.
Grauman Today: Image Filters Smooth/Sharpen Images... Find edges... Find waldo…
Images and Filters CSEP 576 Ali Farhadi Many slides from Steve Seitz and Larry Zitnick.
Lecture 1: Images and image filtering CS4670/5670: Intro to Computer Vision Noah Snavely Hybrid Images, Oliva et al.,
CS558 C OMPUTER V ISION Lecture IV: Image Filter and Edge Detection Slides adapted from S. Lazebnik.
- photometric aspects of image formation gray level images
Linear Filters T-11 Computer Vision University of Houston
Linear Filters April 6th, 2017
Filtering – Part I Gokberk Cinbis Department of Computer Engineering
Lecture 1: Images and image filtering
Image filtering Hybrid Images, Oliva et al.,
Recap from Wednesday Spectra and Color Light capture in cameras and humans.
Image gradients and edges April 11th, 2017
Image filtering Images by Pawan Sinha.
Image filtering Images by Pawan Sinha.
Image filtering Images by Pawan Sinha.
Motion illusion, rotating snakes
Linear filtering.
Lecture: Image manipulations (2) Filters and kernels
Most slides from Steve Seitz
Image filtering Images by Pawan Sinha.
Linear Filtering CS 678 Spring 2018.
Image filtering
Image filtering
Lecture 2: Image filtering
Lecture 1: Images and image filtering
Image Filtering Readings: Ch 5: 5. 4, 5. 5, 5. 6, , 5
Most slides from Steve Seitz
Presentation transcript:

Why is computer vision difficult? Viewpoint variation Scale Illumination

Why is computer vision difficult? Motion (Source: S. Lazebnik) Intra-class variation Occlusion Background clutter

Challenges: local ambiguity slide credit: Fei-Fei, Fergus & Torralba

But there are lots of cues we can exploit… Source: S. Lazebnik

Bottom line Perception is an inherently ambiguous problem Many different 3D scenes could have given rise to a particular 2D picture We often need to use prior knowledge about the structure of the world Image source: F. Durand

Course overview (tentative) Low-level vision image processing, edge detection, feature detection, cameras, image formation Geometry and algorithms projective geometry, stereo, structure from motion, Markov random fields Recognition face detection / recognition, category recognition, segmentation Light, color, and reflectance Advanced topics

Projects (tentative) Roughly five projects First one will be done solo, others in groups You can discuss the projects on a whiteboard, but all code must be your (or your group’s) own First project to be released today or tomorrow

Project: Image Scissors

Project: Feature detection and matching

Project: Creating panoramas

Object category recognition Project: Recognition Location recognition Face recognition Object category recognition

Grading Occasional quizzes (at the beginning of class) One prelim, one final exam Rough grade breakdown: Quizzes: 5% Midterm: 15% Programming projects: 60% Final exam: 15%

Late policy Two “late days” will be available for the semester Late projects will be penalized by 25% for each day it is late, and no extra credit will be awarded.

Questions?

Lecture 1: Images and image filtering CS4670/5670: Intro to Computer Vision Noah Snavely Lecture 1: Images and image filtering Hybrid Images, Oliva et al., http://cvcl.mit.edu/hybridimage.htm

Lecture 1: Images and image filtering CS4670: Computer Vision Noah Snavely Lecture 1: Images and image filtering Hybrid Images, Oliva et al., http://cvcl.mit.edu/hybridimage.htm

Lecture 1: Images and image filtering CS4670: Computer Vision Noah Snavely Lecture 1: Images and image filtering Hybrid Images, Oliva et al., http://cvcl.mit.edu/hybridimage.htm

Lecture 1: Images and image filtering CS4670: Computer Vision Noah Snavely Lecture 1: Images and image filtering Hybrid Images, Oliva et al., http://cvcl.mit.edu/hybridimage.htm

Reading Szeliski, Chapter 3.1-3.2

What is an image?

What is an image? We’ll focus on these in this class Digital Camera We’ll focus on these in this class (More on this process later) The Eye Source: A. Efros

= What is an image? A grid (matrix) of intensity values (common to use one byte per value: 0 = black, 255 = white) 255 20 75 95 96 127 145 175 200 47 74 =

What is an image? We can think of a (grayscale) image as a function, f, from R2 to R: f (x,y) gives the intensity at position (x,y) A digital image is a discrete (sampled, quantized) version of this function x y f (x, y) snoop 3D view

Image transformations As with any function, we can apply operators to an image We’ll talk about a special kind of operator, convolution (linear filtering) g (x,y) = f (x,y) + 20 g (x,y) = f (-x,y)

Question: Noise reduction Given a camera and a still scene, how can you reduce noise? Answer: take lots of images, average them Take lots of images and average them! What’s the next best thing? Source: S. Seitz

Image filtering Modify the pixels in an image based on some function of a local neighborhood of each pixel 5 1 4 7 3 10 Some function 7 Local image data Modified image data Source: L. Zhang

Linear filtering One simple version: linear filtering (cross-correlation, convolution) Replace each pixel by a linear combination (a weighted sum) of its neighbors The prescription for the linear combination is called the “kernel” (or “mask”, “filter”) 6 1 4 8 5 3 10 0.5 1 8 Local image data kernel Modified image data Source: L. Zhang

Cross-correlation Let be the image, be the kernel (of size 2k+1 x 2k+1), and be the output image Can think of as a “dot product” between local neighborhood and kernel for each pixel This is called a cross-correlation operation:

Convolution Same as cross-correlation, except that the kernel is “flipped” (horizontally and vertically) Convolution is commutative and associative This is called a convolution operation:

Convolution Adapted from F. Durand

Mean filtering 90 10 20 30 40 60 90 50 80 * = 1

Linear filters: examples * 1 = Original Identical image Source: D. Lowe

Linear filters: examples * 1 = Original Shifted left By 1 pixel Source: D. Lowe

Linear filters: examples 1 * = Original Blur (with a mean filter) Source: D. Lowe

Linear filters: examples Sharpening filter (accentuates edges) 1 2 - * = Original Source: D. Lowe

Sharpening Source: D. Lowe

Smoothing with box filter revisited I always walk through the argument on the left rather carefully; it gives some insight into the significance of impulse responses or point spread functions. Source: D. Forsyth

Gaussian Kernel Source: C. Rasmussen

Gaussian filters = 1 pixel = 5 pixels = 10 pixels = 30 pixels

Mean vs. Gaussian filtering

Gaussian filter Removes “high-frequency” components from the image (low-pass filter) Convolution with self is another Gaussian Convolving twice with Gaussian kernel of width = convolving once with kernel of width * = Linear vs. quadratic in mask size Source: K. Grauman

Sharpening revisited = = What does blurring take away? – + α original smoothed (5x5) detail = – Let’s add it back: original detail + α sharpened = Source: S. Lazebnik

unit impulse (identity) Sharpen filter blurred image image unit impulse (identity) Gaussian scaled impulse Laplacian of Gaussian f + a(f - f * g) = (1+a)f-af*g = f*((1+a)e-g)

Sharpen filter unfiltered filtered

“Optical” Convolution Camera shake = * Source: Fergus, et al. “Removing Camera Shake from a Single Photograph”, SIGGRAPH 2006 Bokeh: Blur in out-of-focus regions of an image. Source: http://lullaby.homepage.dk/diy-camera/bokeh.html

Questions? For next time: Read Szeliski, Chapter 3.1-3.2