Presentation is loading. Please wait.

Presentation is loading. Please wait.

Introduction to Computer Vision CS / ECE 181B Tuesday, March 30, 2004 Prof. B. S. Manjunath ECE/CS Department  Course introduction and overview [thanks.

Similar presentations


Presentation on theme: "Introduction to Computer Vision CS / ECE 181B Tuesday, March 30, 2004 Prof. B. S. Manjunath ECE/CS Department  Course introduction and overview [thanks."— Presentation transcript:

1 Introduction to Computer Vision CS / ECE 181B Tuesday, March 30, 2004 Prof. B. S. Manjunath ECE/CS Department  Course introduction and overview [thanks to Professor Matthew Turk, CS department for the slides from S2003]

2 CS/ECE 181b2 Web site Last Year (Prof. Turk) –http://www.cs.ucsb.edu/~cs181bhttp://www.cs.ucsb.edu/~cs181b –Has some useful information.

3 CS/ECE 181b3 Text Book NO Text Book –Follow the classroom discussions –Handouts and Lecture slides Reference Books (not required) –Forsyth and Ponce, “Computer Vision: A Modern Approach” (was the text used last year) –Nalwa, “A Guided Tour of Computer Vision”; a good introductory book for casual reading –Jain and Kasturi, “Machine Vision”; I used it in –Horn, “Robot Vision”. A classic, but outdated.

4 CS/ECE 181b4 Computer Vision What is computer vision? –“Making computers see” Nice sunset! “Extracting descriptions of the world from pictures or sequences of pictures”

5 CS/ECE 181b5 Computer Vision Also known as image understanding, machine vision, computational vision CV is about interpreting the content of images* –Field is years old –Child of AI Vision is easy, right? Just open your eyes! –No, it’s a hard problem! –Much of your very complex brain is devoted to doing vision –It involves cognition, navigation, manipulation and learning  Not just simple “match a feature vector to a database” tasks

6 CS/ECE 181b6 Relation to other fields Image processing Pattern recognition Computer graphics Machine learning Probability and statistics Geometry Optics Perception AI, robotics

7 CS/ECE 181b7 Computer Graphics Image Output: Model Synthetic Camera Projection, shading, lighting models

8 CS/ECE 181b8 Computer Vision Model Output: Real Scene Cameras Images

9 CS/ECE 181b9 Aims of Computer Vision Automate visual perception Construct scene descriptions from images Make useful decisions about real physical objects and scenes based on sensed images Produce symbolic (perhaps task-dependent) descriptions from images Produce from images of the external world a useful description that is not cluttered with irrelevant information Support tasks that require visual information

10 CS/ECE 181b10 Some applications of computer vision Photogrammetry, GIS –Commercial, military, government Robotics –Industrial, military, medical, space, entertainment Inspection, measurement Medical imaging –Automatic detection, outlining, measurement Graphics and animation, special effects Surveillance and security Multimedia database indexing and retrieval, compression Human-computer interaction (VBI)

11 CS/ECE 181b11 What Does Computer Vision Do? 3D m 3D models of objects Object recognition Navigation Event/action recognition …

12 CS/ECE 181b12 Vision Transforms From This… A D 1D E 0C 0A 0A 0A B A 08 0B 0A 0D 0B 0B 0C F 0B E 0C A 0A 0B 0F 0A 0C B 07 0B 05 0B E 0C A A A C 0B 0A A 0C 0A 0A 08 0A 0A A B 14 0F 0F 0D 0A 0E 0A 0C 0C 0E 0A 0C 0B 09 0A 09 0A 0A 09 0B 0B 05 0C 0C 0A B 10 0A 0D 0D 0B 0D 0C 0B 0B 0C 0D 0B 0B 0A 0A 0A 0B 0C C 15 0D E C 0D 0C 0C 0A 0B 0B 09 0C 0F D 07 0B D D 0A F 0E B 0C 0F 0F 0E 0C 10 0D E 3F A 09 0A C B 10 0F 0C D 0F 0D 0D 0B 25 7A 7F 79 6D 80 6E 54 0C 0D 09 0A F 0D 12 0E 10 0E 0F F E 8C 73 7A 5C 1E 05 0A 0F 0E 0C F 15 0D D B A 12 0A 0B E 1A F 7D C 7F 6F 5F 0B D 0C F 16 0F D E 21 1F 1C 1D 2D 1A 2D 7C 7A 95 6B C 0A C C A 0D 1A 1A B 3E C 83 5E 7B 94 8A 5A 3D C F 0C A F 1F F 93 B4 AE D F F B F 3D 3E 42 3B 45 2E F 96 6B 24 0F 22 4B C3 A4 3F 4F 0C F C D D A 27 3B 33 A8 A B A1 75 4B AC A1 B5 79 0C 0B 13 0F 0B B 1D 1C 1C 1C 1B 1B 1E A 24 9F AD AC AA B1 9C 8D 5F 3E 98 B7 B7 A A 0D D E 36 5B 29 2C F AF BC AF AB 9E A F AE AD A E 0A 0C B B 1A 1A 2B 1B 2A C 1B 26 4C 40 BA BB B5 AE A 8A 9A B9 BB AD 9C 8A B 0D 0F 0B A 18 1C 1E D 3F 4E B 1B AF AB B1 AC A AA 9F AD 7F 0C 0B 0E 0B 0C 0C A 21 1E 2E 1B E B AA AC B7 AF A6 9A 93 8F 85 7F A0 A4 C2 9F 99 4E A 0D 0C 0A 0C C 1F 2B 1C 8B 9B 42 9B A7 A1 B4 B0 AA A0 9D E A A 0A 0D 0D 09 0D 0C A 1C A C C 90 A1 39 A0 97 B8 AA B2 A5 A6 A D 08 0D C 0B 0E 0D 0D 0A 04 1E 29 1F E 41 4A 2C A A5 89 9E A3 B0 B7 AF AB AB A C C 0D 0B 01 1F A 2C 36 4D B AA 7E AD B3 AA B2 A8 B E 9E 8E A 0D 0D 0D 0F 0C E C 2A A 39 4F B A5 86 AA B2 B3 AE A0 A3 9C B 25 2D 07 0E C 0A 0F 0D 09 0C F 2A C 93 A3 AC 60 BA BD B4 AE A8 A F 52 4F 3F 09 0D 0D 09 0E 0E 0B 12 0B 0B E 2C 29 2A 3B 30 4E 3C E AE 9F A4 B1 4E AA AA A0 A4 9C 94 A2 AB A E 0E 09 0B 0D 10 0C 0C E D 5A B 8C A3 AC A5 3E A1 AF A8 82 A4 AC A B 0B 0B 0E 0F A E 3C 3F 3E C 5C A A2 A2 A6 8E 4E AC A6 A2 89 7E 5B 11 0E D 0C 0D 0C 00 4B C 2E 2D B 0D C A8 B5 AA B3 AE A0 9C 8C 62 0A D E 0D C 24 2E B B 2B A1 AB AC B2 A6 A6 A F 10 1C F 0C 0F A C F 33 1F B 4D A3 A5 99 8D 7A 4E 0E 1B F 0F B 01 1D 1F 2B F 40 2F 2D 2A 25 2B 2C E 55 5E 62 6D 6D 6E 68 5E 43 0D A F 2A B 45 2E 3A D 2F 1F 1E 1B C 3F 3C D 0B 0E 11 1E 23 1B D 10 0F 12 0F A 1B 35 4A 1D 20 2C 2F 1F 1F 3B 34 1A 2A E 0C 0C 06 0C B B 0E 10 0D 0D 0F D 1E C E 0E 1A A 18 2C 2E 19 0F 0D 10 0E 0E 14 0D B 1E 17 1C 1F 1F 1F 1C C 2F B D D A 1D 1F 1D 1B 1E 1B B 22 1A 1E 1B C 0D 11 0E 12 0D C 2E E 20 1F 1F 1D 1B 1C B 1E 15 1A A E 1D B 1B A 4C 33 1C E A E

13 CS/ECE 181b13 To this… Objects –Cat, chair, window, star, bush, water, a shoe, my mother… Properties –Big, bright, yellow, fast, graspable, moving… Relations –In front, behind, on top, next to, larger, closer, identical… Shapes –Round, rectangular, star-shaped, symmetric… Textures –Rough, smooth, irregular… Movement –Turning, looming, rolling…

14 CS/ECE 181b14 “A harbor with many dozens of boats; water is calm and glassy; masts are all vertical; mountains in background, blue sky with a touch of clouds…”

15 CS/ECE 181b15 “J548043”

16 CS/ECE 181b16 “Hallway straight ahead”

17 CS/ECE 181b17 “Angry Surprised Happy Disgusted”

18 CS/ECE 181b18 Why is Vision Difficult? Consider the input... Not this

19 CS/ECE 181b A D 1D E 0C 0A 0A 0A B A 08 0B 0A 0D 0B 0B 0C F 0B E 0C A 0A 0B 0F 0A 0C B 07 0B 05 0B E 0C A A A C 0B 0A A 0C 0A 0A 08 0A 0A A B 14 0F 0F 0D 0A 0E 0A 0C 0C 0E 0A 0C 0B 09 0A 09 0A 0A 09 0B 0B 05 0C 0C 0A B 10 0A 0D 0D 0B 0D 0C 0B 0B 0C 0D 0B 0B 0A 0A 0A 0B 0C C 15 0D E C 0D 0C 0C 0A 0B 0B 09 0C 0F D 07 0B D D 0A F 0E B 0C 0F 0F 0E 0C 10 0D E 3F A 09 0A C B 10 0F 0C D 0F 0D 0D 0B 25 7A 7F 79 6D 80 6E 54 0C 0D 09 0A F 0D 12 0E 10 0E 0F F E 8C 73 7A 5C 1E 05 0A 0F 0E 0C F 15 0D D B A 12 0A 0B E 1A F 7D C 7F 6F 5F 0B D 0C F 16 0F D E 21 1F 1C 1D 2D 1A 2D 7C 7A 95 6B C 0A C C A 0D 1A 1A B 3E C 83 5E 7B 94 8A 5A 3D C F 0C A F 1F F 93 B4 AE D F F B F 3D 3E 42 3B 45 2E F 96 6B 24 0F 22 4B C3 A4 3F 4F 0C F C D D A 27 3B 33 A8 A B A1 75 4B AC A1 B5 79 0C 0B 13 0F 0B B 1D 1C 1C 1C 1B 1B 1E A 24 9F AD AC AA B1 9C 8D 5F 3E 98 B7 B7 A A 0D D E 36 5B 29 2C F AF BC AF AB 9E A F AE AD A E 0A 0C B B 1A 1A 2B 1B 2A C 1B 26 4C 40 BA BB B5 AE A 8A 9A B9 BB AD 9C 8A B 0D 0F 0B A 18 1C 1E D 3F 4E B 1B AF AB B1 AC A AA 9F AD 7F 0C 0B 0E 0B 0C 0C A 21 1E 2E 1B E B AA AC B7 AF A6 9A 93 8F 85 7F A0 A4 C2 9F 99 4E A 0D 0C 0A 0C C 1F 2B 1C 8B 9B 42 9B A7 A1 B4 B0 AA A0 9D E A A 0A 0D 0D 09 0D 0C A 1C A C C 90 A1 39 A0 97 B8 AA B2 A5 A6 A D 08 0D C 0B 0E 0D 0D 0A 04 1E 29 1F E 41 4A 2C A A5 89 9E A3 B0 B7 AF AB AB A C C 0D 0B 01 1F A 2C 36 4D B AA 7E AD B3 AA B2 A8 B E 9E 8E A 0D 0D 0D 0F 0C E C 2A A 39 4F B A5 86 AA B2 B3 AE A0 A3 9C B 25 2D 07 0E C 0A 0F 0D 09 0C F 2A C 93 A3 AC 60 BA BD B4 AE A8 A F 52 4F 3F 09 0D 0D 09 0E 0E 0B 12 0B 0B E 2C 29 2A 3B 30 4E 3C E AE 9F A4 B1 4E AA AA A0 A4 9C 94 A2 AB A E 0E 09 0B 0D 10 0C 0C E D 5A B 8C A3 AC A5 3E A1 AF A8 82 A4 AC A B 0B 0B 0E 0F A E 3C 3F 3E C 5C A A2 A2 A6 8E 4E AC A6 A2 89 7E 5B 11 0E D 0C 0D 0C 00 4B C 2E 2D B 0D C A8 B5 AA B3 AE A0 9C 8C 62 0A D E 0D C 24 2E B B 2B A1 AB AC B2 A6 A6 A F 10 1C F 0C 0F A C F 33 1F B 4D A3 A5 99 8D 7A 4E 0E 1B F 0F B 01 1D 1F 2B F 40 2F 2D 2A 25 2B 2C E 55 5E 62 6D 6D 6E 68 5E 43 0D A F 2A B 45 2E 3A D 2F 1F 1E 1B C 3F 3C D 0B 0E 11 1E 23 1B D 10 0F 12 0F A 1B 35 4A 1D 20 2C 2F 1F 1F 3B 34 1A 2A E 0C 0C 06 0C B B 0E 10 0D 0D 0F D 1E C E 0E 1A A 18 2C 2E 19 0F 0D 10 0E 0E 14 0D B 1E 17 1C 1F 1F 1F 1C C 2F B D D A 1D 1F 1D 1B 1E 1B B 22 1A 1E 1B C 0D 11 0E 12 0D C 2E E 20 1F 1F 1D 1B 1C B 1E 15 1A A E 1D B 1B A 4C 33 1C E A E E 2D 2D 23 1D A 5F A 1B B F B E 1B C 5E F F 21 1B F A 1D B0 2C C 22 1D 1A 10 1A 1D 1A C 21 1B But this…

20 CS/ECE 181b20 Some Possible Outputs depthorsegmentation object pose (facing away, facing forward) actionunderstanding objectrecognition

21 CS/ECE 181b21 What Your Brain Does Almost certain to be Bill Clinton Dark circular overlay Gray hair Neck Right ear Woman’s dress suit Armani suit White shirt Left eye (open) CNN caption (Washington 1995?) Clinton occluding Monica Person contour Person with glasses in crowd Nose Cheek Monica’s mouth (smiling) Lapel Necklace Right eye (open) Dark brown hair Pony tail Clinton greeting Lewinsky Monica Lewinsky Illuminated from above

22 CS/ECE 181b22 Variation in Appearance

23 CS/ECE 181b23 What makes computer vision hard? Underspecified problem! –3D world projected onto 2D sensor(s) Environment –Lighting, background, movement, camera Varying appearance of objects Calibration, FOV, camera control, image quality Computational complexity (speed of processing) Etc., etc….

24 CS/ECE 181b24 Progress in Computer Vision First generation: Military/Early Research –Few systems, each custom-built, cost $Ms –“Users” have PhDs –1 hour per frame Second generation: Industrial/Medical –Numerous systems, of each, cost $10Ks –Users have college degree –RT with special hardware Third generation: Consumer –100000(00) systems, cost $100s –Users have little or no training –RT in software

25 CS/ECE 181b25 Examples CMU NavLab Cognex MIT Media Lab

26 CS/ECE 181b26 Computer Vision Areas of study –Camera (sensor) calibration and image formation –Binary image analysis –Edge detection and low-level filtering –Color representation and segmentation –Texture description and segmentation –Depth/shape from X –Stereopsis –Optical flow –Motion computation –Object matching, detection, and recognition –3D sensing and shape description –Object and scene tracking

27 CS/ECE 181b27 Camera calibration Original planar pattern Corrected for radial lens distortion Two calibrated cameras

28 CS/ECE 181b28 Edge detection

29 CS/ECE 181b29 Stereopsis

30 CS/ECE 181b30 Multi-Camera Stereo Images 3D reconstruction

31 CS/ECE 181b31 Motion computation Optical flow

32 CS/ECE 181b32 Object Detection and Recognition Model, detect, and recognize objects in images Main issues –Object representation, feature extraction, matching, learning from data Faces Cars

33 CS/ECE 181b33 Texture analysis

34 CS/ECE 181b34 Some Computer Vision Examples Academic and industrial research Companies and products

35 CS/ECE 181b35 Early CV research

36 CS/ECE 181b36 Multimedia database applications

37 CS/ECE 181b37 Labeling images

38 CS/ECE 181b38 Vision for HCI

39 CS/ECE 181b39 Eyematic Interfaces Vision and computer graphics

40 CS/ECE 181b40 Vision for Biometrics Viisage

41 CS/ECE 181b41 Examples of Commercial Vision Systems Toshiba Hand Motion Processor Eyematic Interfaces Point Gray Research

42 CS/ECE 181b42 Example application: Face recognition If we could reliably detect and recognize faces in images… –Digital libraries  Find me the picture of my brother and Ted  Show all the scenes with more than four people –Surveillance  Alert the operator when a known suspect enters the area –HCI  Automatically log in as I walk up to the computer  Refuse to open my files when someone else is at my machine –Military  Friend or foe? –Personal/entertainment robots  Follow the master, look the customer in the eyes

43 CS/ECE 181b43 So… why study computer vision? “Engineering” reasons –Images are become more and more ubiquitous –Plenty of useful applications –Great mixture of applied mathematics, signal processing, computer science, etc. –Solving interesting, underconstrained (“ill-posed”) problems –Etc. “Science” reasons –How does the brain do it? –Understanding biological vision –Deep computational issues –Etc.

44 CS/ECE 181b44 © YF Wang

45 CS/ECE 181b45 Psychology, Neurology, and Physiology Understand human (biological) perception –Study the structure of the eye, and the neural-networks for visual information processing –Design computational algorithms to evaluate biological visual perception functions

46 CS/ECE 181b46 Human vision—not perfect.. Check out the illusions…and ambiguities..

47 CS/ECE 181b47

48 CS/ECE 181b48

49 CS/ECE 181b49

50 CS/ECE 181b50

51 CS/ECE 181b51

52 CS/ECE 181b52 This Quarter… Vision is unlike any other course you may have taken in engineering/science. There are MANY questions, very few answers! Remember this as we go through this subject. We are so familiar with seeing, that it takes a leap of imagination to realize that there are problems to be solved ---- Richard Gregory Vision is a bag of tricks --- Ramachandran

53 CS/ECE 181b53 (tentative) Grading Homeworks & Programming Assignments (25%) Midterm (15%) Project on Face Recognition (20%) Final Examination (40%)

54 CS/ECE 181b54 Project Topic: Face recognition More details soon. Individual or groups of no more than 4 students.

55 CS/ECE 181b55 What will we study? Part I: The physics of imaging –Camera models and calibration –Radiometry –Color Part II: “Early” vision –Filtering and edge detection –Stereo, optical flow Part III: “Mid-level” vision –Segmentation –Classification –Tracking Part IV: “High-level” vision and applications –Model-based vision –Various applications of computer vision

56 CS/ECE 181b56 Course web site this is last year’s web site that you might find very useful.http://www.cs.ucsb.edu/~cs181b: Visit it often!


Download ppt "Introduction to Computer Vision CS / ECE 181B Tuesday, March 30, 2004 Prof. B. S. Manjunath ECE/CS Department  Course introduction and overview [thanks."

Similar presentations


Ads by Google