1. http://www.ics.uci.edu/~dramanan/teaching/cs116_fall08/index.html Check web page often T,R 12:30-1:50pm PSCB (Phy Sci Class Blg) 140 Course intro handout Computational Photography and Vision (CS116) Deva Ramanan dramanan@ics.uci.edu Slides adapted from Alyosha Efros, Rick Szeliski and Steve Seitz

2. Agenda Intros Computational photography/vision overview Course overview Image processing – let’s dive in Readings (due next class) Online Book (link from course website): Richard Szeliski, Computer Vision: Algorithms and Applications –Intro: Ch 1.0 & 2.1 Ungraded HW (due next class) MATLAB tutorial (link from course website)

3. About me Deva Ramanan Relatively new faculty (2cnd class ever!) My research focus is on computer vision Part of new Computational Vision Lab http://vision.ics.uci.edu/

4. What is computational photography? Convergence of image processing, computer vision, computer graphics and photography Digital photography: Simply replaces traditional sensors and recording by digital technology Involves only simple image processing Computational photography More elaborate image manipulation, more computation New types of media (panorama, 3D, etc.) Camera design that take computation into account

5. GRAPHICS What is computer graphics? (3D->2D) 3D geometry physics Simulation projection

6. What is computer vision? (2D->3D) 3D geometry physics Estimation

7. What is computer vision? Terminator 2

8. Every picture tells a story Goal of computer vision is to write computer programs that can interpret images

9. Can computers match (or beat) human vision? Yes and no (but mostly no!) humans are much better at “hard” things computers can be better at “easy” things

10. Human perception has its shortcomings… Sinha and Poggio, Nature, 1996

11. Copyright A.Kitaoka 2003A.Kitaoka

12. Current state of the art The next slides show some examples of what current vision systems can do

13. Earth viewers (3D modeling) Image from Microsoft’s Virtual EarthVirtual Earth (see also: Google Earth)Google Earth

14. Photo Tourism http://phototour.cs.washington.edu/ http://labs.live.com/photosynth/

15. Optical character recognition (OCR) Digit recognition, AT&T labs http://www.research.att.com/~yannhttp://www.research.att.com/~yann/ Technology to convert scanned docs to text If you have a scanner, it probably came with OCR software License plate readers http://en.wikipedia.org/wiki/Automatic_number_plate_recognition

16. Face detection Many new digital cameras now detect faces Canon, Sony, Fuji, …

17. Smile detection? Sony Cyber-shot® T70 Digital Still Camera

18. Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart camera is flush-mounted in the checkout lane, continuously watching for items. When an item is detected and recognized, the cashier verifies the quantity of items that were found under the basket, and continues to close the transaction. The item can remain under the basket, and with LaneHawk,you are assured to get paid for it… “

19. Face recognition Who is she?

20. Vision-based biometrics “How the Afghan Girl was Identified by Her Iris Patterns” Read the storystory

21. Login without a password… Fingerprint scanners on many new laptops, other devices Face recognition systems now beginning to appear more widely http://www.sensiblevision.com/ http://www.sensiblevision.com/

22. Object recognition (in mobile phones) This is becoming real: Microsoft Research Point & Find, NokiaPoint & FindNokia

23. The Matrix movies, ESC Entertainment, XYZRGB, NRC Special effects: shape capture

24. Pirates of the Carribean, Industrial Light and Magic Special effects: motion capture

25. Special effects: image-based rendering

26. Sports Sportvision first down line Nice explanation on www.howstuffworks.comexplanation

27. Smart cars Mobileye Vision systems currently in high-end BMW, GM, Volvo models By 2010: 70% of car manufacturers. Slide content courtesy of Amnon Shashua

28. Vision-based interaction (and games) Nintendo Wii has camera-based IR tracking built in. See Lee’s work at CMU on clever tricks on using it to create a multi-touch display!Lee’s work at CMU multi-touch display DigimaskDigimask: put your face on a 3D avatar. “Game turns moviegoers into Human Joysticks”, “Game turns moviegoers into Human Joysticks”, CNET Camera tracking a crowd, based on this work.this work

29. Robotics http://www.robocup.org/ NASA’s Mars Spirit Rover http://en.wikipedia.org/wiki/Spirit_rover

30. Medical imaging Image guided surgery Grimson et al., MIT 3D imaging MRI, CT

31. Current state of the art You just saw examples of current systems. Many of these are less than 5 years old This is a very active research area, and rapidly changing Many new apps in the next 5 years To learn more about vision applications and companies David Lowe maintains an excellent overview of vision companiesDavid Lowe –http://www.cs.ubc.ca/spider/lowe/vision.htmlhttp://www.cs.ubc.ca/spider/lowe/vision.html

32. This course http://www.ics.uci.edu/~dramanan/teaching/cs116_fall08/index.html Prerequisites Calculus, linear algebra + probability helpful Interest in playing with images Emphasis on programming projects! Best way to learn is to build something from scratch MATLAB has a low learning curve 5 projects (15% of grad) + final exam (25%) Project due every 2 weeks For larger projects, “part 1” due first week

33. Project 1: Demosaicing 1)Get feet wet with MATLAB 2)Turn raw output of digital camera into a color image

34. Project 2: hole-filling and blending The fun stuff! Tools: bayesian modelling, differential equations

35. Project 3: Image re-targeting Click on video Tools: combinatorial optimization, dynamic programming

36. Project 4: Automatic mosaicing Tools: linear algebra, signal processing

37. Project 5: Face detection & recognition Tools: probabilistic modeling

38. Cameras Don’t need for class, but really cool Digital SLRs are ideal Point–and-shoots still nice and not too expensive (<$200) e.g. Canon A550

39. References There is no required text. Various course notes and papers will be made available. We will often use an online draft of an upcoming book: Richard Szeliski, Computer Vision: Algorithms and Applications There is a number of other fine texts that you can use for general reference: Computer Vision: The Modern Approach, Forsyth and Ponce Vision Science: Photons to Phenomenology, Stephen Palmer Multiple View Geometry in Computer Vision, Hartley & Zisserman The Computer Image, Watt and Policarpo Linear Algebra and its Applications, Gilbert Strang

40. A little bit of teaching philosophy… 1) Prefer discussion vs lectures – ask questions! 2) Readings before class help ‘set the stage’ 3) We learn best by doing it ourselves – progamming projects important! 4) Don’t like powerpoint (makes students sleep) -But visual aides are nice -Slides will be available online *after* class -I encourage you to take notes during class

41. Favor to ask… Need to boost enrollment - spread the gospel about this cool class!