1. Advanced Computer Vision Devi Parikh Electrical and Computer Engineering

2. Plan for today Topic overview Introductions Course overview: –Logistics –Requirements Please interrupt at any time with questions or comments

3. Computer Vision Automatic understanding of images and video –Computing properties of the 3D world from visual data (measurement) –Algorithms and representations to allow a machine to recognize objects, people, scenes, and activities. (perception and interpretation) –Algorithms to mine, search, and interact with visual data (search and organization) Kristen Grauman

4. What does recognition involve? Fei-Fei Li

5. Detection: are there people?

6. Activity: What are they doing?

7. Object categorization mountain building tree banner vendor people street lamp

8. Instance recognition Potala Palace A particular sign

9. Scene and context categorization outdoor city …

10. Attribute recognition flat gray made of fabric crowded

11. Why recognition? Recognition a fundamental part of perception – e.g., robots, autonomous agents Organize and give access to visual content –Connect to information –Detect trends and themes Where are we now? Kristen Grauman

12. We’ve come a long way…

15. Posing visual queries Kooaba, Bay & Quack et al. Yeh et al., MIT Belhumeur et al. Kristen Grauman

16. Exploring community photo collections Snavely et al. Simon & Seitz Kristen Grauman

17. http://www.darpa.mil/grandchallenge/gallery.asp Autonomous agents able to detect objects Kristen Grauman

18. We’ve come a long way… Fischler and Elschlager, 1973

19. We’ve come a long way…

20. Dollar et al., BMVC 2009

21. Still a long way to go… Dollar et al., BMVC 2009

24. Challenges

25. Challenges: robustness IlluminationObject pose Viewpoint Intra-class appearance Occlusions Clutter Kristen Grauman

26. Challenges: context and human experience Context cues Kristen Grauman

27. Challenges: context and human experience Context cues FunctionDynamics Video credit: J. Davis Kristen Grauman

28. Challenges: scale, efficiency Half of the cerebral cortex in primates is devoted to processing visual information ~20 hours of video added to YouTube per minute ~5,000 new tagged photos added to Flickr per minute Thousands to millions of pixels in an image 30+ degrees of freedom in the pose of articulated objects (humans) 3,000-30,000 human recognizable object categories Kristen Grauman

29. Challenges: learning with minimal supervision More Less Cropped to object, parts and classes labeled Classes labeled, some clutter Unlabeled, multiple objects Kristen Grauman

30. Slide from Pietro Perona, 2004 Object Recognition workshop

32. Recognizing flat, textured objects (like books, CD covers, posters) Reading license plates, zip codes, checks Fingerprint recognition Frontal face detection What kinds of things work best today? Kristen Grauman

33. Inputs in 1963… L. G. Roberts, Machine Perception of Three Dimensional Solids, Ph.D. thesis, MIT Department of Electrical Engineering, 1963.Machine Perception of Three Dimensional Solids Kristen Grauman

34. Personal photo albums Surveillance and security Movies, news, sports Medical and scientific images Slide credit; L. Lazebnik … and inputs today

35. Images on the Web Movies, news, sports 916,271 titles 10 mil. videos, 65,000 added daily 350 mil. photos, 1 mil. added daily 1.6 bil. images indexed as of summer 2005 Satellite imageryCity streets Slide credit; L. Lazebnik Understand and organize and index all this data!!

36. Introductions What is your name? Which program are you in? How far along? What is your research area and current project about? –Take a minute to explain it to us –In a way that we can all follow Have you taken a computer vision course before? Machine learning or pattern recognition? What are you hoping to get out of this class?

37. This course ECE 5984 TR 3:30 pm to 4:45 pm Hutcheson (HUTCH) 207 Office hours: by appointment (email) Course webpage: http://filebox.ece.vt.edu/~S14ECE5984/ (Google me  My homepage  Teaching)

38. This course Focus on current research in computer vision High-level recognition problems, innovative applications.

39. Goals Understand state-of-the-art approaches Analyze and critique current approaches Identify interesting research questions Present clearly and methodically

40. Expectations Discussions will center on recent papers in the field [15%] Paper reviews each class [25%] –Can have 3 late days over the course of the semester Presentations (2-3 times) [25%] –Papers and background reading –Experiments Project [35%] No “Assignments”, Exams, etc.

41. Prerequisites Course in computer vision Courses in machine learning is a plus

42. Paper reviews For each class –Review one paper in detail –Review one paper at a high-level –(Reduced from last time I offered this course) Email me reviews by noon (12:00 pm) the day of the class Skip reviews the classes you are presenting.

43. Paper review guidelines One page Detailed review: –Brief (2-3 sentences) summary –Main contribution –Strengths? Weaknesses? –How convincing are the experiments? Suggestions to improve them? –Extensions? Applications? –Additional comments, unclear points High-level review: –Problem being addressed –High-level intuition/idea of approach Relationships observed between the papers we are reading Will pick on students in class during discussions Write in your own words Write well, proof read

44. Paper presentation guidelines Papers Experiments

45. Papers Read selected papers in topic area and look at background papers as necessary Well-organized talk, 45 minutes What to cover? –Topic overview, motivation –For selected papers: Problem overview, motivation Algorithm explanation, technical details Experimental set up, results Strengths, weaknesses, extensions –Any commonalities, important differences between techniques covered in the papers. See class webpage for more details.

46. Experiments Implement/download code for a main idea in the paper and evaluate it: –Experiment with different types of training/testing data sets –Evaluate sensitivity to important parameter settings –Show an example to analyze a strength/weakness of the approach –Show qualitative and quantitative results

47. Tips Look up papers and authors. Their webpage may have data, code, slides, videos, etc. –Make sure talk flows well and makes sense as a whole. –Cite ALL sources. Don’t forget the high-level picture. Give a very clear and well-organized and thought out talk. Will interrupt if something is not clear

48. Tips Make sure you are saying everything we need to know to understand what you are saying. Make sure you know what you are talking about. Think about your audience. Make your talks visual (images, video, not lots of text).

49. Projects Possibilities: –Extension of a technique studied in class –Analysis and empirical evaluation of an existing technique –Comparison between two approaches –Design and evaluate a novel approach –Be creative! Can work with a partner Talk to me if you need help with ideas

50. Project timeline Project proposals (1 page) [10%] –March 6 th Mid-semester presentations (10 minutes) [20%] –March 27 th and April 1 st Final presentations (20 minutes) [35%] –April 24 th to May 6 th Project reports (4 pages) [35%] –May 12 th –Could serve as a first draft of a conference submission!

51. Implementation Use any language / platform you like No support for code / implementation issues will be provided

52. Miscellaneous Best presentation, best project and best discussion prizes! –We will vote –Dinner Feedback welcome and useful

53. Coming up Read the class webpage –Schedule is up –Tour of schedule Select 6 dates (topics) you would like to present –Email me by Wednesday (tomorrow) –Webpage shows how many people have already signed up for a topic –Select those that have fewer selections Overview of my research on Thursday –How many of you were at the ECE grad seminar in November?

54. Questions? See you Thursday!