23D Computer Vision: CSc 83020 Instructor: Ioannis Stamos istamos (at) hunter.cuny.eduOffice Hours: Tuesdays 4-6 (at Hunter) or by appoitnmentOffice: 1090G Hunter North (69th street bw. Park and Lex.)Computer Vision Lab: 1090E Hunter NorthCourse web page:
3GoalsTo familiarize you with basic the techniques and jargon in the fieldTo enable you to solve computer vision problemsTo let you experience (and appreciate!) the difficulties of real-world computer visionTo get you excited!
4Class Policy You have to Late policy Teaming Turn in all assignments (60% of grade)Complete a final project (30% of grade)Actively participate in class (10% of grade)Late policySix late days (but not for final project)TeamingFor final project you can work in groups of 2
5About me 11th year at Hunter and the Graduate Center Graduated from Columbia in ’01CS Ph.D.Research Areas:Computer Vision3D ModelingComputer Graphics
6BooksComputer Vision: Algorithms and Applications, Richard Szeliski, 2010 (available online for free)Robot VisionB. K. P. Horn, The MIT Press (great classic book)Introductory Techniques for 3-D Computer VisionEmanuele Trucco and Alessandro Verri, Prentice Hall, 1998 (algorithmic perspective)Computer Vision A Modern ApproachDavid A. Forsyth, Jean Ponce, Prentice Hall 2003An Invitation to 3-D VisionYi Ma, Stefano Soatto, Jana Kosecka, S. Shankar SastrySpringer 2004.Three-Dimensional Computer Vision: A Geometric Viewpoint Olivier FaugerasThe MIT Press, 1996.
7Journals/Web International Journal of Computer Vision. Computer Vision and Image Understanding.IEEE Trans. on Pattern Analysis and Machine Intelligence.SIGGRAPH (mostly Graphics)(CMU’s Robotic Institute)(The Vision Home Page)(CV Online)(Annotated CV Bibliography)
8Class History Based on class taught at Columbia University by Prof. Shree Nayar.New material reflects modern approach.Taught similar class at HunterTaught “3D Photography” class at the Graduate Center of CUNY.My active research areaFunded by the National Science Foundation
9Class Schedule Check class website Final project proposals Due Nov. 7Design your own or check list of possible projects on class websiteFinal project presentations and reportMay 16 (last class)
10What is Computer Vision? SensorsImages or VideoIlluminationVision SystemPhysical 3D WorldScene DescriptionMeasuring Visual Information
11Computer Graphics Output Image Model Synthetic Camera (slides courtesy of Michael Cohen)
12Computer Vision Output Model Real Scene Real Cameras (slides courtesy of Michael Cohen)
13Combined Output Image Model Real Scene Synthetic Camera Real Cameras (slides courtesy of Michael Cohen)
14Cont. Vision is automating visual processes (Ball & Brown). Vision is an information processing task (Marr).Vision is inverting image formation (Horn).Vision is inverse graphics.Vision looks easy, but is difficult.Vision is difficult, but it is fun (Kanade).Vision is useful.
71Scanning the David Marc Levoy, Stanford 2:30 through closeup of eye0:40So here we are scanning our “piece de resistance”the gantry is extended to nearly its full heightIf you’ve looked at our web siteyou know the little story about the gantry being too short initiallybecause we relied on art history books for the height of the Davidand they're all wrong – by 3 feet !!so at the last minute we had to add a piece to the gantry…hereand load another few hundred pounds of ballast to the base…here – to keep the gantry from tipping over on the statueBut as you can see from the photo on the right,we did finally reach the top of the statueheight of gantry: metersweight of gantry: kilograms4:00 through architectural reps0:45
72Statistics about the scan 0:20So here’s a rendering of our computer model of the Davidwe rolled the gantry, which now weighed about a ton, 480 times2 billion polygonsyou can read the restIt took 30 days to scan the statuebut we were only allowed to scan at night, when the museum was closedso it was basically 30 all-nighters in a row480 individually aimed scans2 billion polygons7,000 color images32 gigabytes30 nights of scanning22 peopleIoannis Stamos – CSc Spring 20070:30
73Head of Michelangelo’s David 0:20We haven’t assembled the David at full-resolution yetHere’s a 1 mm model of the headwatertightwith colorThe photograph at the left was taken with uncalibrated lightingso the two images don’t match exactlybut hopefully you agree that we’ve basically captured the statue’s appearancephotograph1.0 mm computer model
74David’s left eye 0.25 mm model photograph 0:30Here’s a closeup of David’s left eyeat the maximum resolution of our model – ¼ mmwe can see some interesting things…These bumpsare holes from Michelangelo’s drillso that’s real geometryare artifacts from space carving, the hole-filling part of our range image merging algorithmsmoothing out those artifacts, while preserving the observed surfaces, is future workFinally, these bumpsare noise from laser subsurface scatteringLet’s zoom in some moreand look at the triangle meshnoise from laser scatterholes from Michelangelo’s drillartifacts from space carving0.25 mm modelphotographIoannis Stamos – CSc Spring 20070:45