1. Data-Driven 3D Voxel Patterns for Object Category Recognition Andrew Sharp

2. Outline 3D Warehouse CAD Voxelization Provided Data Affinity Propagation Results Author’s code

3. 3D Voxel Exemplars from Data As long the incoming pipeline provides the 2D segmentation mask and the 3D voxel model it can be used to build exemplars. Depth sensors or 3D scanners 3D Warehouse: large database of SketchUp models (cars, buildings, furniture, etc.) This system could be trained to recognize other objects Opens up possibilities for simulated training data https://3dwarehouse.sketchup.com/

4. Other examples from the Trimble 3D Warehouse https://3dwarehouse.sketchup.com/

5. Voxelization Depth images of a CAD model rendered at 8 azimuths and 6 elevations, producing 48 depth images Fuse depth images to obtain a 3D point cloud surface of the object Voxelize the 3D space using the surface point cloud Obtain mean voxel model by averaging the seven CAD models http://cvgl.stanford.edu/papers/xiang_cvpr15_3dvp_tr.pdf

6. Provided data Annotations built from KITTI f or each car in the training set of the KITTI detection benchmark: 2D segmentation mask 3D voxel model Alignment to its KITTI annotation http://cvgl.stanford.edu/projects/3DVP/

7. Affinity Propagation 'Clustering by Passing Messages Between Data Points‘ by B. J. Frey and D. Dueck Iterative refining to find high-quality set of exemplars and corresponding clusters Similarity values are needed B. J. Frey and D. Dueck. Clustering by passing messages between data points. science, 315(5814):972– 976, 2007

8. Affinity Propagation 'Clustering by Passing Messages Between Data Points‘ by B. J. Frey and D. Dueck Iterative refining to find high-quality set of exemplars and corresponding clusters Similarity values are needed B. J. Frey and D. Dueck. Clustering by passing messages between data points. science, 315(5814):972–976, 2007

9. Affinity Propagation Create similarity matrix by summing the number of points with the same value non-zero in the voxel model. Zero points do represent some similarity but number (approximately 98%) drown out the clustering information. http://cvgl.stanford.edu/papers/xiang_cvpr15_ 3dvp.pdf

10. Affinity Propagation Results AP Clustering with Varying Perference P Value10.0050.0010.0005 Clusters56322144237228 Max Points1140253260 Largest Cluster ID119627834912 Exemplar IDNumber in ClusterCluster Point IDs 783253214299169190232328439522565 9221998592189220250332382546669 10915631109141187215257367459559715 26151426341221613785076178171026 47014733608614519227760583510351262 B. J. Frey and D. Dueck. Clustering by passing messages between data points. science, 315(5814):972–976, 2007

17. Results Comparison Author’s results, 227 3DVPs: 91 3DVPs are fully visible 121 3DVPs are partially occluded 15 3DVPs are truncated My results 238 3DVPs but with a similar breakdown.

18. Updated Authors Code is available https://github.com/yuxng/3DVP_RCNN RCNN detection network Uses Caffe Uses Cython (http://cython.org/) Github error page (https://github.com/BVLC/caffe/issues/782)

19. Outline 3D Warehouse CAD Voxelization Provided Data Affinity Propagation Results Author’s code

20. References http://cvgl.stanford.edu/projects/3DVP/ http://cvgl.stanford.edu/papers/xiang_cvpr15_3dvp.pdf http://cvgl.stanford.edu/papers/xiang_cvpr15_3dvp_tr.pdf https://yuxng.github.io/Xiang_CVPR15_06082015.pdf http://www.psi.toronto.edu/affinitypropagation/FreyDueckScience07.pdf http://www.psi.toronto.edu/index.php?q=affinity%20propagation https://3dwarehouse.sketchup.com/