Quan Yu State Key Lab of CAD&CG Zhejiang University Personalized Bare Hand Modeling Based on Stereo Vision and Geometry Deformation Quan Yu State Key Lab of CAD&CG Zhejiang University

Outline Introduction System Overview Algorithm Result & Feature Work Input Features Alignment & Deformation Result & Feature Work Conclusion

Introduction Personalized Hand Modeling(NSFC, No.60970078) Challenge No markers nor gloves Low-end devices (web cameras) Challenge Lack of strong features Lack of solution

Introduction(Cont.) Idea Solution Corse features come from stereo vision Fine features come from a template Solution Deform a template under constrains of vision data step by step

System Overview Fig. 1 System overview. (a)extract convexity defects of contours and build a local coordinate; (b)align the template with defects and refine alignment with ICP algorithm; (c)laplacian deformation under constrains of defects (point level); (d)generate contour points with a single image; (e)laplacian deformation under constrains of contours (line level); (f)extract surface features and construct a point cloud; (g)laplacian deformation under constrains of surface points (surface level).

Input Two pairs of stereo images A generic template Camera parameters front and back faces of a hand A generic template Denote contours and defects Camera parameters Web cameras No markers

Features: defect points Convexity defects of contours Stable Strong Used to determine: Size Position Alignment with the template

Features: contour points Generate contour points from a single image Contours of left and right image are different. Assume the depths of contour points are constant A non-linear interpolation between defects Disparities are unknown.

Features: contour points(Cont.) Approximate contours as Correspondences: arc length matching mean of contours arc length matching

Features: surface points Image enhancement Contrast-Limited Adaptive Histogram Equalization Hard to extract robust features of hand skin. SIFT SURF GLOH ? DAISY ? Efficient Large-Scale Stereo Matching(ACCV 2010) Sobel responses on a regular grid

Features: surface points(Cont.) Find correspondences Estimate normals (MLS) Project 3D points onto the template Split the template at the projection point

Features: surface points(Cont.) Iterative deformation to eliminate outliers Reject Threshold correspondences deformation iter=1 iter=2 iter=3 Details or Outliers?

Alignment Extract defects Local coordinate Refine: ICP Efficient Variants of the ICP Algorithm[S. R. 2001]

Laplacian Deformation Laplacian Mesh Processing Ogla Sorkine, 2005 Laplacian Mesh Optimization Andrew Nealen, 2006

Result & Feature Work Result Feature Work Demo Resampling Geometry Optimization Texture

Conclusion A novel approach to construct personalized hand model with low-end equipments; Generate 3D contour points from a single image; Eliminate outliers with an iterative deformation.

Thank you! Question & Suggestion ?