1. Quan Yu State Key Lab of CAD&CG Zhejiang University
Personalized Bare Hand Modeling Based on Stereo Vision and Geometry Deformation
Quan Yu
State Key Lab of CAD&CG
Zhejiang University

2. Outline Introduction System Overview Algorithm Result & Feature Work
Input
Features
Alignment & Deformation
Result & Feature Work
Conclusion

3. Introduction Personalized Hand Modeling(NSFC, No.60970078) Challenge
No markers nor gloves
Low-end devices (web cameras)
Challenge
Lack of strong features
Lack of solution

4. Introduction(Cont.) Idea Solution
Corse features come from stereo vision
Fine features come from a template
Solution
Deform a template under constrains of vision data step by step

5. System Overview
Fig. 1 System overview. (a)extract convexity defects of contours and build a local coordinate; (b)align the template with defects and refine alignment with ICP algorithm; (c)laplacian deformation under constrains of defects (point level); (d)generate contour points with a single image; (e)laplacian deformation under constrains of contours (line level); (f)extract surface features and construct a point cloud; (g)laplacian deformation under constrains of surface points (surface level).

6. Input Two pairs of stereo images A generic template Camera parameters
front and back faces of a hand
A generic template
Denote contours and defects
Camera parameters
Web cameras
No markers

7. Features: defect points
Convexity defects of contours
Stable
Strong
Used to determine:
Size
Position
Alignment with the template

8. Features: contour points
Generate contour points from a single image
Contours of left and right image are different.
Assume the depths of contour points are constant
A non-linear interpolation between defects
Disparities are unknown.

9. Features: contour points(Cont.)
Approximate contours as
Correspondences: arc length matching
mean of contours
arc length matching

10. Features: surface points
Image enhancement
Contrast-Limited Adaptive Histogram Equalization
Hard to extract robust features of hand skin.
SIFT
SURF
GLOH ?
DAISY ?
Efficient Large-Scale Stereo
Matching(ACCV 2010)
Sobel responses on a regular grid

11. Features: surface points(Cont.)
Find correspondences
Estimate normals (MLS)
Project 3D points onto the template
Split the template at the projection point

12. Features: surface points(Cont.)
Iterative deformation to eliminate outliers
Reject Threshold
correspondences
deformation
iter=1
iter=2
iter=3
Details or Outliers?

13. Alignment Extract defects Local coordinate Refine: ICP
Efficient Variants of the ICP Algorithm[S. R. 2001]

14. Laplacian Deformation
Laplacian Mesh Processing
Ogla Sorkine, 2005
Laplacian Mesh Optimization
Andrew Nealen, 2006

15. Result & Feature Work Result Feature Work Demo Resampling
Geometry Optimization
Texture

16. Conclusion
A novel approach to construct personalized hand model with low-end equipments;
Generate 3D contour points from a single image;
Eliminate outliers with an iterative deformation.

17. Thank you! Question & Suggestion ?