A Practical System for Modelling Body Shapes from Single View Measurements Yu Chen 1, Duncan Robertson 2, Roberto Cipolla 1 Department of Engineering, University of Cambridge 1 Me_tail Inc. 2

Motivations and Tasks Our Goal: –A reliable and practical 3D body modelling system. Requirements: –Good accuracy; –Quick; –Easy to manipulate for non-expert users (a bit of interactions)

Different Means of Input Tape measurements –E.g. [Magnenat-Thalmann et al., 03, 07], etc. –Sparse representation; –Relatively invariant to pose changes; –Anthropometrically meaningful; –Many users don’t know their body measurements like chest, waist, and hips; –Obtaining accurate tape measurements is not very easy.

Different Means of Input Full Silhouettes –E.g. [Balan et al. 08], [Guan et al. 09], etc. –Dense representation; –Informative; better local constraints; –Hard to decouple pose variations from body shape variations; –Hard to extract silhouettes from images with arbitrary background;

A Solution Silhouettes (dense) Tape Measurements (sparse) Our Solution: Image Measurements

The Interface How it work? 1.Upload a photo of the user standing against a doorway; 2.Mark up the 4 corners of the doorway; 3.Rectify the photo automatically; 4.Measure the several image measurements on the rectified image. 5.Provide body dimensions: height + weight; 6.Obtain the 3D body shape.

The Interface How it work? 1.Upload a photo of the user standing against a doorway; 2.Mark up the 4 corners of the doorway; 3.Rectify the photo automatically; 4.Measure the several image measurements on the rectified image; 5.Provide body dimensions: height + weight; 6.Obtain the 3D body shape.

The Interface How it work? 1.Upload a photo of the user standing against a doorway; 2.Mark up the 4 corners of the doorway; 3.Rectify the photo automatically; 4.Measure the several image measurements on the rectified image; 5.Provide body dimensions height + weight; 6.Obtain the 3D body shape.

The Interface How it work? 1.Upload a photo of the user standing against a doorway; 2.Mark up the 4 corners of the doorway; 3.Rectify the photo automatically; 4.Measure the several image measurements on the rectified image; 5.Provide body dimensions: height + weight; 6.Obtain the 3D body shape.

The Interface How it work? 1.Upload a photo of the user standing against a doorway; 2.Mark up the 4 corners of the doorway; 3.Rectify the photo automatically; 4.Measure the several image measurements on the rectified image; 5.Provide body dimensions height + weight; 6.Obtain the 3D body shape.

The Interface How it work? 1.Upload a photo of the user standing against a doorway; 2.Mark up the 4 corners of the doorway; 3.Rectify the photo automatically; 4.Measure the several image measurements on the rectified image; 5.Provide body dimensions height + weight; 6.Obtain the 3D body shape.

Image Rectification Automatic photo rectification. Use vanishing point geometry. Compute the homography from the quadrilateral to the rectangle.

Homography H is dependent on the focal length f. Regular case: –Both vanishing points are at finite positions. –[Cipolla et al. 99]: Near-degenerated case: –One of the vanishing points is at infinity; –Result in an ambiguity in the aspect ratio α. Image Rectification Aspect ratio distortion need to be corrected! V 2 at infinityV 1 at infinity or

Selection of Image Measurements Criteria: –Well defined; unambiguous to users; –Good correlation with the corresponding 3D measurements; –Good representative power. Selected measurement set: –Height; –Crotch height; –Under bust; –Waist; –Hips;

Probabilistic Body Shape Estimation 3D training data: –Female CAESAR Data; Registration: –Victoria 4 morphable model; –10 different morphs are defined. Regressor: Gaussian Process (GP) –Input: body dimensions (height & weight) + normalised measurements; –Output: morph weights.

Prediction and Aspect Ratio Correction (ARC) Given a learned GP regressor G. Input: –Actual body dimensions z V –Image measurements z I Horizontal image measurements z I,h ; Vertical image measurements z I,v ; –Complete set of testing measurements: z = {z V, α z I,h, z I,v } Output: –Morph weights y; aspect ratio factor α.

Prediction and Aspect Ratio Correction (ARC) The shape prior gives clues of the proper aspect ratio. GP Joint posterior. MAP estimate of the y and α Optimisation: –y = µ y ; –α : fixed point equations.

Prediction and Aspect Ratio Correction (ARC) ARC, How effective is it? A simulation of distortion and correction –Stretch a frontal view image into different aspect ratio α from 0.9 to 1.1. ARC can neutralise more than 60% of distortion.

Experiments and Testing Data Render >1000 synthetic doorway images using CAESAR laser scans –Frontal-view set; –Un-rectified set; Cross validation. Input combinations –H + W: height and weight only; –H + W + T: height, weight, and tape measurements; –H + W + I: height, weight, and image measurements.

Average error over 15 body measurements defined in CAESAR dataset. Image measurements are good substitutions when tape measurements are not available. Test on the Frontal-view Image Set

Test on the un-rectified image set The efficacy of aspect ratio correction

Test on Real Photos User study by inviting volunteers –Take photos at home; uncontrolled environment; –Tape measurements are collected for evaluations. Performance –Average time for annotation (doorway + 5 image measurements): 1.5 to 3 min; –Computation time: <1s for 2.4GHz CPU; –Accuracy: Body partChestWaistHipsInner leg length Error(cm)1.52 ± ± ± ± 0.90

Qualitative Results of Real Users

Take-home Messages Image measurements can define body shapes well; Image rectification may lead to an aspect ratio distortion; The 3D shape prior gives important clues to correct the aspect ratio distortion; A novel working system for quick 3D body shape modelling and garment fitting.

The End Questions?

Our First Task A Case Study: SVR of human body shapes –Input: A 2D doorway photo; –Output: 3D body shape.

Image Rectification Using vanishing point geometry. Rotation and Translation Homography: