1. Digital Face Cloning David Bennett Christophe HerySteve Sullivan George Borshukov J.P. Lewis Lance Williams Paul Debevec Fred Pighin Li Zhang

2. David Bennett Christophe HerySteve Sullivan George Borshukov J.P. Lewis Lance Williams Paul Debevec Fred Pighin Li Zhang BostonSeattle Los Angeles San Francisco

3. Definition Process of capturing an actor’s performance and/or likeness in a digital face model

4. E.g. photogrammetry (input)

5. E.g. photogrammetry (output)

6. Motivation Replicate a live actor (e.g. celebrity) – Stunts – Extreme makeup – Younger/older/slimmer clone Creating realistic faces from first principles is difficult (digitizing is easier than modeling)

7. Purpose of the course 1. Review the field 2. Suggest directions

8. Digital Clones: History

9. CG Humans in the movies First 3D actor representation in the movies: Peter Fonda in FutureWorld (1976!) Not intended to be photorealistic

10. 1990s

11. Terminator 2 (brief, texture projection) Jurrasic Park (legs only, does not count) Titanic (distant) Startup attempt: Virtual Celebrity/Global Icons, purchased virtual rights to Sammy Davis Jr., others

12. 1998: Hollywood is ready "I don't see them as competing with real actors,” Richard Masur, head of Screen Actors Guild Projected uses: de-aging, celebrity vacuum ads. A television series with a virtual cast was considered

13. Recent and current

14. Space Cowboys (2000), Enemy at the Gates (2001): brief and distant, but successful Disney Facial Cloning test (afternoon session) Matrix sequels (2003): “superpunch” full-screen for 13 seconds (afternoon session) SpiderMan 2 (reflectance capture) Lemony Snicket: CG baby in several shots (afternoon session) Discovery Channel Churchill/Hitler clones (MPC)

15. Observation There has not been a “breakthrough” introduction of digital clones Rather, clones were initially used in the distance, and briefly; they are getting closer, and withstanding longer study. Current state of the art: full screen for several seconds (see afternoon Perception session)

16. Syllabus – morning (theory) 8:30 Introduction and Overview Fred Pighin 9:00 Face Scanning and Dense Motion Capture Technologies Li Zhang 10:00 Break or Mathematical Background (schedule change) 10:15 Reflectance Modeling and Capture Paul Debevec and J.P. Lewis 11:15 Facial Parameterization and Cross-Mapping J.P. Lewis and Fred Pighin 12:15Lunch

17. Syllabus – afternoon (practice) 1:30Case study: Face Cloning at ILM Steve Sullivan and Christophe Hery 2:15Perception Experiment (schedule change) 2:30Case Study: Disney Facial Cloning Lance Williams 3:15 Case study: Leaping the Uncanny Valley with Data (Face Cloning in the Matrix sequels) George Borshukov 4:00 Break 4:15 Case study: Polar Express David Bennet 5:00 Perception of Facial Realism J.P. Lewis 5:15 Panel on the Future of Digital Face Cloning All

18. Face Cloning at Siggraph Courses – 10. Realistic Materials in Computer Graphics – 28. From Mocap to Movie: The Making of "The Polar Express“ – 29. High-Dynamic-Range Imaging and Image-Based Lighting Papers (Skin & Faces session on Monday) – Automatic Determination of Facial Muscle Activations From Sparse Motion Capture Marker Data – Face Transfer With Multilinear Models – Postproduction Relighting and Reflectance Transformation with Time-Multiplexed Illumination Sketches – Image-Based Rendering From a Sparse Set of Images – Implementation of Modeling Hair From Multiple Views – Model Flowing: Capturing and Tracking of Deformable Geometry – Performance Geometry Capture for Spatially Varying Relighting – Joint Motion and Reflectance Capture for Relightable 3D Video