1. Graduate Programs in Computer Science http://www.cs.utsa.edu A Soft Hand Model for Physically-based Manipulation of Virtual Objects Jan Jacobs Group Research Virtual Technologies Volkswagen AG Bernd Froehlich Virtual Reality Systems Group Bauhaus-Universitat Weimar In Virtual Reality Conference (VR), 19 - 23 March, 2011 Singapore, IEEE, 2011.

2. Graduate Programs in Computer Science http://www.cs.utsa.edu Outline Background Introduction Related Work System Design Results and Discussion Conclusions Future Work

3. Graduate Programs in Computer Science http://www.cs.utsa.edu Background What is VR?  Virtual Reality Applications of VR  Video Games  Training Systems  Simulator  3D/4D Movies, Games  Auto Design

4. Graduate Programs in Computer Science http://www.cs.utsa.edu Background The integration of physical behavior has significantly increased the quality of games and virtual environments overall The interaction with simulated objects also needs to occur on a physical basis. The representation of a user in the virtual world needs to be physically modeled to achieve a realistic interaction between user and virtual objects. The problem: the modeling of the finely articulated human hand to enable finger-based interaction.

5. Graduate Programs in Computer Science http://www.cs.utsa.edu Introduction A direct and robust finger-based manipulation relies on three major issues  Stable grasping of objects  Robust manipulation  Controlled releasing of objects. In general, there are two common ways to achieve these goals  Grasping through heuristics  Collision based physical simulations.

6. Graduate Programs in Computer Science http://www.cs.utsa.edu Introduction In 2005, Borst et al. [1] relies on a hand model constructed from rigid bodies. Problems:  It could not correctly consider friction between fingers and virtual  It required careful tuning of parameters for a reasonably stable interaction.

7. Graduate Programs in Computer Science http://www.cs.utsa.edu Introduction Real World Grasping Increasing contact area with increasing contact force. Left: loose touch. Right: strong pressure between finger and a pane of glass.

8. Graduate Programs in Computer Science http://www.cs.utsa.edu Introduction Physics problem  f = μN  μ is the friction coefficient  N is the pressure  The friction doesn’t related to the contact area VR Simulation  No haptic system  Mapping: the higher the pressure, the bigger the contact area

9. Graduate Programs in Computer Science http://www.cs.utsa.edu Introduction A soft body model for each finger phalanx was introduced to enable pressure-based deformation of the soft finger contact areas. The system allows for very precise and robust finger-based grasping, manipulation and releasing of virtual objects in real-time.

10. Graduate Programs in Computer Science http://www.cs.utsa.edu Related Work This idea is originally from Duriez et al. in 2008 [2]. They addressed this problem by directly calculating friction at skin level.  Complexity The FastLSM algorithm by Rivers et al. [3].

11. Graduate Programs in Computer Science http://www.cs.utsa.edu System Design Software Architecture Tracking Hand Model

12. Graduate Programs in Computer Science http://www.cs.utsa.edu System Design Software Architecture  Scenegraph System: OpenSG  Physics Engine: bullet  User Input

13. Graduate Programs in Computer Science http://www.cs.utsa.edu System Design Tracking  An optical finger tracking system  Seven evenly spread cameras for a 3m 3 volume

14. Graduate Programs in Computer Science http://www.cs.utsa.edu System Design Hand Model  A rigid body (grey)  A soft body (green)

15. Graduate Programs in Computer Science http://www.cs.utsa.edu Results and Discussion Unconstrained interaction with a horse model. The fingerpads adapt to the geometries’ shape, enabling stable and robust interaction.

16. Graduate Programs in Computer Science http://www.cs.utsa.edu Results and Discussion Two handed interaction with non-constrained objects. Collision response between torus and stick is enabled through physics simulation.

17. Graduate Programs in Computer Science http://www.cs.utsa.edu Results and Discussion Interaction within an immersive display system. A user interacts with a constrained steering wheel using both hands, thus reproducing a real-world interaction.

18. Graduate Programs in Computer Science http://www.cs.utsa.edu Conclusions A new hand model is based on soft bodies coupled to a rigid body hand skeleton  Precise and robust finger-based grasping, manipulation and releasing  Dynamic adaptation of the stiffness values  The implicit friction model The pressure-based increasing and decreasing of the contact area of the simulated finger phalanxes

19. Graduate Programs in Computer Science http://www.cs.utsa.edu Future Work A soft body for palm Using different deformation algorithm depending on the situation A skinned hand representation

20. Graduate Programs in Computer Science http://www.cs.utsa.edu References [1] C. W. Borst and A. P. Indugula. Realistic virtual grasping. In Virtual Reality Conference (VR), 2005 IEEE, pages 91–98, 320, 2005. [2] C. Duriez, H. Courtecuisse, J. P. de la Plata Alcalde, and P.-J. Bensoussan. Contact skinning. In Eurographics 2008 (short paper), pages 313–320, New York, NY, USA, 2008. [3] A. R. Rivers and D. L. James. FastLSM: Fast lattice shape matchingfor robust real-time deformation. ACM Transactions on Graphics (Proc. SIGGRAPH 2007), 26(3):82, July 2007.

21. Graduate Programs in Computer Science http://www.cs.utsa.edu Thank you! Questions?