1 07/11/07 Using an Event-Based Approach to Improve the Multimodal Rendering of 6DOF Virtual Contact Jean Sreng, Florian Bergez, Jérémie Le Garrec, Anatole Lécuyer, Claude Andriot

207/11/07 Outline Context Contact states and events Computation Multimodal feedback –Visual –Audio –6DOF Haptic Conclusion OUTLINE

307/11/07 Outline Context Contact states and events Computation Multimodal feedback –Visual –Audio –6DOF Haptic Conclusion OUTLINE

407/11/07 Context Virtual contact between objects –Virtual prototyping –Assembly/Maintenance simulations –Video games –…–…

507/11/07 Context Multimodal rendering of contact –Visual –Audio –Haptic Some limitations –Computational resources –Sensor resolution 500 Hz1 kHz48 kHz60Hz

607/11/07 Context To overcome these limitations : –Sound synthesis algorithms (modal synthesis) [van den doel et. al.] –Event-based haptic [kuchenbecker et. al.] –…–… However : –Often tied to a specific physical engine –Demonstrated in a simple context

707/11/07 We propose… An event-based approach : –Integrate various techniques Visual Audio 6DOF Haptic –Based only on position information Not to be tied to a specific physical engine Can be easily adapted to an existing simulation

807/11/07 Outline Context Contact states and events Computation Multimodal feedback –Visual –Audio –6DOF Haptic Conclusion OUTLINE

907/11/07 Contact information (states) Continuous, “time independent” states : –Free motion (no contact) –Friction (contact) Free motionFriction

1007/11/07 Contact information (events) Discrete events describing the “evolution” : –Impact –Detachment Free motionFriction ImpactDetachment

1107/11/07 Contact information (states and events) The classic continuous information : –Is essential to graphic and closed-loop haptic rendering –Cannot convey efficiently the transient nature of an impact The different states and events information can be used to improve the rendering of : –Transient phenomena (impact) –Continuous phenomena (friction)

1207/11/07 Outline Context Contact states and events Computation Multimodal feedback –Visual –Audio –6DOF Haptic Conclusion OUTLINE

1307/11/07 Computation of contact states and events The contact condition : The events are defined by : –Impact : –Detachment : Local linear velocity Normal Friction ImpactDetachment

1407/11/07 Velocity computation To estimate the velocity : Some issues : –Time stepping –Approximation errors

1507/11/07 Computation of contact states and events On the timestep –Impact event : –Friction state : –Detachment event :

1607/11/07 Computation of contact states and events In a nutshell… Friction ImpactDetachment

1707/11/07 Outline Context Contact states and events Computation Multimodal feedback –Visual –Audio –6DOF Haptic Conclusion OUTLINE

1807/11/07 Multimodal feedback The generated states and events can be used with specific rendering techniques Implementation on an existing 6DOF haptic simulation

1907/11/07 Visual feedback Particle effect : –Impact event –Friction state –Detachment event Pencil effect on friction state : –Color : Force –Thickness : Velocity

2007/11/07 Audio feedback Modal synthesis of impact event and friction state :

2107/11/07 Haptic feedback 6DOF event-based haptic feedback of impact event –Traditional closed loop feedback –High frequency force pattern Elementary wrench associated to an impact point

2207/11/07 Haptic feedback

2307/11/07 Conclusion We proposed : –An event-based approach to improve the rendering of contact : Impact Friction Detachment –A computation method based only on object’s positions –A multimodal implementation on a complex virtual prototyping simulation Visual Audio 6DOF Haptic

2407/11/07 Further work A formal user evaluation –Compare with traditional feedback –Complex industrial context Test some other feedback : –Detachment event –6DOF haptic texture

2507/11/07 ? Questions Thank you for listening Questions ?