1. Interactive Mesh Sculpting Using a Haptic Device

2. Goal A method to continuously sculpt or deform a planar mesh using haptic interaction is implemented. Moreover, this mesh has an image of the final model mapped on.

3. Approach Image mapped on a planar mesh Phantom Omni Basic Haptic Device NURBS surface is used Different tools to improve sculpting Image mapped on a planar mesh Phantom Omni Basic Haptic Device NURBS surface is used Different tools to improve sculpting

4. Force Feedback Foundation It allows the user to interact with it by applying and receiving forces. Interacts with graphics applications Can make objects feel sticky, soft, rough, viscous It allows the user to interact with it by applying and receiving forces. Interacts with graphics applications Can make objects feel sticky, soft, rough, viscous

5. Previous Work

6. Haptic Device Avoiding interpenetration between virtual objects Allowing the user to paint in virtual environments Teleoperation of objects Touching textures Avoiding interpenetration between virtual objects Allowing the user to paint in virtual environments Teleoperation of objects Touching textures

7. Mesh Sculpting Free Form Deformation  Control points define the vertices of a parallelepiped  Using control points can be cumbersome Subdivision solids  Each subdivided solid has damping, mass and stiffness Virtual clay Free Form Deformation  Control points define the vertices of a parallelepiped  Using control points can be cumbersome Subdivision solids  Each subdivided solid has damping, mass and stiffness Virtual clay

8. Characteristics of the Project

9. Haptic Device Movement in 3 dimensions Mouse is constrained to 2 dimensions Limited range of motion Force rendering Movement in 3 dimensions Mouse is constrained to 2 dimensions Limited range of motion Force rendering

10. NURBS Provide advantages over other surfaces Using OpenGL implementation Automatic texture mapping Knot array affects continuity of the surface Provide advantages over other surfaces Using OpenGL implementation Automatic texture mapping Knot array affects continuity of the surface

11. Particle System Defined as a set of particles that evolves over time Control points in NURBS are the particles Particles have attributes like position, velocity and force Defined as a set of particles that evolves over time Control points in NURBS are the particles Particles have attributes like position, velocity and force

12. Runge-Kutta4 Method used to solve differencial equations

13. Implementation

14. HLAPI and OpenGL High level haptic rendering Provides rendering/collision detection and useful data structures hlBeginFrame and hlEndFrame High level haptic rendering Provides rendering/collision detection and useful data structures hlBeginFrame and hlEndFrame

15. Data Structures Vertex class  Has attributes like position and velocity ParticleSystem  Contains list of control points NURBS  Draws the mesh Vertex class  Has attributes like position and velocity ParticleSystem  Contains list of control points NURBS  Draws the mesh

16. Features/Tools Sculpting Forces are rendered when we touch the surface. Sculpting includes both pushing and pulling the surface. We are using the stylus button to switch between push and pull modes. Sculpting Forces are rendered when we touch the surface. Sculpting includes both pushing and pulling the surface. We are using the stylus button to switch between push and pull modes.

17. Brush Induces force on more than one control point at a time This is the point of contact The green points are also affected

18. 3D Paint Painting in the 3D model is reflected on the 2D texture

19. Show/Hide Particles Showing particles Hiding particles Closer look at the image

20. Rotations It allows us to look the image from different angles

21. 3D Cursor It simulates the shape of a pen

22. Force Feedback This is the default mode, but can be disabled. If disabled, no forces are rendered, but the mesh can still be sculpted. Results of this comparison are shown in the Results section. This is the default mode, but can be disabled. If disabled, no forces are rendered, but the mesh can still be sculpted. Results of this comparison are shown in the Results section.

23. Basic Algorithm 1. Initialize the application 2. Create the mesh and texture map it with the image 3. Sculpt the mesh using the haptic device: a. Rotate the mesh b. Select a brush or single point sculpting c. Apply forces to the mesh 4. The new position for the control point is calculated and the mesh is drawn again. 5. Repeat step 3 until sculpting is done. 1. Initialize the application 2. Create the mesh and texture map it with the image 3. Sculpt the mesh using the haptic device: a. Rotate the mesh b. Select a brush or single point sculpting c. Apply forces to the mesh 4. The new position for the control point is calculated and the mesh is drawn again. 5. Repeat step 3 until sculpting is done.

24. Major Issues Sense of depth is difficult to implement NURBS sampling parameters needs to be adjusted Haptic device kicked occasionally while rotating the surface Sense of depth is difficult to implement NURBS sampling parameters needs to be adjusted Haptic device kicked occasionally while rotating the surface

25. Results

26. Comparing the Haptic Device and the Mouse MouseHaptic device No force feedback Points moved one at a time, but can move continuously if implemented User touches the control points Picking control points may be difficult Force feedback exists Points are moved continuously User touches the mesh since the haptic device provides this configuration No need to pick control points

27. Mapping Haptic Device to Graphics Simulation

28. NURBS in OpenGL An efficient way to draw NURBS Provide different parameters to control the surface The mesh is 12x12 control points Hardware acceleration in the graphics card should be disabled An efficient way to draw NURBS Provide different parameters to control the surface The mesh is 12x12 control points Hardware acceleration in the graphics card should be disabled

29. Interactive Sculpting Initial imageFinal image

30. Lips Deformation Initial image

31. Sculpting the lower lip

32. Side view of the image

33. Common error in the sculpting process

34. Side view of the final mesh

35. Front view of the mesh

36. Comparing Force Feedback and No Force Feedback Haptic device as a 3D mouse Identifying if we are touching the surface Haptic device as a 3D mouse Identifying if we are touching the surface Haptic feedbackNo haptic feedback Lips4 minutes2.5 minutes Nose1.6 minutes2 minutes Water bottle6 minutes Finger3 minutes Door handle3.5 minutes3 minutes Mouse2 minutes Mountain3 minutes2 minutes Cream bottle4 minutes

37. Example This curve in the lips can be clearly created in the first image but does not exist in the second image. The height of the lips does not seem natural in the second image. This line between the lips clearly separates them but it is not the case for the second image.

38. Results Aesthetic quality of the image and 3D model are better when using haptic interaction Some curves in the mesh are not steady Same results can be achieved without force feedback but time will increase. Aesthetic quality of the image and 3D model are better when using haptic interaction Some curves in the mesh are not steady Same results can be achieved without force feedback but time will increase.

39. Conclusion

40. Haptic interaction provides an easier to use and faster way to sculpt a mesh. We implemented a few of the many tools that will enhance the sculpting process. Practice is needed to perceive the cursor’s Z location. Haptic interaction provides an easier to use and faster way to sculpt a mesh. We implemented a few of the many tools that will enhance the sculpting process. Practice is needed to perceive the cursor’s Z location.

41. Conclusion Real time interaction is important NURBS provided flexibility and important properties Sense of touch can provide new forms of interaction Real time interaction is important NURBS provided flexibility and important properties Sense of touch can provide new forms of interaction

42. Future Work

43. New editing tools  E.g. a selection tool Adding knots on the fly  NURBS parameters should be adjusted NURBS limitations  How to model hands, bottles? New editing tools  E.g. a selection tool Adding knots on the fly  NURBS parameters should be adjusted NURBS limitations  How to model hands, bottles?

44. Future Work Different views of the mesh  Each view has a different angle Integration with image editing software  Attract users attention Image recognition  Identify boundaries and apply rules to the control points near the boundary 3D Paint Different views of the mesh  Each view has a different angle Integration with image editing software  Attract users attention Image recognition  Identify boundaries and apply rules to the control points near the boundary 3D Paint