1. A haptic presentation of 3D objects in virtual reality for the visually disabled MSc Marcin Morański Professor Andrzej Materka Institute of Electronics, Technical University of Lodz, Poland www.eletel.eu MSc Marcin Morański Professor Andrzej Materka Institute of Electronics, Technical University of Lodz, Poland www.eletel.eu

2. Plan of the presentation Aim of the project Background A haptic presentation system The system working principle Experiments with blind users Conclusions ICDVRAT 2012: A haptic presentation of 3D objects in virtual reality for the visually disabled 2

3. 3 Aim of the project Goals: to construct a prototype system for the tactile presentation of real objects in a virtual reality; to check the opportunity of the application of the tactile presentation system for the blind and visually impaired; to examine the usability and potential of the force feedback device, Falcon Novint, for a 3D virtual object presentation without the usage of vision. Assumptions: The prototype system should integrate a depth camera and a haptic device.

4. Background ICDVRAT 2012: A haptic presentation of 3D objects in virtual reality for the visually disabled4 Source : http://www.niewidomi.com.pl

5. ICDVRAT 2012: A haptic presentation of 3D objects in virtual reality for the visually disabled 5 A haptic presentation system The block diagram of the prototype

6. The system working principle ICDVRAT 2012: A haptic presentation of 3D objects in virtual reality for the visually disabled6

7. Virtual scene modeling ICDVRAT 2012: A haptic presentation of 3D objects in virtual reality for the visually disabled7 a) b) c) a) 2.5D depth map of the scene, b) the segmented scene, c) the reconstructed scene for the tactile presentation.

8. Experiments (1) Participants: 8 blind persons (2 women and 6 men). Setup: the objects in each scene were located on the ground, against a background; an exploration time of every scene was measured; the task was carried out successfully when the blind person correctly identified all obstacles in the presented scene. ICDVRAT 2012: A haptic presentation of 3D objects in virtual reality for the visually disabled8 a) b) a) An example of the scene, b) the experimental setup

9. Experiments (2) The experiments were divided into 4 stages: 1.Training phase; 2. Scene content recognition; 3. Distance estimation to each obstacle from a chosen point of observation; 4. Estimation of obstacles’ height. ICDVRAT 2012: A haptic presentation of 3D objects in virtual reality for the visually disabled9

10. 10 Summary of experiment results

11. Comments and suggestions of the blind participants ICDVRAT 2012: A haptic presentation of 3D objects in virtual reality for the visually disabled11  a special focus is required when creating small objects and scene containing many details;  adding vocal information about the 3D position of the probe (the virtual finger in the system)  the sonification of the some of the scene points or objects would also be very useful  there are a couple of potential application when such a system could help the blind in everyday activities.

12. Experiment conclusions ICDVRAT 2012: A haptic presentation of 3D objects in virtual reality for the visually disabled12 The performed experiments have shown that the system can be useful for the blind, although there are still some challenges to overcome. Many requirements need to be met, mainly: choosing the scale of virtual objects in the ratio to the real ones; solving the problem of losing oneself in VR; solving the problem of presenting scenes consisting of many objects/details (each of a different size), as the system’s resolution is finite.

13. Conclusions 1.A prototype system for tactile presentation of real objects in a virtually reality was presented. 2.The system usability was examined by the blind participants. 3.The designed system have potential to be a very useful tool for visually disabled in many applications, such as building virtual maps and creating simulators where real objects are presented virtually. ICDVRAT 2012: A haptic presentation of 3D objects in virtual reality for the visually disabled13

14. Acknowledgements This work has been supported by the Ministry of Science and Higher Education of Poland research grant no. NR02008310 in years 2010 - 2013. The presented study is carried out with cooperation with Lodz Chapter of the Polish Society for the Blind. ICDVRAT 2012: A haptic presentation of 3D objects in virtual reality for the visually disabled14