A haptic presentation of 3D objects in virtual reality for the visually disabled MSc Marcin Morański Professor Andrzej Materka Institute of Electronics, Technical University of Lodz, Poland MSc Marcin Morański Professor Andrzej Materka Institute of Electronics, Technical University of Lodz, Poland

Plan of the presentation Project motivation Background A haptic presentation system The system working principle Experiments with blind users Conclusions ICDVRAT 2012: A haptic presentation of 3D objects in virtual reality for the visually disabled 2

3 Project motivation 4 out of 1000 Europeans are visually impaired or blind Lack of sight is a deprivation of 80%-90% perceptual abilities The ageing demographics increases the number of vision problems Source:

ICDVRAT 2012: A haptic presentation of 3D objects in virtual reality for the visually disabled 4 Aim of the project Goals: to construct a prototype system for the tactile presentation of real objects in a virtual reality; to check the opportunity of the application of the tactile presentation system for the blind and visually impaired; to examine the usability and potential of the force feedback device, Falcon Novint, for a 3D virtual object presentation without the usage of vision. Assumptions: The prototype system should integrate a depth camera and a haptic device.

Background ICDVRAT 2012: A haptic presentation of 3D objects in virtual reality for the visually disabled5 Source:

ICDVRAT 2012: A haptic presentation of 3D objects in virtual reality for the visually disabled 6 A haptic presentation system The block diagram of the prototype

The system working principle ICDVRAT 2012: A haptic presentation of 3D objects in virtual reality for the visually disabled7

Virtual scene modeling ICDVRAT 2012: A haptic presentation of 3D objects in virtual reality for the visually disabled8 a) b) c) a) 2.5D depth map of the scene, b) the segmented scene, c) the reconstructed scene for the tactile presentation.

Experiments (1) Participants: 8 blind persons (2 women and 6 men). Setup: the objects in each scene were located on the ground, against a background; an exploration time of every scene was measured; the task was carried out successfully when the blind person correctly identified all obstacles in the presented scene. ICDVRAT 2012: A haptic presentation of 3D objects in virtual reality for the visually disabled9 a) b) a) An example of the scene, b) the experimental setup

Experiments (2) The experiments were divided into 4 stages: 1.Training phase; 2. Scene content recognition; 3. Distance estimation to each obstacle from a chosen point of observation; 4. Estimation of obstacles’ height. ICDVRAT 2012: A haptic presentation of 3D objects in virtual reality for the visually disabled10

ICDVRAT 2012: A haptic presentation of 3D objects in virtual reality for the visually disabled11 Summary of experiment results

Comments and suggestions of the blind participants (1) ICDVRAT 2012: A haptic presentation of 3D objects in virtual reality for the visually disabled12  a special focus is required when creating small objects and scene containing many details;  adding vocal information about the 3D position of the probe (the virtual finger in the system)  the sonification of the some of the scene points or objects would also be very useful

Comments and suggestions of the blind participants (2) ICDVRAT 2012: A haptic presentation of 3D objects in virtual reality for the visually disabled13 There are a couple of potential application when such a system could help the blind in everyday activities:  new route learning in buildings (e.g. post office, bank, school);  tactile interaction when using various computer applications (e.g. web browsers);  tactile interactions with objects in museums.

Experiment conclusions ICDVRAT 2012: A haptic presentation of 3D objects in virtual reality for the visually disabled14 The performed experiments have shown that the system can be useful for the blind, although there are still some challenges to overcome. Many requirements need to be met, mainly: choosing the scale of virtual objects in the ratio to the real ones; solving the problem of losing oneself in VR; solving the problem of presenting scenes consisting of many objects/details (each of a different size), as the system’s resolution is finite.

Conclusions 1.A prototype system for tactile presentation of real objects in a virtually reality was presented. 2.The system usability was examined by the blind participants. 3.The designed system have potential to be a very useful tool for visually disabled in many applications, such as building virtual maps and creating simulators where real objects are presented virtually. ICDVRAT 2012: A haptic presentation of 3D objects in virtual reality for the visually disabled15

Acknowledgements This work has been supported by the Ministry of Science and Higher Education of Poland research grant no. NR in years The presented study is carried out with cooperation with Lodz Chapter of the Polish Society for the Blind. ICDVRAT 2012: A haptic presentation of 3D objects in virtual reality for the visually disabled16