Spatiotemporal Information Processing No.4 3 components of Virtual Reality-3 Display System Kazuhiko HAMAMOTO Dept. of Information Media Technology, School of Information and Telecommunication Eng., Tokai University, Japan

Today’s Contents  Somatic sense Skin sense Deep sense Haptic sense  Display system – skin sense The principle of tactile primary color  Display system – deep sense  Display system – haptic sense

The relationship among 3 components (review) Person real spacevirtual space Simulation system Display system Sensing system computer

Somatic sense (1/4)  Cutaneous sense (or sense of touch or skin sense) A sense by some sensory receptors which lie beneath the surface of the skin tactile sense, temperature sense, pain sense

Somatic sense (2/4)  Deep sense (or proprioceptive sense) A sense of the relative position of neighboring parts of the body, whose receptors are found in muscles, tendons and joints kinetic sense, postural sense

Somatic sense (3/4)  How can “iron ball” be felt by human sense ? Shape ?  Skin sense or deep sense Surface pattern (texture) ?  Skin sense or deep sense Recognition of “iron” ?

Somatic sense (4/4)  Recognition of shape If we wear gloves…. We can feel -> deep sense  Recognition of pattern If we wear gloves…. We cannot feel -> skin sense  Recognition of “iron” Temperature sense

Haptic sense  “Active touch” can improve the precision of the recognition  Haptic sense Active touch sense, which is integrated sense of skin sense and deep sense with action of the body

Skin sense and receptors  Tactile sense Merkel disk  displacement Meissner corpuscle  velocity Ruffini corpuscle  displacement  Sense of contact force (pressure) Ruffine corpuscle  displacement Pacinian corpuscle  acceleration

Skin sense and receptors

Deep sense and receptors  Receptor in muscle Muscle spindle Stretch receptor, which sends muscle’s length information to CNS  Receptor in tendon (joint) Golgi tendon organ Another type of receptor, which provides information about changes in muscle tenstion

Muscle spindle CNS : Central nervous system

Golgi tendon organ

Display system – skin sense  Virtual sandpaper (M.Minsky, MIT, 1995) Recognition of “type of Texture” Fine uneven surface  Vibration of small amplitude, high frequency Rough uneven surface  Vibration of large amplitude, low frequency

Virtual Sandpaper Margaret Minsky, MIT, 1995 Experimental setup Screen image

Display system – skin sense  The principle of tactile primary colors A kind of functional electrical stimulation (FES) Each receptor is stimulated electrically and independently 3 receptors can be stimulated independently by the differences of how deep seated or singularity of feature along axone.

Prototype using the principle of tactile primary colors  Sense of pressure, low and high freq. vibration can be presented (2005) Prototype of the tactile display

Display system – deep sense  Equipment type A display device is equipped on hand or fingers. Action of hand or fingers is restricted by the display device. The restriction causes a feeling of “existence of an object”

Display system – deep sense Cyber Grasp

 SPIDAR  8 wires and 8 motors detect the position and the direction of the ball  User controls the ball in 6DOF  Sense of touch or collision is transformed to tension of the wires Display system – deep sense

Display system – Haptic sense  Haptic display Present tactile information Stimulus of tactile organ Force feedback system Shape, hardness, softness, smoothness, friction and gravity are perceived in virtual space.

Display system – Haptic sense  PHANToM Desktop (SensAble Device, Inc.)  6DOF  Refresh rate  1000 [Hz]  Simple structure  Torque generated by motors is transmitted to active part by wire.

Display system – deep sense  Problems Always feel a display’s weight and movement Difficult to create “we don’t touch anything” Response of a motor (actuator)

 Deep sense Artificial muscle is used as switching device  Skin sense Uneven feeling can be realized by MR fluid Haptic interface Actuated with Magnetorheological fluid and Artificial muscle HAMA デバイス Novel Development in my Lab.

Power supply New haptic device Electric circuit for drive Artificial muscle switch for deep sense MR fluid for sense of cutaneous

New device for deep sense with Artificial muscle movable part Lock by BMX Unlock by BMX Restricted fingerReleased finger

New device for skin sense with MR fluid When MR fluid into electro magnetic field, the viscosity becomes higher