1 High Performance Computing symposium HPC2000, April 16-20, Washington, D.C. Samir Otmane CEMIF, Complex System Laboratory Http : 40 Rue du Pelvoux Evry, France Tél : 01/69/47/75/04 Fax : 01/69/47/75/99 High Performance Computing symposium HPC2000, April 16-20, Washington, D.C. Samir Otmane CEMIF, Complex System Laboratory Http : 40 Rue du Pelvoux Evry, France Tél : 01/69/47/75/04 Fax : 01/69/47/75/99 A R I T I Augmented Reality Interface for Telerobotic applications via Internet

2 Contents b Introduction b Human Computer Interaction in Tele-robotic environment Interaction with local and remote task environmentInteraction with local and remote task environment b Interactions with ARITI b System description : Robot slave - Task board - Virtual fixture board Robot slave - Task board - Virtual fixture board Hardware and communication board Hardware and communication board b Experiments and Results  Conclusion and perspectives

3 Introduction - Tele-work - Master site Communication support Slave site Master site Communication support Slave site The slave site is distant from the master site.The slave site is distant from the master site. Information feedback is corrupted by a bandwidth limitation of communication support.Information feedback is corrupted by a bandwidth limitation of communication support. Time delay is not constant when using any communication network.Time delay is not constant when using any communication network. No portable and user-friendly Tele-work systems.No portable and user-friendly Tele-work systems. Human performances are decreased during direct control of remote Tele-manipulation task.Human performances are decreased during direct control of remote Tele-manipulation task. Sending orders Sending orders Information feedback Information feedback

4 Visual Interaction with a local environment and completion of task through the use of a Robotic Interface.Interaction with a local environment and completion of task through the use of a Robotic Interface. Interaction between Human and local Task Environment Interaction between Human and local Task Environment Visual Auditory Tactile Robotic System System I / O Hardware Robotic Interface Task Environment Perceptual Cognition Motor Human Processor

5 Interaction between Human and remote Task Environment Interaction between Human and remote Task Environment b During interaction control of a remote robotic terminal tool, the user must Perform a physical action to initiate motion from the robot,Perform a physical action to initiate motion from the robot, Wait for the system to respond,Wait for the system to respond, Perceive the physical effect onto the robot and task environment,Perceive the physical effect onto the robot and task environment, Decide what to do next,Decide what to do next, Repeat the cycle until the task is completed.Repeat the cycle until the task is completed. * Motor activity is initiated through interaction with a software interface via Keyboard and mouse, joystick, master arm, etc... HARDHARDHARDHARD SOFTSOFTSOFTSOFT Human Operator Robotic Interface Remote Robotic Environment NETW

6 Interactions with A R I T I b Three kinds of visual assistance are given to human operator for friendly human computer interaction using the ARITI interface. These visual helps are devoted to : ÊEnvironment perception ËRobot control ÌRobot supervision Perception Control Supervision Human Operator HARDHARDHARDHARD NETW Remote Environment Robotic Interface

7 Assistance for Environment Perception Assistance for Environment Perception Several Virtual view points + Video image feedback Perception module Human Operator In Control module

8 Assistance for Robot Control Assistance for Robot Control Virtual robot + Virtual Fixtures Control module Human Operator In Supervision module

9 Assistance for Robot Supervision Assistance for Robot Supervision Textual information of the current task + Overlaid Model / Image Supervision module Human Operator

10 System description Robot slave b The CEMIF experimental site is a mechanism with four DOF (degrees of freedom) 2 Translations parallel to the ground2 Translations parallel to the ground 2 Rotations (Site and Azimuth)2 Rotations (Site and Azimuth) Peg mounted on a turret

11 System description Task Board b The robot is assumed to assemble (place) and disassemble (pick) objects hanging on a metal stand Objects Metal stand Head of the robot peg Target

12 System description Virtual Fixtures Board System description Virtual Fixtures Board b Human operator can create and use virtual fixtures to control the robot very easier. b Some examples of simple Virtual Fixtures (VF) : DiscSpherePlan Super-ellipsoidCone CylinderPipe Cube or Square

13 System description Virtual Fixtures Structure

14 System description Hardware b ARITI system has been implemented on a PC Pentium 233 Mhz with a 128 Mo RAM. b The PC is equipped with a Matrox Meteor video acquisition card connected to a black and white camera. b The orders are sent via the RS232 serial link. RS232 serial link Orders Video acquisition Video acquisition

15 System description Software b ARITI system is implemented under LINUX operating system. b ARITI interface is written based on JAVA object programming language b Video server is written using the C standard language. b Control server is written using the C and ASM (Microprocessor Assembly Language ) Video Client Control Client -ARITI- INTERFACE Applet JAVA L I N U X - O S - Video server Control server C and ASM SOKET

16 To use the ARITI system A R I T I System Camera Robot WWW CLIENTS + Internet Browser

17 The ARITI Display The ARITI Display

18 Experiments r Pick and place task rTele-operation mode r Control the real robot via the virtual robot r10 human operators (HO) r3 kinds of test r Without Virtual Fixtures r With passive Virtual Fixtures r With active (attractive) Virtual Fixtures rEach HO makes 10 tests for each kind

19 Results Reach a cylinder N° 1 è Reach a 3D target point on the peripheral of the cylinder N°1 è Without virtual fixtures there is 1,49 collision for each test

20 Results Reach a cylinder N° 1 Imprecision errors on X, Y, Z axis Err < 0,25 mm with active VFs Average time 7,7 sec with active VFs

21 Results Pick and place a cylinder N° 1 Results Pick and place a cylinder N° 1 b - Blue - with passive VFs. b - Red - with active (attractive potential fields) VFs. Unhook a cylinder N° 1 Passive VFs - 12,78 sec Active VFs - 9,5 sec Place a cylinder into the stand, Passive VFs - 37,96 sec Active VFs - 7,86 sec

22

23

24

25 Conclusion è Virtual reality and Augmented Reality technologies are used to : 4Overcome the instability of time delay, 4Complete or compensate the information feedback (video feedback for instance) 4Increase the human operator performances 4Thanks to Virtual Fixtures : 4best accuracy < 0,25 mm 4best completion time 4best safety è JAVA programming Language is used to implement the Man Machine Interface of ARITI system to : 4 Give a portable system and 4 User-friendly Tele-work system

26 Perspectives b Use the ARITI system for mobile robot application, such as Navigation, obstacles avoidance, to assist disable person.Navigation, obstacles avoidance, to assist disable person.

27 Perspectives b Add Audio and Tactile feedback modules b Extend ARITI system to do a cooperative Tele-Work Extended A R I T I System User 1 User 2 User n... Robot 1 Robot 2 Robot m... NETNETNETNET