1 33rd Annual Simulation symposium ANSS2000, April 16-20, Washington, D.C Samir Otmane CEMIF, Laboratoire Systèmes Complexes Http : 40 Rue du Pelvoux Evry, France Tél : 01/69/47/75/04 Fax : 01/69/47/75/99 33rd Annual Simulation symposium ANSS2000, April 16-20, Washington, D.C Samir Otmane CEMIF, Laboratoire Systèmes Complexes Http : 40 Rue du Pelvoux Evry, France Tél : 01/69/47/75/04 Fax : 01/69/47/75/99 Active Virtual Guides as an Apparatus for Augmented Reality Based Telemanipulation System on the Internet

2 Contents b Introduction b ARITI System b Virtual guides/fixtures Unified FormalismUnified Formalism Simple and Complex Virtual fixturesSimple and Complex Virtual fixtures Virtual guides representationVirtual guides representation Manipulation on the screenManipulation on the screen Deforming guides to generate an appropriate guidesDeforming guides to generate an appropriate guides b Experiments and Results  Conclusion and perspectives

3 Introduction - Tele-work - Master site Communication support Slave site Master site Communication support Slave site The slave site is distant from the master site.The slave site is distant from the master site. Information feedback is corrupted by a bandwidth limitation of communication support.Information feedback is corrupted by a bandwidth limitation of communication support. Time delay is not constant when using any communication network.Time delay is not constant when using any communication network. No portable and user-friendly Tele-work systems.No portable and user-friendly Tele-work systems. Human performances are decreased during direct control of remote Tele-manipulation task.Human performances are decreased during direct control of remote Tele-manipulation task. Sending orders Sending orders Information feedback Information feedback

4 è Virtual reality and Augmented Reality technologies are used to : 4Overcome the instability of time delay, 4Complete or compensate the information feedback (video feedback for instance) è JAVA programming Language is used to implement the Man Machine Interface of ARITI system to : 4 Give a portable system and 4 User-friendly Tele-work system ARITI system

5 Interaction between Human and remote Task Environment Interaction between Human and remote Task Environment b During interaction control of a remote robotic terminal tool, the user must Perform a physical action to initiate motion from the robot,Perform a physical action to initiate motion from the robot, Wait for the system to respond,Wait for the system to respond, Perceive the physical effect onto the robot and task environment,Perceive the physical effect onto the robot and task environment, Decide what to do next,Decide what to do next, Repeat the cycle until the task is completed.Repeat the cycle until the task is completed. * Motor activity is initiated through interaction with a software interface via Keyboard and mouse, joystick, master arm, etc... HARDHARDHARDHARD SOFTSOFTSOFTSOFT NETW Human Operator Robotic Interface Remote Robotic Environment

6 Interactions with A R I T I b Three kinds of visual assistance are given to human operator for friendly human computer interaction using the ARITI interface. These visual helps are devoted to : ÊEnvironment perception ËRobot control ÌRobot supervision Perception Control Supervision Human Operator HARDHARDHARDHARD NETW Remote Environment Robotic Interface

7 Assistance for Environment Perception Several Virtual view points + Video image feedback Perception module Human Operator In Control module

8 Assistance for Robot Supervision Assistance for Robot Supervision Textual information of the current task + Overlaid Model / Image Supervision module Human Operator

9 Assistance for Robot Control Assistance for Robot Control Virtual robot Control module Human Operator (HO) In Supervision module

10 System description Hardware b ARITI system is implemented on a PC Pentium 233 Mhz with a 128 Mo RAM. b The PC is equipped with a Matrox Meteor video acquisition card connected to a black and white camera. b The orders are sent via the RS232 serial link. RS232 serial link Orders Video acquisition Video acquisition

11 System description Software b ARITI system is implemented under LINUX operating system. b ARITI interface is written based on JAVA object programming language b Video server is written using the C standard language. b Control server is written using the C and ASM (Microprocessor Assembly Language ) Video Client Control Client -ARITI- INTERFACE Applet JAVA L I N U X - O S - Video server Control server C and ASM SOKET

12 To use the ARITI system A R I T I System Camera Robot WWW CLIENTS + Internet Browser

13 The ARITI Display The ARITI Display

14 Question !! How to increase Human Operator performances to do Telemanipulation task very easier ?

15 Increase Assistance for Robot Control Increase Assistance for Robot Control Virtual robot + Virtual Fixtures Control module Human Operator In Supervision module

16 Virtual Fixtures Structure Virtual Fixtures Structure

17 Simple Virtual Fixtures Simple Virtual Fixtures b Human operator can create and use virtual fixtures to control the robot very easier. b Some examples of simple Virtual Fixtures (VF) : DiscSpherePlan Super-ellipsoidCone CylinderPipe Cube or Square

18 Complex Virtual Fixtures Complex Virtual Fixtures Delimiting the workspace between two robots in cooperation Following an arbitrary trajectory by the robot Reaching a dangerous target with the robot end tool

19 Method of construction b taking some significant points on the surface of the guide b joining these points in order to get a wire frame representation  Example : If a parametric equation of the guide is :If a parametric equation of the guide is : with with Then the vertexs are :Then the vertexs are : And the segments are :And the segments are :

20 Manipulation on the screen b Use of the graphic camera model Is the matrix M which transform point coordinates (Xo, Yo, Zo ) In the referential Ro, onto the screen point coordinates (U, V).Is the matrix M which transform point coordinates (Xo, Yo, Zo ) In the referential Ro, onto the screen point coordinates (U, V).

21 Selection of the 3D Fixtures on the screen Selection of the 3D Fixtures on the screen è The designation on the screen = 2D point (U, V) è Determining what object 3D wanted to designate è The designated point belongs to a D segment, witch equation is :

22 Selection of the 3D Fixtures on the screen è Determining what vertex on the 3D objects having the smallest distance from the line segment D. Selected point Screen Selected fixture

23 Deforming Virtual Fixtures  Each virtual fixture is associated to a graph where X is a set of vertexs, and U a set of lines. where X is a set of vertexs, and U a set of lines. b We define an application V which associates any vertex x in X a set of his neighbors : b We call a distance between the vertex x and y.

24 Deforming Virtual Fixtures b If x 0 is a start point of deformation and b  0 the value of this deformation, b Then the value of deformation of the fixture is given by : b Where p is called initial propagation factor b and f is called the dissipation of propagation factor

25 Deforming Virtual Fixtures P = 0.99, f = 0.9 P = 0.99, f = 0.99 P = 0.99, f = 0.9

26 Experiments r Pick and place task rTele-operation mode r Control the real robot via the virtual robot r10 human operators (HO) r3 kinds of test r Without Virtual Fixtures r With passive Virtual Fixtures r With active (attractive) Virtual Fixtures rEach HO makes 10 tests for each kind

27 Experiments Task Board b The robot is assumed to assemble (place) and disassemble (pick) objects hanging on a metal stand Objects Metal stand Head of the robot peg Target

28 Fixture to reach a target b A simple geometric primitive : Cone.Cone.

29 Fixture to pick the object and unhook it b A complex Virtual guide Combining a 3 simple guides (cylinders)Combining a 3 simple guides (cylinders)

30 Fixture to place the object onto the stand Fixture to place the object onto the stand b A complex Virtual guide Combining 4 simple guides ( 1 cone + 3 cylinders)Combining 4 simple guides ( 1 cone + 3 cylinders)

31 Results Reach a cylinder N° 1 è Reach a 3D target point on the peripheral of the cylinder N°1 è Without virtual fixtures there is 1.49 collision for each test

32 Results Reach a cylinder N° 1 Imprecision errors on X, Y, Z axis Err < 0,25 mm with active VFs Average time 7,7 sec with active VFs

33 Results Pick and place a cylinder N° 1 Results Pick and place a cylinder N° 1 b - Blue - with passive VFs. b - Red - with active (attractive potential fields) VFs. Unhook a cylinder N° 1 Passive VFs - 12,78 sec Active VFs - 9,5 sec Place a cylinder into the stand, Passive VFs - 37,96 sec Active VFs - 7,86 sec

34

35

36

37 Conclusion è Thanks to Virtual Fixtures the human operator performances are increased : 4 best accuracy < 0,25 mm 4 best completion time 4 best safety è JAVA programming Language is used to implement these Virtual Fixtures into the ARITI system to : 4 Give a portable and interactive fixtures.

38 Perspectives b Use the Virtual Fixtures for mobile robot application, such as Navigation, obstacles avoidance, to assist disable person.Navigation, obstacles avoidance, to assist disable person.

39 Perspectives b Add Tactile functions to activate fixtures b Extend the use of virtual fixtures to do a cooperative Tele- Work Extended A R I T I System User 1 User 2 User n... Robot 1 Robot 2 Robot m... NETNETNETNET