Scenarios and Clinical Trials Annemarie Kokosy, Gareth Howells, Mohamed Sakel & Matthew Pepper ISEN/University of Kent/EKHUFT January 27 th, 2012 Ecole Centrale of Lille 1 Part-financed by the European Regional Development Fund

Part-financed by the European Regional Development Fund Plan of presentation I.First Scenario 1.Description 2.Prototype 3.Clinical evaluation II. Second Scenario 1.Description 2.Prototype 3.Clinical evaluation III. Third scenario 2

Part-financed by the European Regional Development Fund First Scenario – Collision Avoidance Description – The user drives the powered wheelchair via a joystick – The intelligent device must Detect obstacles Slow down the PW proportionally to the distance between the obstacle and the wheelchair Stop the PW if the distance between the PW and the obstacles is less than the security distance, overriding any action by the user Provide the user with visual feedback on the distance (measured by the sensors) between the wheelchair and the obstacles in its way Have a switch to allow the user to enable or disable the device 33

Part-financed by the European Regional Development Fund First Scenario Hardware Implementation at ISEN ISEN Prototype – 2 electronic boards: 1 for the joystick and 1 (Arduino) for the data processing and navigation strategy – 9 US sensors – 2 IR sensors – visual feedback DupontMedical with DynamicControls 4 Invacare Storm 3 with DynamicControls

Part-financed by the European Regional Development Fund First Scenario Hardware Implementation at UoK ISEN Prototype – The technical report to replicate the intelligent module (deliverable A3D1) is available and was sent to University of Kent at the end of November 2011 DupontMedical with DynamicControls 5 Invacare Storm 3 with DynamicControls

Part-financed by the European Regional Development Fund First scenario Hardware Implementation at UoK Prototype at University of Kent INVACARE Harrier Plus wheelchair Dynamics Control System 6 Processing, Beagle board or equivalent. Sensor boards, Arduino or equivalent. Sonar 5m (LV-MaxSonar-EZ4 MB1040). I/R 5m (SHARP - GP2Y0A710K0F). I/R 1.2m (SHARP-P2Y0A02YK0F). Optical camera and Fisheye lens. Magnetic compass (CMPS10). Floor colour sensor (Inex). Dynamic object sensor (GE/ ZTP-135S). Drive shaft angular rate (existing motor). Angular body rate (MLX90609-E2). Acceleration/velocity (ADXL320). Position (optical mouse sensor).

Part-financed by the European Regional Development Fund First scenario Clinical Trial: Hospital of Garches 77 First part Second part

Part-financed by the European Regional Development Fund First scenario: Clinical Trial - HG Pilot study of 27 people divided into 3 groups – Group1: 9 healthy volunteers who have never driven an electric wheelchair – Group2: 9 experienced electric wheelchair users – Group3: 9 users who didn’t pass the electric wheelchair driver’s licence Goal: reduce the number and severity of collisions Methodology: create a circuit that simulates a real life indoor environment (walls, doors, obstacles) 88

Part-financed by the European Regional Development Fund First scenario: Clinical Trial - HG Evaluation Criteria: the number of collisions during the test, and the time the user needed to finish it. Total duration of the test: 3 hours Duration of the study: 5 months Ethics Approvals: The clinical protocol has been approved by AFSSAPS (French agency for health and sanitary security) and the CPP (Commission for the protection of people) at the end of December 2011 » The clinical protocol and the observation booklet were sent to EKHUFT in November

Part-financed by the European Regional Development Fund First Scenario: Clinical Trial - HG 3 users (from groups 1 & 3), have already taken the test Some technical problems were identified – To go through doors (90cm) – To navigate along the corridor with obstacles (distance wall – obstacle = 1m) – For reverse navigation Solution (in progress) To better locate the sensors to reduce dead zones To change the navigation strategy 10

Part-financed by the European Regional Development Fund First Scenario: Clinical Trial - EKHUFT Pilot Study to Confirm Findings from HG – 9 healthy volunteers – 9 Patients selected by Dr Sakel Provision of ISEN interface by UoK - ? Testing of ISEN Interface at UoK - ? Ethics Approval: The clinical protocol has to be developed and application made to the National Research Ethics System via the Integrated Research Application System (IRAS) – October

Part-financed by the European Regional Development Fund First Scenario: Clinical Evaluation - EKHUFT Repeated for each patient over Hospital stay Evaluation Criteria: – Number of collisions during the completion of the clinical test path – The time to complete the path. Patient Evaluation/Experience – Questionnaire to be developed Subject Participation duration: 1hour? Duration of the study: 5 months Possibly at same time as Scenario 2? 12

Part-financed by the European Regional Development Fund Second scenario (HG) Description – The user drives the powered wheelchair using a joystick – The intelligent device must Detect obstacles Detect Steps/Stairs Avoid obstacles, overriding any user action if necessary Go autonomously through doorways Provide feedback to the user (must be visual or audible) Have a switch to allow the user to enable or disable the device Have a switch to allow the user to enable the “pass through doorways” mode 13

Part-financed by the European Regional Development Fund Second scenario Prototype: new challenges Obstacle avoidance  idea: use the potential field method (the joystick direction=attractor; obstacles = repulsor) Second prototype vs first one – Use of US and IR sensors for obstacle detection – Use the Arduino board for data processing – New data: camera and/or laser for position and velocity estimations – Use of a PC or mother board (If PC not allowed for the ethics approval) for data processing and navigation algorithm – What kind of device for the user's feedback ? – A switch to change between the autonomous or semi-autonomous navigation 14

Part-financed by the European Regional Development Fund Second scenario Prototype: who uses what kind of sensors and for which goal? 15 ISEN/EC LilleUniversity of Kent University of Essex US and IRObstacle detection localisationperception cameralocalisationlocalization Kinectperception Laserperception Inertial sensorslocalization GPSlocalization

Part-financed by the European Regional Development Fund Second scenario Main results 16 ISEN/EC LilleUniversity of KentUniversity of Essex LocalizationUse one camera and one or two landmarks (OK on simulation) Use US (indoor), GPS (outdoor), inertial navigation system on the real robots Perception Use of US and IR sensors Static obstacles Drawback: the system is too slow ?Kinect, laser, camera, to build the maps (SLAM) Speed estimation Use one camera and one or two landmarks (OK on simulation) Observers and estimators Inertial navigation system Path Planning MichelA* Tracking and Control Sarah & ThierryPID Control, Fuzzy (Huosheng)

Part-financed by the European Regional Development Fund Second Scenario: Clinical trial Hospital of Garches – Define the clinical protocol with the SYSIASS team – Ask for the Ethics approval – Initial Evaluation of the prototype in a clinical environment – January 2013? 17

Part-financed by the European Regional Development Fund Third Scenario Based on the first results of the survey on user needs Survey – Start date: November 2011 – End date: April 2012 – Questionnaire available in French and English on our web site ( – Realized by ISEN/GHICL in collaboration with 2 rehabilitation centers (Berck sur Mer and Villeneuve d’Ascq) and the Social Institute of Lille 18

Part-financed by the European Regional Development Fund First results of survey: Conclusion Scenario 1: collision avoidance  The intelligent device doesn’t correct the trajectory Scenario2: assisted navigation in semi-autonomous way  The user drives the powered wheelchair  The intelligent device avoids obstacles with the user action on the loop  The going through doorways is autonomous (the user chooses this option) The survey results allow us to define a third scenario: autonomous navigation EKHUFT also proposes a scenario (see the next presentation) 19 Third scenario

Part-financed by the European Regional Development Fund Project plan – Time line 20 Feb/2012Aug/2012Feb/2013Aug/ Dec/2013 End 1st scenario Technical work 4/126/1210/1212/124/136/1310/1312/13 Clinical trials 2nd scenario 3rd scenario Technical work Clinical trials Technical work Clinical trials Ethics Approval