Presentation is loading. Please wait.

Presentation is loading. Please wait.

An Assistive Navigation Paradigm for Semi-Autonomous Wheelchairs Using Force Feedback and Goal Prediction Master’s Thesis Defense Candidate: John Staton.

Similar presentations


Presentation on theme: "An Assistive Navigation Paradigm for Semi-Autonomous Wheelchairs Using Force Feedback and Goal Prediction Master’s Thesis Defense Candidate: John Staton."— Presentation transcript:

1 An Assistive Navigation Paradigm for Semi-Autonomous Wheelchairs Using Force Feedback and Goal Prediction Master’s Thesis Defense Candidate: John Staton Advisor: Dr. Manfred Huber Committee Members: Dr. David Levine, Dr. Gergely Zaruba John Staton 2008Computer Science & Engineering

2 Outline Introduction Related Work Concept Review Design Methodology Implementation Experiments Concluding Thoughts John Staton 2008Computer Science & Engineering

3 Introduction - Motivation “49.7 million: Number of people age 5 and over with a disability, according to Census 2000; this is a ratio of nearly 1-in-5 U.S. residents, or 19 percent.” 25 million had difficulty walking a quarter mile or climbing a flight of 10 stairs, or used an ambulatory aid, such as a wheelchair (2.2 million) or a cane, crutches or a walker (6.4 million). “The rate of power wheelchair prescriptions increased 33 fold from 1994 to 2001” John Staton 2008Computer Science & Engineering

4 Introduction – Early Work System NameSensors CPWNS Vision, Dead Reckoning The Intelligent Wheelchair Vision, Infrared, Sonar Intelligent Wheelchair System Vision, Sonar, Gesture Recognition INRO GPS, Sonar, Drop-Off Detector MAid Sonar, Infrared, Laser Range Finder, Dead Reckoning OMNI Sonar, Infrared, Bump, Dead Reckoning RobChair Sonar, Infrared, Bump Rolland Vision, Sonar, Dead Reckoning, Infrared, Bump Sensors SENARIO Dead Reckoning, Sonar SIRIUS Sonar, Dead Reckoning Smart Wheelchair Line Trackers, Bump Sensors Smart Wheelchair Ultrasonic Beacons TetraNauta Vision, Infrared, Sonar, Bump Sensors VAHM Sonar, Infrared, Dead Reckoning Wheelesely Vision, Infrared, Sonar John Staton 2008Computer Science & Engineering Intelligent Wheelchair Started in the 1990’s Numerous universities and labs However Only two companies sell smart wheelchairs for research use Only one is commercially available, only in Europe

5 Introduction – Inspiration John Staton 2008Computer Science & Engineering “The majority of research and development activity in the field of control and automation applied to powered mobility for people with disabilities has concerned sophisticated technology and techniques. … A more effective approach is to make use of the most flexible and adaptable intelligence on the chair – the user. To accomplish this, researchers must design, build and test their systems with real users and contexts in mind.” - Paul D. Nisbet University of Edinburgh

6 Introduction – Thesis John Staton 2008Computer Science & Engineering Something is disconnected –Smart wheelchair projects treat the wheelchair as an “autonomous unit” –Adults prefer individual independence Solution –Semi-Autonomous, assistive wheelchair –Communicate with user –User still provides drive direction

7 Introduction – Thesis John Staton 2008Computer Science & Engineering What communication technique to use? –Aural and Visual = distracting, both to user and to nearby observers –Haptics! Subtle Effective Intuitive Non-distracting Already shown to be useful for various tasks (mobility aids, steering tasks, bio-manipulation in virtual reality, surgical tasks…)

8 Related Work “Luoson III” – Lio, Hu, Chen, Lin National Chung Cheng University, Taiwan Specifics: –Blind user –Ultrasonic sensors –Motion Prediction –MS FF Pro John Staton 2008Computer Science & Engineering

9 Related Work Wheelchair University – Protho, LoPresti, Brienza University of Pittsburgh Specifics –Two design philosophies –Passive Assistance –Active Assistance –VR System John Staton 2008Computer Science & Engineering

10 Related Work Metz University, France – Fattouh, Sahnoun, Bourhis Reminiscent of Luoson III –Distance sensors –Averaged feedback forces –VR System –MS FF 2 John Staton 2008Computer Science & Engineering

11 How Is This Thesis Different? Previous research –Obstacle avoidance This research project –Obstacle avoidance & –Goal guidance Seeks to intuit the user’s intended goal Guide the user to that goal & away from obstacles –Active assistance John Staton 2008Computer Science & Engineering

12 Concept Review – Force Feedback “Haptics” – anything related to or based on the sense of touch Force Feedback – Haptics applied to an I/O device Touch Sensations –Vibration –Robust effects Emulate the feeling of weight, friction, liquid, and more. John Staton 2008Computer Science & Engineering

13 Concept Review – Force Feedback John Staton 2008Computer Science & Engineering

14 Concept Review – Force Feedback John Staton 2008Computer Science & Engineering Simple Vertical grip that pivots around a fixed end Angle of the joystick Or Displacement from neutral position Intuitive Effective Used in many applications Flight control Video games Electric Powered Wheelchairs Therapy

15 Concept Review – Force Feedback John Staton 2008Computer Science & Engineering “Effect” – The encapsulated force-response data sent to the FF device Categorized by three distinct dimensions: StaticDynamic One-shotOpen-ended InteractiveTime-based

16 Concept Review – Harmonic Functions Formal definition: “Harmonic Function” –Real function –Range in the real numbers –With continuous second partial derivatives –Satisfy Laplace’s Equation The sum of the second partial derivatives equal zero No local maxima or minima Smooth and differentiable John Staton 2008Computer Science & Engineering

17 Concept Review – Harmonic Functions John Staton 2008Computer Science & Engineering

18 Concept Review – Harmonic Functions Used repeatedly, and with great success Path planning in a known environment Potential value = probability of collision 1.0 – obstacle, 0.0 – goal Gradient = direction away from obstacles and toward goals John Staton 2008Computer Science & Engineering

19 Concept Review – Applying Harmonic Functions John Staton 2008Computer Science & Engineering Iterate through the entire grid, where for every grid[i,j]: R e = 0.25 x (nb1+nb2+nb3+nb4) – grid[i,j]; grid[i,j] = grid[i,j] + R e ; nb1 = grid[i-1,j] x w; nb2 = grid[i+1,j] x w; nb3 = grid[i,j-1] x w; nb4 = grid[i,j+1] x w; The maximum R e is saved for each iteration through the grid. As long as R e > then the process repeats. Successive Over- Relaxation (SOR) Iterative, numerical method Speed up convergence of the Gauss-Seidel method for solving linear systems of equations

20 Design Methodology Objectives –The ability to infer the user’s intention –The ability to help direct the user towards the intended goal and away from obstacles Design Methodology –Two looping procedures –Outer loop Infers goal –Inner loop Directs user to goal John Staton 2008Computer Science & Engineering

21 Design Methodology – Outer Loop John Staton 2008Computer Science & Engineering External User Preference System Goal Selection Harmonic Function Path Planning Run- Time System Goals Predicted Goal Grid Location, Orientation, Past Behavior Outer Loop

22 Design Methodology – Goal Selection John Staton 2008Computer Science & Engineering Inputs –Series of goals Each goal is initially weighted based on the knowledge of the external system of user preferences –Recent User Behavior Current Location Current Orientation Series of past user actions Output –Predicted user goal –Predicted likelihood for every goal Heuristic –“Confidence” –Modified (increased or decreased) based on the similarity of the user actions to the actions that would lead to the goal(s). External User Preference System Goal Selection Goals Predicted Goal Location, Orientation, Past Behavior Outer Loop

23 Design Methodology – Harmonic Function John Staton 2008Computer Science & Engineering Inputs –Predicted Goal –Environmental Data Grid Output –Harmonic function as applied to grid Potential value for every location Goal = 0.0 Obstacle = 1.0 All other locations 0.0 < grid[x, y] < 1.0 Algorithm –Successive Over-Relaxation (SOR) Harmonic Function Path Planning Run- Time System Predicted Goal Grid Outer Loop

24 Design Methodology – Inner Loop John Staton 2008Computer Science & Engineering Run- Time Loop Force Effect Generation Force Effect Playback (Joystick) Wheelchair Location, Orientation Motors Risk, Direction Force VectorMotor Command Movement External User Preference System Goal Selection Harmonic Function Path Planning Run- Time System Goals Predicted Goal Grid Location, Orientation, Past Behavior Outer Loop

25 Design Methodology – Force Vector Creation John Staton 2008Computer Science & Engineering Inputs –Wheelchair Location –Wheelchair Orientation Output –Force Vector Heuristic –Force Vector = direction, amount of force –Direction = away from obstacles, toward goal –Amount of force = contingent upon the “riskiness” of user action Run- Time Loop Force Effect Generation Wheelchair Location, Orientation Risk, Direction Force Vector Movement

26 Design Methodology – Force Direction John Staton 2008Computer Science & Engineering Force Direction –Direction of the harmonic function gradient (slope) relative to the wheelchair’s current orientation –Compute the angular difference between the gradient direction and the orientation of the wheelchair for use as the direction of the force vector Wheelchair Orientation Gradient

27 Design Methodology – Risk John Staton 2008Computer Science & Engineering Amount of force –“Risk” of current action Local Risk vs. Future Risk –Local: Wheelchair Velocity Current Potential Value (from Harmonic Function) Next Potential Value (from Harmonic Function) Difference between Current and Next –Future: Current Potential Value (from Harmonic Function) Potential Value some distance ahead, calculated based on current velocity Difference between Current and Future Allows for locally “risky” behavior if future risk is minimized

28 Design Methodology – Risk John Staton 2008Computer Science & Engineering More Formal –(V + Pc + (Pc-Pn)) = Local Risk Velocity = V, Pc = Current Potential, Pn = Next Potential –(Pc - Pf) = Future Risk Pc = Current Potential, Pf = Future Potential –grid[i+x, j+y] = Future potential X & Y are scaled based on V (and are dependant on orientation) How was this actually implemented? –“Levels” of risk For every risk factor that was present, a “level” of risk was added Discrete force effects

29 Design Methodology – Force Feedback Playback John Staton 2008Computer Science & Engineering Run- Time Loop Force Effect Playback (Joystick) Motors Force VectorMotor Command Movement Convert Force Vector to Force-Feedback effect –Thus communicating to the user: Severity of the situation/current action –The amount of force What action should be performed next –The direction of the force Send joystick position to motors as a motor command –Produces movement –Updates wheelchair’s position, orientation & velocity –Repeat loop

30 Implementation Dell Dimension 8250 –Pentium Ghz –512 MB RAM –Windows XP –Microsoft Sidewinder FF 2 Microsoft Visual Studio ‘05 –C# –Modifications to Microsoft Robotics Studio John Staton 2008Computer Science & Engineering

31 Implementation – Microsoft Robotics Studio John Staton 2008Computer Science & Engineering Joystick Path Planning Console Sensor Data

32 Implementation – Microsoft Robotics Studio John Staton 2008Computer Science & Engineering

33 Experiments Two loops –Goal Selection/Harmonic Function Path Planning –Force Feedback/Simulation Environment Two major sets of experiments –Goal Selection –User Testing of Simulation Environment Quantifiable data (time, number of collisions) Survey data John Staton 2008Computer Science & Engineering

34 Experiments – Goal Selection Experiments tested: –Similar or clustered goals vs. semi- similar goals vs. one distinct goal –User actions towards a goal, neutral actions, actions away from goal –High, medium and low initial goal weight –System predicts one goal and produces a prediction of it’s likelihood (100%, 50%, etc) John Staton 2008Computer Science & Engineering

35 Experiments – Goal Selection Results/Analysis John Staton 2008Computer Science & Engineering When goal is distinct 100% accuracy (average predicted likelihood: 100%) When goals are semi-similar 100% accuracy (average predicted likelihood: 80%) When goals are similar/clustered Each of the three clustered goals averaged a predicted likelihood of ~ 33% The goal of the three with the highest weight was selected by the system for each run Predicts the goal properly when it is reasonable to expect so!

36 Experiments – User Testing Each test subject was given time to familiarize themselves with the simulation, both with force-feedback and without Six test runs were given, three with FF, three without –Time to complete course –Number of Collisions Post Test Survey –Helpful for avoiding obstacles, helpful for approaching goal –Too forceful Not forceful John Staton 2008Computer Science & Engineering

37 Experiments – User Testing Results/Analysis John Staton 2008Computer Science & Engineering Without FF CollisionsWith FFCollisions Subject s s0 Subject s s0 Subject s s0 Subject s s0 Subject s s0.333 Subject s s0.333 (average) Six Subjects –Three pairs (one male, one female) –Ages: 20’s 40-50’s 70’s Subject 6 has hand tremors Three runs without force- feedback, three with, alternating All subjects showed: –Improved time to complete course with Force-Feedback –Fewer collisions with Force-Feedback

38 Experiments – Survey Results John Staton 2008Computer Science & Engineering First Question: “Were the force-feedback suggestions helpful in avoiding obstacles?” –All subjects answered “Yes” –Subject 6 (elderly gentleman with hand tremors) indicated that it didn’t necessarily help “at first”, but he caught on to the idea of the system, and once that happened, it helped “greatly”. Second Question: “Were the force-feedback suggestions helpful in approaching the goal?” –5 of the 6 subjects answered “Yes” –Subject 5, an elderly woman, answered “Somewhat” –When asked why, she indicated that, to her, what held the system back from a full “Yes” was the strength of the force-feedback effects. Final Question: “Between too forceful and not forceful enough, where would you rank the force-feedback effects?” –Answers varied –Subjects 2, 4 and 6, all males, indicated that the force-feedback suggestions could have been more forceful. –Subjects 1, 3, and 5, all females, indicated that the suggestions were too forceful, Subject 5 wrote that the joystick was driving her, and not the other way around

39 Experiments – Survey Analysis John Staton 2008Computer Science & Engineering What can we gather from this survey? –Force-feedback effect “forcefulness” Results indicate possible gender bias in response to effect force More testing needed to expand on this pattern Potential solutions: –User-controlled amount of force –Training period –Not “getting it” at first Subject 6 needed a longer period of adjustment before truly understanding the system Potential solution: –First –Time Walkthrough/Explanation/Example Training System Positive results! –All subjects found it helpful, and were genuinely excited at the potential of the system

40 Concluding Thoughts Positive early results! –Faster times, fewer collisions –Positive survey answers and excited test subjects Future work –Training system, help user “get” the concept, and determine user strength to adjust effect force –Real world issues: How to get environment data and user position (GPS?), other issues that come with applying to a real chair John Staton 2008Computer Science & Engineering


Download ppt "An Assistive Navigation Paradigm for Semi-Autonomous Wheelchairs Using Force Feedback and Goal Prediction Master’s Thesis Defense Candidate: John Staton."

Similar presentations


Ads by Google