1 Assistive Human-Machine Interfaces via Artificial Neural Networks Wei Tech Ang & Cameron N. Riviere The Robotics Institute Carnegie Mellon University Pittsburgh, PA 15213, USA Funded by: National Institute of Health Pittsburgh Foundation

2 Introduction  Objective  To create assistive human-machine interfaces to enhance positioning accuracy for patients with movement disorders  Applications  Computer mouse/joystick control  Powered wheel chair control

3 Movement Disorders  Common Types:  Pathological Tremor  any involuntary, approximately rhythmic, and roughly sinusoidal movement  Higher frequency band than voluntary motion  Myoclonic Jerk  sudden muscle contractions that can occur alone or in a sequence  Aperiodic, erratic, unpredictable  Can overlap in frequency with voluntary motion.  Sources:  Multiple Sclerosis (MS), Parkinson diseases, essential tremors etc.

4 Error Compensation Approaches  Tremor Modeling and Compensation  Frequency selective approaches  Low-pass filtering (Riley & Rosen ‘87)  Signal equalizer technique (Gonzalez et al. ‘95)  Adaptive noise canceller (Riviere et al.‘98)  Others: Viscous damping (Beringhause et al. ‘89, Rosen et al. ’95)  Non-Tremulous Error Compensation  Interfaces of sufficiently low bandwidth and input gain  keyboards with large key-pitch or/and ‘sticky operation’  Use other body part for control  Artificial Neural Network approach  simultaneously modeling and canceling both tremulous and non-tremulous types of movement disorder

5  Dynamically adjusted network architecture  Flexibility  Node decoupled extended Kalman filtering learning rule  Faster convergence over backprop Cascade Correlation Neural Networks with Kalman Filtering Output Hidden Input

6 Experimental Data  Collected from 11 test subjects with Multiple Sclerosis (MS) by Univ. of Pittsburgh  Subjects used HeadMaster Plus TM computer head control system (Prentke Romich Company, Wooster, OH)  Icon selection exercise  Move cursor from center of 14” screen (1024 x 768) to a series of circular targets(30-pixel radius)  Dwell in target for > 500ms

7 Neural Networks Training  Multiple Sclerosis (MS)  A serious progressive disease of the central nervous system, caused by malfunction in the immune system  Intention tremor  decent start, tremulous to chaotic trajectory close to target  Training targets  Phase corrected, low- passed trajectory Low-passed trajectory Raw trajectory  Start position □ End position * Target

8 Neural Networks Training  Lack of training data  Screen segments into 8 bearing sectors  N, NE, E, SE, S, SW, W, NW  Each sector we train 2 neural networks  X- & Y-direction  15 input nodes – time series of 15 data points  1 output node – compensated position of 15 th data point   10 hidden nodes N NE SE SW SE W E S 2D cursor trajectory Low-pass & phase correction Forward difference X netY net Training X target Y target X input Y input Split X & Y trajectories

9 Bearing Determination  Exploiting movement disorder characteristics of MS patient:  Intention tremor – decent start, tremulous to chaotic finish  Bearing estimation based on gradients of 1 st 14 data points using maximum likelihood criterion  81.2% success rate  Real-time interactions issues

10 Result - West  Total tests = 29  W – 9  NE – 12  S – 8  Smoother trajectories  Reached and dwelled in target 31.8% (ave) faster Target circle  NN output Raw trajectory  Start position □ End position * Target circle 10-pixel circle

11 Result - Northeast Target circle  NN output Raw trajectory  Start position □ End position * Target circle 10-pixel circle

12 Result - South Target circle  NN output Raw trajectory  Start position □ End position * Target circle 10-pixel circle

13 Result – Decent Trajectories  No over correction  18.4% faster completion time over 4 tests  NN output Raw trajectory  Start position □ End position * Target circle 10-pixel circle

14 Discussion  Demonstrated the feasibility of the ANN approach in modeling and canceling of movement disorders at assistive human- machine interface  The current experiment has not fully exploited the non-linear capability of ANN  Handicapped by the data we inherited  Strength of ANN will become apparent in more complex task scenarios and movement disorders

15 Future Works  Design our own data collection exercise for hand movement disorders  Implement real-time error compensation system  Evaluate subjects’ interaction with system  Extend the method to other type of diseases other than MS, e.g. Parkinson diseases