Novel Smart Sensor Glove for Arthritis Rehabilitation Names: Alon Tsilik, Gal Halevi, Boaz Katzover 21/05/2014

Overview RA – What is it? Statistics Detection and detection issues Main goal and motivations Current gloves systems Tyndell/uu glove Tyndell/uu glove software Gloves systems in VR and gaming

RA RA affects the synovial tissue that lubricates the joints. This condition affects bones, joints and muscles. Stiffness, swelling and Lack of Flexibility are common symptoms.

Statistics In 2010, RA affected 500,000 of the UK populations. 4-10 Patients lose their jobs in 5 years of diagnosis. the most common condition for which people receive Disability Living Allowance

Current detection methods RA is currently diagnosed by clinicians and therapists using x-rays and manual evaluation methods. Current techniques HAQ, Grip Strength Dynamometer, Goniometer, Tape measure

Current detection issues Outcomes are easily influenced by clinicians training and experience. All measurements are recorded in handwritten form. Joint stiffness is currently unmeasured. Current process is time consuming.

Main goal This project focuses on an accurate hand measurement tool to detect hand Range of Motion (ROM), joint stiffness and to match each specific patient the treatment that is the best for him. The motivation for developing a smart sensor glove: More accurate diagnostic . Measure joint angles in a dynamic way. Less time spend on diagnostic. Patient can perform tasks at home.

Gloves Systems Gloves structure Sensors on a cotton glove Gloves roles 1.Measuring fingers movement 2.Measuring fingers range of movement Disadvantages 1.Physically fits to a specific size 2.Adjust calibration for each patient

Popular Data Gloves Systems currently available Cyber glove 22 bend sensor virtual bend calibration calibration calibration 5DT 14 optic fiber sensors X-IST 5 bend sensors + 5 pressure sensors or 15 bend sensors + double axis sensor cotton + silk

Tyndell/UU 20 bend sensors 16 triple axis sensors 11 force sensors Flexible PCB No calibration requires Fits with both hands by flipping the PCB wireless data transformation Visual image by program

Rehabilitative System Design - Overview Biomechanical model – Model of the human hand illustrating bones & joints of interest. Sensors choice – Sensor that have been picked, where they are placed and why. Hardware spec – Flexible PCB ,Zigbee. Patient’s tasks – Preconditions and how it will be done.

Biomechanical model

Sensors choice Main objective The glove integrates a total number of 47 sensors mainly to capture hand and finger motion. Main objective Measurement of joint range of the hand, including: Extension Flexion Adduction/Abduction of the MCP, PIP and DIP joints of the fingers and thumb in degrees.

Bend sensors The bend sensors measures the finger flexion, joint movement, splaying of fingers and wrist rotation. The position of the bend sensors cross over the DIP, PIP and MCP joints, between the fingers and an additional bend sensor on the lateral side of the wrist.

Kapandji index measurement

Force sensors The 11 force sensors are mainly used for the Kapandji index measurement (also known as opposition scale). The force sensors are located on the tip of the index, middle, ring and little fingers. In addition there are 3 more force sensors along the index finger and 4 more on the little finger.

tri-axial accelerometers The glove integrates 16 3-axial accelerometers. One on each of the finger’s phalanx and one on the palm. They provide additional useful information on the rotation motion of the hand such as shaking/vibration motion.

Flexible Circuit PCB What is a PCB? Contains 8 layers with overall thickness of 0.45 mm. Meander type structure. How data is transferred? Zigbee.

Patients tasks– First steps Before talking about the UI we need to explain how the treatment with the help of the glove will carry out. The software system manages objective routines defined by a clinician and performed by the patient at home at set times thorough the day.

Patients tasks 1. Attend a clinic session - Each patient has to undergo some basic training. 2. During this visit the results are recorded and used as a comparison to future objectives completed at home. 3. The objectives are typically completed first thing in the morning when the patient arises, at lunchtime and in the evening.

Software system and task’s results Software system calculates angular and velocity data generated from the glove when the patient is performing an objective routine. Completed routines are uploaded to a cloud system for immediate access by the clinician for analysis and patient feedback.

The Analysis of Data The software as an assistive tool for analyzing data. The benefit in the use of sensors. The software as a measuring tool. The control system (Clinician side). Patient side interface.

The Analysis of Data An objective consists of 12 to 20 repetitions. Each one is analyzed for time taken during flexion and extension, minimum and maximum ROM and minimum and maximum velocity. Analysis of angular and velocity percentage change throughout each flexion and extension portion of a repetition provides dynamic analysis of joint movement and an indicator of change between repetitions Information is used by the clinician as an assistive tool to detect change in movement over time for each patient.

The use of sensors – The Benefit Multiple accelerometers, bend sensors and force sensors eliminates the need for calibration and offers accessibility to users with limited ROM. Accelerometers - improve accuracy of both static and dynamic finger joint movement. Bend sensors - work in collaboration with accelerometers to accurately measure finger joint angle, velocity and acceleration. Force sensors - provide measurement for opposition testing of the thumb to the fingers and palm, such as Kapandji scoring.

The software as a measuring tool Minimal and maximum flexion and extension ranges of each digit. Adduction and abduction range for each digit. Thumb-index finger web space. Amount of joint stiffness for each digit. Recording, storage and analysis of patient data. Analysis of historical patient movement

The Control System Live movement data captured from the data glove is displayed numerically and graphically. Numeric data includes bend sensor, force sensor and three axes from each accelerometer. A 3D hand graphically represents numerical data captured from the glove. The hand moves synchronously with all glove sensors.

How is the data received from the glove displayed by the controlling software? Objective summary window Objective detail window Bend sensor data is converted to degrees and verified against accelerometer calculations. Completed objective routines are shown in the objective summary window (a). An objective consists of many repetitions that are displayed in the objective detail window(b). One repetition is achieved once the joint under investigation is opened and closed. Repetitions consist of velocity (c) and angular movement (d). Charts show repetitions 1-4 in overlay mode allowing instant comparison of repetitions. Velocity movement Angular movement

Dynamic Movement Characterization Using A Data Glove Flexion and extension movement is sigmoidal shaped and an open-closed hand movement produces a Gaussian shaped curve. Angular velocity and angular acceleration become inaccurate..

Summary panel for each group of repetitions within the reference point Detailed breakdown of each repetition for each reference point sensor The comparison objective summary panel Fig. 12(A) is a summary panel for each group of repetitions within the reference point. Each summary displays the maximum degree change and overall percentage change for total flexion and extension movement. Fig. 12(B) provides a detailed breakdown of each repetition for each reference point sensor. It displays peak flexion and extension angular and percentage change, together with peak velocity change. Fig. 12(C) displays the comparison objective summary panel. It too displays maximum angular and velocity degree change for flexion and extension movement. Fig 11(D) displays a detailed breakdown for each glove sensor included in the selected objective summary. Detailed breakdown for each glove sensor included in the selected objective summary

Patient Interface Provides live feedback of glove sensor movement and progress of the objective routine performed by the patient. The repetition progress panel provides real-time indicators on the status of each flexion and extension action for each sensor included in the objective routine. Hand movement performed by the patient is displayed as a 3D animated hand

Conclusions To date, Clinicians use manual, time consuming techniques to quantify hand limitations. Current methods have the capability to detect hand ROM, with some issues. The proposed system will provide an automatic hand ROM measuring tool capable of measuring joint movement, joint stiffness and comparison analysis of historical movement data.

Gloves in VR and Gaming Cyber gloves can be used in the virtual reality and gaming industry In the past there were wired gloves that can be used even as a mouse Today most gloves are wireless or Bluetooth technology Most gloves are also can provide there position in the 3D space together with the specific hand/figure position

Cyber Glove

CyberGlove 3 provides up to 22 high-accuracy joint-angle measurements Sensor resolution: <1 degree up to 120 records/sec 30 meter radius from Wi-Fi source Wireless technology Operating system Windows 7 Interface: Wi-Fi, USB, micro SD card

CyberGlove 3 8-sensor model features : 2 bend sensors on each finger 4 abduction sensors Sensors measuring thumb crossover, palm arch, wrist flexion, and wrist abduction. Each sensor is extremely thin and flexible being virtually undetectable in the lightweight elastic glove. Designed for Comfort and Functionality

System extras Two batteries and an external battery charger Provisions for InterSense, Polhemus, and Ascension six degrees of freedom (DOF) tracking sensors are available for the glove wristband. That supported in VirtualHand software.

Of VR Examples

The virtual glove in gaming

P5 Based upon proprietary bend sensor and remote tracking Provides users total intuitive interaction with 3D and virtual environments, such as games, websites and educational software. Retail price of $39.00

Specs Lightweight, ergonomic design for easy, intuitive play. Weighs just 125 grams. 5 independent finger measurements optical tracking system 6 degrees of tracking (X, Y, Z, Yaw, Pitch and Roll) to ensure realistic movement Bend-sensor and optical-tracking technology to provide true-to-life mobility Easy, plug-and-play setup – plugs right into the USB port of your PC Infrared control receptor

P5 Glove

Have a pleasant evening Thanks for listening… Have a pleasant evening