Presentation is loading. Please wait.

Presentation is loading. Please wait.


Similar presentations

Presentation on theme: "KINECT REHABILITATION"— Presentation transcript:

Stroke Therapy Research Kathryn LaBelle kathryn labelle kinect stroke therapy research with prof streigel started spring 2011 for honors thesis research supported by a contribution by tom meurer

2 RESEARCH TOPIC Can the Kinect’s joint-tracking capability be used in clinical and in-home stroke rehabilitation tools?

3 OUTLINE Background Potential of Kinect in Rehabilitation
Stroke Therapy Kinect Potential of Kinect in Rehabilitation Research Questions Software Data Gathering Data Analysis Conclusions

4 STROKE THERAPY Stroke survivors can experience:
restricted movement loss of sense of balance decreased strength Regained through physical therapy balance exercises range of motion activities coordination practice

5 MICROSOFT KINECT Developed for the Xbox 360 gaming console
Tracks your movements: you are the controller Sensors Depth Camera and Sensors RGB Camera Microphone array Motorized base Released November 2010 Depth Camera and Sensors infra red projector monochrome CMOS sensor creates depth image RGB Camera on screen display facial identification Microphone array four microphones speech recognition acoustic source localization Motorized base tilts to adjust view vertically

6 DEPTH IMAGING Infra-red projector shines grid of light on the scene, encoded with data. Light bounces off objects in the scene. Kinect light sensors receive reflected light. By analyzing time of flight and distoritions in the encoded data, the Kinect makes a depth map of the scene.

Input: depth map Machine learning algorithm Collected recordings of people using the Kinect Joint positions marked by hand Algorithm was fed this “training” data and learned how to correctly identify joints from a depth image Output: x, y, z joint positions This is Microsoft’s implementation

Clinical applications: assess patients’ performance track patients’ progress pinpoint areas for improvement At-home exercise aids: provides constructive feedback to patients give encourgement and motivation generate summary reports for doctors

9 RESEARCH QUESTIONS What SDKs and drivers are available for use with a PC? What type of information can be obtained? What is the quality of the joint data obtained from the Kinect? Sampling rates Consistency How resilient is the Kinect’s joint data and performance to variation in testing conditions? What functionality could be provided in a stroke therapy application that uses the Kinect? What SDKs and drivers are available for using the Kinect with a PC? What capabilities do these provide and which of them will be most useful for stroke therapy? What type of data and information can be obtained from the Kinect using these SDKs and APIs? What is the quality of the joint data obtained from the Kinect? What is the sampling rate and consistency of this information? How resilient is the Kinect’s joint data and performance to variables such as distance, body type, clothing, number of subjects, and amount of movement? What functionality could be provided in a stroke therapy application that uses the Kinect, and what are the limitations of such a program?

10 SDK COMPARISON OpenNI Microsoft Raw depth and image data Yes
Joint position tracking Save raw data stream to disk No Joint tracking without calibration Easy installation Number of joints available 15 20 Quality of documentation Adequate Excellent Even before Microsoft released their SDK in June 2011, there were many open source sdks available. openni suited our needs best so i developed some software using that later microsoft’s sdk came out and i used that too

11 SOFTWARE DEVELOPED Display depth video and skeleton
Joint positions and instantaneous frames per second written to file Balance board integration Record depth stream to file Obtain joint positions from recording

12 DATA GATHERING to gather data for analysis, used six subjects and did three variations of sit to stand exercises explain sit to stand subject sat in a chair directly facing the kinect with their feet on a balance board. the tv displayed the running program so they could see the depth map and skeleton on the screen

13 DATA ANALYSIS Sampling rates of joint position data
Identifying phases of movement from joint positions Consistency and stability of joint positions

14 SAMPLING RATE OpenNI Microsoft Average Frame Rate (fps) 25.0 19.6
Std Deviation (between trials) 5.8 2.3 Minimum 9.8 14.1 Maximum 30.0 23.7 OpenNI a little faster Microsoft more consistent Both more than adequate even at their worst

sit to stand exercise easy to pick out the parts of the exercise note head bob

16 Standard Deviation of Joint Positions while Subject is Motionless
DATA STABILITY Standard Deviation of Joint Positions while Subject is Motionless Joint OpenNI (cm) Microsoft (cm) Head 0.34 1.8 Hip 0.42 1.2 Knee 0.70 1.5

17 DATA STABILITY: Assisted Tests
Clinical therapy often involves an assistant supporting a patient while he performs exercises Test procedure: subject begins by sitting alone assistant joins, putting hands on subject’s shoulders subject stands up conducted tests with many variations like type of exercise and distance, but for the sake of time i’ll focus on the effects of adding a person in the scene

18 DATA STABILITY: Assisted Tests
one type of irregularity observed occurred as assistant entered the scene probably because the algorithm was determining which parts of the depth image belonged to which person not a huge problem because they could be easily identified as errors and corrected: it is clear where the true value lies

19 DATA STABILITY: Assisted Tests
another type of variation observed was skeleton merging after the patient stood, the algorithm identified the people as one and produced a strange-looking skeleton (ex: identify assistant’s head as if it were the patient’s shoulder) incorrect positions, and data was less stable more of a problem because it is not certain where the true value of the person’s head position lies

20 DATA STABILITY: Assisted Tests
It is at least possible to distinguish between normal deviation and deviation with skeleton merging More investigation needs to be done to iron out this obstacle since it could become a real problem in a clinical setting

21 CONCLUSIONS OpenNI Framework and Microsoft SDK for Windows are best tools to use Can provide significant functionality in a joint-tracking application track and record joint positions in three dimensions display image of tracked joints in real time integrate Kinect with the Wii balance board Sampling rate exceeds acceptable level Phases of movement are easily identifiable from graphs of joint positions Joint position stability is more than adequate with one subject in view Skeleton merging could pose a problem for clinical use of Kinect

22 FUTURE WORK Deeper investigation into assisted exercises
Different types of exercises Position the assistant differently Determine conditions causing skeleton merging Further development of software Investigate applications in other fields of physical therapy -conditions causing skeleton merging: relative heights of assistant and patient -further development of software: example -- guided in-home exercises, with constructive feedback, variation in difficulty levels, performance reports for physicians -other physical therapy: athletic injuries, chronic pain, neurological diseases (parkinsons, multiple sclerosis, cerebral palsy)

23 Questions?


Similar presentations

Ads by Google