1. Mitja Luštrek Jožef Stefan Institute Department of Intelligent Systems

2.  Environment should be ◦ Intelligent ◦ Require no special skills of the user ◦ Require minimal interaction from the user  The technology should disappear  Its advantages should remain  Defined by objectives, not methods  Interdisciplinary

3.  On the go: ◦ Wearable sensors ◦ Smart phone applications  At home: ◦ Sensors ◦ Computer controlled appliances ◦ Home automation  Living labs (Philips...)

4.  Pupulation is aging – over 65 in Europe: ◦ 17.9 % in 2007 ◦ 53.5 % in 2060  Not enough young people to care for the old  Technology must step in ◦ Assistance with activities of daily living (ADL) ◦ Detection of health problems

5.  Equip elderly with radio tags  Sensors determine tag coordinates: ◦ Installed in the appartments ◦ Included in tags and portable device outdoors  Detect falls and other health problems Portable device Body tags Sensors in the appartment

6.  Equip elderly with radio tags  Sensors determine tag coordinates: ◦ Installed in the appartments ◦ Included in tags and portable device outdoors  Detect falls and other health problems Intelligence

7.  Radio tags and sensors to be developed in the project ◦ Distance to tag – time needed for signal to travel from tag to sensor ◦ Direction of tag – angle of arrival of the signal  Expected standard deviation of noise: ◦ ~5 cm when stationary (Ubisense × 1) ◦ ~10 cm when moving (Ubisense × 2)

8.  6 infrared cameras  12 reflective markers on the body  Multiple cameras see a marker ⇒ location can be computed  Standard deviation of noise: ◦ ~1 mm  Add more noise to simulate radio hardware

9.  815 recordings: ◦ Walking ◦ Sitting ◦ Lying ◦ Falling – 11 types ◦ Lying down ◦ Sitting down ◦ Health problems:  Limping  Hemiplegia (stroke)  Parkinson’s disease  Dizziness  Epilepsy Six basic activities

10. Input: sequence of snapshots of tags (each consisting of coordinates of all tags) Attributes Output: posture/activity (walking, lying...) Class Machine learning

11.  Manually segment and label recordings  Compute attributes for each snapshot  Concatenate to create attribute vectors

12.  Z coordinates of tags  Absolute, z velocities of tags  Absolute, z distances between tags Attributes – angles

13.  All coordinates of tags  Velocities of tags (absolute, direction) One coordinate system per snapshot One coordinate system per 1-second interval Two options Two more options: each coordinate system can use reference z axis

14.  Attributes : ◦ Reference coordinate system ◦ Angles ◦ One-per-snapshot body coordinate system ◦ One-per-interval body coordinate system ◦ One-per-snapshot body coordinate system with reference z ◦ One-per-interval body coordinate system with reference z  Machine learning algorithms: ◦ C4.5 decision trees ◦ RIPPER decision rules ◦ Naive Bayes ◦ 1-nearest neighbor ◦ 3-nearest neighbor ◦ 5-nearest neighbor ◦ 10-nearest neighbor ◦ SVM ◦ Random forest ◦ Bagging ◦ Adaboost M1 boosting

15.  Attributes: reference coordinate system  Machine learning algorithms: ◦ SVM ◦ Random forest ◦ Bagging ◦ Adaboost M1 boosting ◦ 3-nearest neighbor  Winner: ◦ Reference coordinate system + angles ◦ SVM

16. Sitting down, no noise Falling, Ubisense × 1 noise

17.  Tag placement ◦ More tags ⇒ better performance ◦ More tags ⇒ worse user acceptance  Noise level ◦ We are only estimating noise of the radio hardware

18. 12 11 10 9 8 7 6 5 4 3 2 1 L R

20.  We can recognize walking  Can we recognize abnormal walking?  Gait (way of walking) important to physicians  Used to recognize health problems in clinical setting

21.  Support (foot on the ground), swing (foot off the ground) and step (support + swing) times  Double support time (both feet on the ground)  Step length and width  Maximal distance of the foot from the ground  Ankle, knee and hip angles upon touching the ground  Knee angle when the ankle of the leg on the ground is directly below the hip and knee angle of the opposite leg at that time  Minimal and maximal knee and hip angles, the angle of the torso with respect to the ground, and the range for each  Hip and shoulder sway (the difference between the extreme left and right deviation from the line of walking)

22.  X, y coordinates of ankles  L: lowest distance travelled (standing still)  H: highest distance travelled (moving)

23.  Normal: ◦ Completely normal ◦ With a burden  Abnormal: ◦ Limping ◦ Hemiplegia (stroke) ◦ Parkinson’s disease ◦ Dizziness

26.  In-depth analysis of activities other than walking  Attributes other than walking signature  Macroscopic movement (about the appartment)