Cognitive Assistance in Smart Homes Sylvain Giroux and Hélène Pigot

Plan Cognitive Assistance Context, objectives, and approach From homes… Hardware level: networks, sensors, effectors … to smart homes … Middleware level: pervasive infrastructure … to smart care ! Application level: cognitive assistance & tele-monitoring Validation Usability and clinical studies Conclusion

Context People suffering of cognitive impairments in Quebec Alzheimer disease : 5.1% of people over 65 years old Head trauma : 3000 new cases each year Schizophrenia : 1% of the population In many cases, they would be able to stay at home if light assistance was provided. But healthcare resources are scarce So relatives have to take responsibility for care It then turns to an exhausting burden Hence relatives and caregivers urge for help.

Objectives Provide adapted and personalized environmental cues to Foster the autonomy of cognitively impaired people Reduce risks and hazards Pervasive computing & Tangible user interfaces Keep ensuring continuous cognitive assistance outside people’s home Mobile computing & Location-based services Help relatives and caregivers to stay in touch at distance with cognitively impaired people Our intention is NOT to replace relatives and caregivers

From homes… Smart homes are augmented environments Heterogeneous networks Sensors networks Embedded processors in devices, clothes, jewels… Information devices Networked communicating objects

DOMUS: an augmented apartment

… smart homes … Smart homes are augmented environments Heterogeneous networks Wireless : WiFi, Bluetooth, RFID, UWB... Wired : Ethernet, Electrical wires, X10, power line… Servers Full control over audio and video streams

… smart homes … Smart homes are augmented environments Sensors networks Identification and localization of objects and people Ubisense tags, UWB Smart tags (RFID)

… smart homes … Smart homes are augmented environments Embedded processors in devices and clothes Under process Perkowitz et al., Mining Models of Human Activities from the Web, WWW 2004, May 17-22, 2004, New York, NY USA.

… smart homes … Smart homes are augmented environments Information devices Fixed: smart boards, Icebox… Mobile: laptop, wireless screen, PDAs…

… smart homes … Smart homes are augmented environments Networked communicating objects Sight Hearing Smell (not yet investigated) Touch (not yet investigated) Taste (is it possible ?)

From homes to smart homes…

… to smart care Application level Cognitive assistance Tele-monitoring

Cognitive deficits Initiation inactive periods whereas the person is supposed to perform actions Memory difficulties to remember the activity to perform, the steps, the locations of the objects involved Planning difficulties to perform an appropriate sequence of actions in the right order to achieve a goal Attention shifts of attention from the activity under progress to a stimulus causing interference

Cognitive Assistance: a Demo Cognitive deficits Initiation, memory, planning, attention User interfaces Traditional GUI vs Tangible user interfaces : Transforming the whole house into a cognitive orthotic Who initiate the interactions? User vs Assistant Telemonitoring Asynchronous vs Synchronous

Usability & clinical studies Where : Domus laboratory Task : Meal preparation Whom : Intellectually handicapped people When : January 2007

Conclusion Going beyond the usual view of computing as “PC-based” Pervasive computing enables a seamless integration of assistance in residents’ everyday life Going beyond traditional human computer interfaces Tangible user interfaces help to turn the whole house into a cognitive orthotic Such smart homes can Provide cognitive assistance Foster people’s autonomy Lead to smarter care

What’s next ? Current prototype Monitoring of one activity by one person The assistance system reacts after the user “error” In a real environment, Must monitor many concurrent activities, many persons Next issues Manage concurrent activities recognition Predict the user behaviour (and ideally anticipating his “errors”) Disambiguate activities from observed events Identify opportunities for assistance

Cognitive assistance What is the available information ? Identification and location of people and objects Objects involved in an activity Primitive actions Who the user is ? personalization What is the user doing ? Activity recognition Hierarchical models Lattice-theory How to assist the user? Highligth objects Tangible user interfaces

Initiation deficits Initiation deficits leads to inactive periods whereas the person is supposed to perform actions Wandering for a long time could be attributed to an initiation deficit >> Prompt the resident

Memory deficits Difficulties to remember the activity to perform the steps of the activity the locations of the tools and materials involved in that activity. >> Show-me objects + « Follow-me » applied to objects The lamp turns off when the object is too far away. The lamp turns on to highlight the searched red book

Planning deficits Difficulties to perform an appropriate sequence of actions in the rightorder to achieve a goal. >> Prepare_coffee = {take_milk, take_cup…} >> Show to the user where to perform the next action

Usability & clinical studies Clinical studies using prototypes are in preparation and will soon be on-going at Fernand-Séguin research center, L-H Lafontaine Hospital, Montreal Dr Emmanuel Stip, psychiatrist Schizophrenia Centre de Réadaptation Estrie, Sherbrooke Head trauma Domus laboratory Intellectually handicapped people