1 cse 590 hk assisted cognition what’s on the market

2

3 Elliot Cole, Ph.D. Founder, Institute for Cognitive Prosthetics Dr. Cole is one of the few computer scientists worldwide working in the field of cognitive rehabilitation. His research at Syracuse University, Drexel University, University of Pennsylvania and ICP, has focused on human- computer interaction, information systems and the design and application of user -friendly assistive technology and technology transfer

4 What is a Cognitive Prosthetic? A cognitive prosthetic is assistive technology that helps a person with cognitive deficits function more independently in certain tasks. A cognitive prosthetic is a compensatory strategy that directly assists the individual in performing daily activities. It helps a person to maximize their strengths and abilities to compensate for their deficits. A cognitive prosthetic is computer-based, meaning the system uses a desktop or laptop computer. It may also include additional components such as a cell phone, pager, or digital camera. The cognitive prosthetic is set up in the client's home or office from the start, where it is available for them to use.

5 Cont. A cognitive prosthetic is always developed and implemented through a therapeutic process that includes an occupational or speech therapist, the client, and computer programmers. A cognitive prosthetic is custom- designed for each individual client. This enables: oSpecific adaptation to the individual, the activities and the environment oFaster acquisition of skills. Our client's are always able to start using the system from the first day of therapy. While it is being used by the client, a cognitive prosthetic captures data about how and when it is being used. This data is very helpful to the therapist to further customize the system to meet the client's needs.

6 Ex: CellMinder Difficulty initiating activities KT lived at home with her mother, spending her days going to therapy and working toward returning to finish her degree in athletic training. Her college career had been on hold since being hit by a drunk driver in her junior year, while she and her mother dedicated themselves to KT's rehabilitation. KT had severe short-term memory deficits. She often forgot what she was doing in the middle of an activity and was unable to recall anything she had done during a given time period. She relied on her mother to cue her to initiate and complete all of her activities.

7 Keeping a schedule First, ICP provided KT with a computer- based scheduling system and taught her to use it. She printed her schedule and carried it with her throughout the day. When her mom prompted her, KT was able to pull out her schedule and identify what she should be doing. In addition, with her mother's prompts, she kept notes on her activities on this schedule, which were later entered into the computer. This strategy enabled her to recall the things she had done that day. However, KT still required cue from her mother to look at her schedule and to take notes so that she could later remember what she had done.

8 The CellMinder System To allow KT to be independent of her mother's cueing, ICP added a new feature to her scheduling system - CellMinder. CellMinder can call KT on her cell phone at the times she specified, and provide her with prompts and reminders. For example, using her computer KT scheduled a CellMinder call to take place at 12:30, shortly before her ride would taking her to a local rehab hospital, where she is doing a volunteer rotation. At 12:30 she received a call on her cell phone reminding her "Howard will be picking you up in 10 minutes - get ready. Bring your notebook and the phone". Moving on toward independence CellMinder was the link that helped KT reach greater independence. With this system in place, KT was able to move into an apartment with her brother and begin an internship program at a local university.

9 MultiMedia Album "Lost in a land of murkiness" MV had significant memory problems, which affected her deeply when it came to her family. She often could not recall how old her grandchildren were or what they did together the last time she saw them. Holding a conversation was difficult for MV. She talked about being in "a land of murkiness, where there was a lot of mud and you can't define things." She struggled to define who she was and grasped for a connection to the rest of the world. Holding onto a voice As MV struggled to hold onto information and connect to her family, she tried using Polaroid photographs, with notes written on them. But MV had difficulty with visual processing, so she often could not recognize the person in the picture nor associate that face with a memory

10 ICP focuses on strengths, and realized that MV had very good auditory processing. On the phone, she never hesitated to think about who you were, and could easily connect to memories about the person she was speaking to. ICP realized that this strength, coupled with technology designed for MV's particular needs, could help her "connect". Multimedia Album The program that ICP designed enabled MV to capture an audio recording at the same time that she took a picture. She could then transfer these into her computer, title them, and add notes. MV could look through the photos and play the sound that was recorded at the time the photo was taken.

11

12 How was your Christmas? One day, to evaluate whether or not she was benefiting from this system, we asked her how her Christmas was. She struggled for a moment, searching in her mind for anything that would help her remember Christmas - had it even happened? she wondered. Then she turned to her computer and her album. "Oh, I can tell you how my Christmas was", and she proceeded to show us her photos with sound. She told wonderful stories about the day she spent with her two young grandchildren, recalling memories with ease. Judging by the loving grandmotherly affection on her face, she was feeling connected to Christmas and her family

13

14

15

16 PEAT adjusts the schedule if the user delays the start or stop of Bank task or changes its duration. If Bank takes 20 minutes instead of 10, there won't be enough time for Shopping before the Movie. Consequently, Shopping is delayed until after the Movie. This in turn pushes Exercise past its 5:30 deadline. When Peat detects the problem that Exercise will miss its deadline, it alerts the user as shown in the second picture above and provides options for explaining and fixing the problem. Selecting "explain" tells the user that Exercise missed the deadline by 5 minutes. The other options help solve the problem. Selecting "shrink duration" solves the problem by trimming 5 minutes from the Exercise task, which now can be finished on time (shown in last picture above).

17