1. Informing the Design of an Automated Wayfinding System for Individuals with Cognitive Impairments
Alan L. Liu, Harlan Hile, Gaetano Borriello
Computer Science & Engineering
University of Washington, USA
Pat A. Brown, Mark Harniss, Kurt Johnson
Rehabilitation Medicine
Henry Kautz
Computer Science
University of Rochester, USA

2. Adapting to changes in routes
Wayfinding Concerns Limit Independence of Persons with Cognitive Disabilities
Learning new routes
training from job coaches / caregivers
burden on caregivers and Access transit
Adapting to changes in routes
missing a bus
encountering an unexpected barrier
Stress of being lost lowers meta-cognition
can become overwhelmed and “stuck”

3. Convergence of Technologies
Location-sensing technologies
Geographic information
Mobile computing
Online photo collections
Intelligent wayfinding system
User modeling

4. Our Earlier Results on Wayfinding
Wide variation in user preference and expectation of directions
modality (image, audio, text)
complexity (e.g., direction length)
visual distinctiveness
timing
Direction ambiguity problematic
nonstandard turns, especially outdoors
Attention demand problematic
being distracted or becoming unaware of surroundings `
Liu et al. ASSETS 2006, PervasiveHealth 2009

5. Needs for More Intelligent Wayfinding Assistance
Map and/or turn representation not suitable for all
Using good landmarks can beneficially augment directions
Current systems employ static navigation strategies
Ignore health conditions, error behavior, desired detail, safety concerns, place familiarity, and other individual differences
Need to support customization and adaptation

6. Contributions Automatic landmark selection system
Markov-Decision Process based system for optimizing routes and cues
User study to establish further customization / adaptation needs

7. Landmark selection system
Leverage collections of geo-tagged photos
Select image of landmark based on proximity to user location and near user orientation
Heuristic: landmarks useful for wayfinding tend to be more popular
Further enhancements and alternative views are ongoing

8. Modeling Wayfinding with Markov Decision Processes
MDPs have been used in robot navigation planning
MDP defined by states, actions, transitions, costs
state: user position and history of recent events
action: direction cue
transition: next states and their likelihood given action
cost: notion of effort or preference for route
A solution to a MDP is a policy that minimizes expected costs
The MDP framework can support adaptation using reinforcement learning

9. Modeling Wayfinding using Markov Decision Processes
Observation
Next state transitions
State
State
(location,
orientation,
recent history)
System Action
(direction cue)
Transition
probabilities
State
Pr(St+1|St,A)
State
Cost / Reward

10. Modeling customization
Alter costs based on individual preferences
Preference of direction types
Route concerns (distance, accessibility, complexity)
Alter transition probabilities based on user ability
Does user move in correct direction in response to cue?
Study: Does system customization affect user experience?
Model 1: Naïve model: Use any direction that guides user along shortest path
Model 2: Landmark cues more costly (require more cognitive resources) than turn-based directions, but can take user farther with each cue
Policy will use landmarks for long segments, turns for short segments

11. Location Wizard-of-Oz interface

12. Wayfinding application user interface
Image, audio (TTS), text, vibration output
Repeat audio and Help input
Help button requests alternative direction (and possibly route)

13. 7 participants (6 male) with cognitive impairments
Age ranging from (mean 34)
Due to limited size of study, focus on notable user reaction to wayfinding experience

14. Results Participants split on usefulness of landmark directions
Participants 1, 2, 3*, 5 expressed affinity for landmarks
Participants 4, 6, 7 preferred turn-based directions
Difference in location between photo and participant problematic
P1: When you have an actual picture of what's in front of the person, that's excellent.
P2: The towers [of the Campanile] were behind trees, but I trusted myself to go forward.

15. Results cont'd
Some participants indifferent to unfamiliar landmark names, others distracted by them
Suggestions included only referring to landmark name when it was destination, and in other important situations
Compound directions, meant to give user knowledge of upcoming turns, caused confusion
Participants often thought follow-up turn direction was separate
Participant 6 sometimes reversed the directions

16. Results cont'd Attentiveness
Most participants stated that they paid attention to surroundings, while others commented that familiarity would change their interaction mode
I was thinking for a while, maybe I should stick this in my pocket and react when the thing goes off. I didn't do it because [I thought], “Well, by the time I take it out, will I miss the instruction?” but obviously not, the instruction stayed on which was good, so I should have.

17. Current steps Learning user-specific cost function
Use of help button
Time required to understand cue
Experience sampling
Learning transition probabilities
Predict success or failure of cue based on cue type and features of the state
Better landmark-based images.

18. Learning Transition Probabilities by Linear Regression
Difficulty incidence rate
Model's predicted difficulty

19. Better Landmark-Based Images

20. Conclusion
Wayfinding support for individuals with cognitive disabilities should be customized – no one strategy is optimal for all users
The Markov Decision Process framework provides a way to generate custom strategies that take into account user's needs (abilities) and preferences
Effectiveness of a strategy is inseparable from details of presentation
Provided initial results on learning custom models (success probabilities)

21. Thank You Alan L. Liu, Harlan Hile, Gaetano Borriello
Computer Science & Engineering
University of Washington, USA
Pat A. Brown, Mark Harniss, Kurt Johnson
Rehabilitation Medicine
Henry Kautz
Computer Science
University of Rochester, USA
This material is based upon work supported by the National Institute on Disability and Rehabilitation Research (NIDRR) Grant #H133A031739, Microsoft Research Intelligent Systems for Assisted Cognition Award, and Nokia Research.