Project ACCESS Henry Kautz, Dieter Fox, Gaetano Boriello (UW CSE) Don Patterson (UW / UC Irvine) Kurt Johnson, Pat Brown, Mark Harniss (UW Rehabilitation Medicine)

The Need: Community Access for the Cognitively Disabled

Problems in Using Public Transportation Learning bus routes and numbers Transfers, complex plans Recovering from mistakes

Current GPS Navigation Devices  Designed for drivers, not bus riders! oShould I get on this bus? oIs my stop next? oWhat do I do if I miss my stop?  Destination manually entered oHigh cognitive overhead  Device decides which route is “best” oFamiliar route better than shorter one  Catastrophic failure when signal is lost oFrequent “dead zones” in urban areas

Idea  User carries GPS cell phone  System infers user’s state oWalking? Getting on a bus?  System learns about user oImportant places, routes  Breaks from routine = user may be confused or lost oOffer help oCall caregiver

Transportation Routines BA  Goal: intended destination oWorkplace, home, friends, restaurants, …  Trip segments: oHome to Bus stop A on Foot oBus stop A to Bus stop B on Bus oBus stop B to workplace on Foot WorkplaceHome

GPS reading z k-1 zkzk Edge, velocity, position x k-1 xkxk  k-1 kk Data (edge) association Time k-1 Time k m k-1 mkmk Transportation mode t k-1 tktk Trip segment g k-1 gkgk Goal Cognitive mode { routine, novel, error } Dynamic Bayesian Network c k-1 ckck

Predicting Goal and Path Predicted goal Predicted path

Error Detection: Missed Bus Stop

 GPS camera-phone  “Knocks” when there is an opportunity to help oCan I guide you to a likely destination? oI think you made a mistake! oThis place seems important – would you photograph it? Prototype: Opportunity Knocks

Status  User needs study  Algorithms for learning and predicting transportation behavior  Proof of concept prototype  Current work: user interface studies oModality: Audio, Graphics, Tactile,... oGuidance strategies: Landmarks, User frame of reference, Maps,... oDecision-theoretic control: when to interrupt?

More Information Opportunity Knocks: a System to Provide Cognitive Assistance with Transportation Services D. J. Patterson, L. Liao, K. Gajos, M. Collier, N. Livic, K. Olson, S. Wang, D. Fox, and H. Kautz. Sixth International Conference on Ubiquitous Computing, Nottingham, England,