Ubiquitous Computing and Augmented Realities Material from Authors of Human Computer Interaction Alan Dix, et al

Overview Breaking the tradition of the computer box ubiquitous computing virtual reality augmented reality information visualization

Introduction Ubiquitous computing (pervasive computing) single user --> groups and larger organizations traditional desk top --> computational power in the environment user search out computer --> interface locates and serves user

Three challenges Defining appropriate physical interaction experience Discovering general application features Theories for designing and evaluation human experience with ubicomp

Challenge 1: Input Textual input --> pointing devices for variety of data types Explicit input --> implicit input (natural interactions with environment) tipping a PDA to move between pages walking into a space announces one’s presence Better support for natural human communication (handwriting, gestures, etc.)

Challenge 1: Input Large scale touch interactive surfaces (capacitive coupling) RFID sensors (security tags) Infrared range finders Incorporation into artifacts

Challenge 1: Output Concerns must address technology and aesthetics Size of displays Multiple displays Multiple modalities Ambient forms of communication

Challenge 2: Output Size of displays Small (inch)- mobile devices Medium (foot)- traditional desktop displays Large (yard) multiple low-resolution projected displays display wall coordinate interaction between them

Challenge 1: Output Ambient displays peripheral output information is medium to low priority aesthetic appeal Examples: Dangling String (detects network traffic) Audio Aura (auditory cues eg. email) Kandinsky (collage of images from keywords)

Challenge 1: Seamless integration Merge computation artifacts with world of physical artifacts Augmented reality Use objects in physical world to manipulate electronic artifacts

Challenge 2: Application themes To make computing disappear, need a combination of a large range of services Context-aware computing Capture and record experiences

Challenge 2: Context Aware Computing Location aware appliances simple context - user location (where and who) provide services like automatic cell forwarding updated maps of user locations GPS navigation systems tour guide systems need more context When - relative changes of time, vary from routine What - perceiving and interpreting human activity (Cookies) Why

Challenge 2: Automated capture and access Augment inefficiency of human record taking Record multiple streams of activity Old idea : Vannevar Bush and Memex to store artifacts we come in contact with in everyday lives and the associations that we create between them Applications Meeting capture at Xerox Parc

Understanding interaction Everyday computing Continuous interaction orchestrates daily routines, communicate with others, and manages information Focus on activities (eg. writing), not tasks Problems: no clear start or end interruptions concurrent activities

Understanding Interaction In Ubiquitous Computing Integrate numerous devices in one setting Need knowledge in the world (rather than in the head) Need models that consider nature of relationship between internal cognitive processes and the outside world Three main theories serve as guides Activity Theory Situated Action Distributed Cognition

Challenge 3: Evaluation Challenges Develop assessment and evaluation techniques for ubicomp Predict how novel technologies serve human need Observe authentic use and co-evolution of human activities and novel technologies

Challenge 3: Evaluation Challenges Little research to date on this topic understanding is on structuring activities and tasks not clear when we move away from structured work ubicomp systems not yet reliable and robust to support continuous activity --> how to evaluate? Essentially we have living laboratories

Virtual and Augmented Reality VR refers to computer-generated simulation of a world or a subset of it immersive (cave) desktop (command and control situations) augmented reality where virtuality and reality meet

Information and Data Visualization Scientific and technical data virtual reality 3D maps Structured information Hierarchies and networks Time and interactivity Use of color to determine patterns