KAIST, AIM Lab Ubiquitous Computing Ubiquitous computing is the method of enhancing computer use by making many computers available throughout the physical environment, but making them effectively invisible to the user ( Mark Weiser, Xerox PARC)
KAIST, AIM Lab Ubiquitous Computing We are surrounded by computing Computing and processing is embedded into everyday devices There are many computers/processors per person Information access and communication is possible virtually everywhere Dedicated computing devices – information appliances – are all around us Devices can be connected and networked What gets us here?
KAIST, AIM Lab Ubiquitous Computing Mark Weiser: Computers enter everyday life Help people with everyday tasks in the office and at home (at any time, any place) A good tool is an invisible tool. By invisible, I mean that the tool does not intrude on your consciousness; you focus on the task, not the tool. [Weiser 94]
KAIST, AIM Lab HCI themes with U-Life Three past interaction themes: Natural Interfaces Context-Aware Interaction Automated Capture & Access to Life Experiences New interaction theme proposed: Everyday Computing
KAIST, AIM Lab Natural Interfaces Forms of Natural Interfaces Speech, gestures handwriting (pen-based/free-form) Issues Encountered Need a way to represent information with new interface Error-prone, even humans can ’ t perfectly read handwriting
KAIST, AIM Lab Context-Aware Interaction What is appropriate context to use? Current – user and location Future – time, history, other users How to represent this context? Incorporate hierarchal info and relations Truly Ubiquitous? Limitation of many technologies.
KAIST, AIM Lab Everyday Computing: Things to be Considered No clear beginning & end to all activities Interruption is expected Multiple activities operate concurrently Time is important discriminator Associative models of information
KAIST, AIM Lab U-Life Web servers Web browsers Mobile browsers Electronic servers ?
KAIST, AIM Lab Change of UI Paradigm Single Screen-based UI Interact with a number of U-devices (distributed + interconnected) Highly personal and mobile appliances Systems that are integrated in everyday environment Ubiquitous Computin g
KAIST, AIM Lab User interface in U-C Requirements Distribution of UI All U-devices are distributed. Implicit HCI To reduce the need for explicit HCI To let explicit interfaces virtually disappear into the environment Awareness of the situation, the environment and the aims of the user Being noticed only when needed
KAIST, AIM Lab User interface in U-C Current Interaction Explicit HCI By command-line By direct manipulation using a GUI, gesture, or speech input Interaction in U-C Implicit HCI It allows the computer to interpret the user’s behavior and the surrounding situation and use this information as input
KAIST, AIM Lab What is different from traditional ‘HCI’ and ‘HCI in UbiComp’ ? Output modalities not just an audio visual channel all senses! Input modalities more than pressing buttons and moving an object in two dimensions Distribution – physical and conceptual Magic beyond the screen … it is a vivid physical relationship
KAIST, AIM Lab Development Process? Research Approach? Not anymore designing and programming a GUI Interdisciplinary teams – ethnography, design, CS It is about creating an experience by Understanding the interaction and process Designing and constructing a set of devices and an environment Implement the human-information interface based on the created devices/environment Test it yourself Test it with users … go back an refine the hardware and start again
KAIST, AIM Lab Prototypes Functional prototypes are essential to learn, understand and experience how to interact with the ubiquitous computer From the idea to knowledge Prototyping has been central to hallmark research in the area (e.g. ParcTab, ActiveBadge) Learning occurs when along the prototyping process as well as in use Evaluation Functional prototypes are the means for evaluation “Confronting” real people – already with version 0.001 Deployment in a living lab environment Facilitating everyday environments with real users
KAIST, AIM Lab Ubi-Comp Environment is itself the Interface Everyday objects augmented with sensing table chairs glasses … Creating a digital shadow reflecting the interaction
KAIST, AIM Lab Embedding Interaction Basic technologies for embedding interaction Sensing technologies Environmental conditions Users’ location Co-location with others Physiological and emotional state of the user User goals User schedules … Agent technologies Combining a multitude of sometimes contradictory inputs to make sense at a higher level Adopting a system’s output to be appropriate to whatever situation might arise
KAIST, AIM Lab Implicit Interaction (1/2) Implicit Human-Computer Interaction (iHCI) iHCI is the interaction of a human with the environment and with artifacts which is aimed to accomplish a goal. Within this process the system acquires implicit inputs from the user and may present implicit output to the user. Implicit Input Implicit inputs are actions and behaviour of humans, which are done to achieve a goal and are not primarily regarded as interaction with a computer, but captured, recognized and interpret by a computer system as input. Implicit Output Output of a computer that is not directly related to an explicit input and which is seamlessly integrated with the environment and the task of the user.
KAIST, AIM Lab OXYGEN Project Speech and vision technologies enable us to communicate with Oxygen as if we’re interacting with another person, saving much time and effort MIT Media Lab
KAIST, AIM Lab AwareHome Designing the Interactive Experience Digital Family Portrait reconnects geographically distant extended family members by allowing them to remain aware of each other in a non-obtrusive, lightweight manner What Was I Cooking? a context-aware system that captures the transient information of recent activities and passively displays them as visual cues. Gesture Pendant Gesture Pendant recognizes and then translates gestures into commands for your home appliances AwareHome with human-like perception could improve quality of life for many, especially seniors. Georgia Tech.
KAIST, AIM Lab Easy Living(1) EasyLiving is developing a prototype architecture and technologies for building intelligent environments System Architecture Key features Computer vision for person- tracking and visual user interaction. Multiple sensor modalities combined. Use of a geometric model of the world to provide context. Automatic or semi-automatic sensor calibration and model building. Fine-grained events and adaptation of the user interface. Device-independent communication and data protocols. Ability to extend the system in many ways. Rules Engine Person Tracker Person Detector Person Detector Seat Sensors PC Logon Fingerpri nt Logon Room Lights A/V Media Systems Terminal Server Room Control UI KB/Mou se Redirect Desktop Manage r World Model Agent Lookup person tracking world model room control authentication Microsoft
KAIST, AIM Lab Easy Living(2).... Personal Detection Stereo Processing with commercial software Background subtraction and person detection Reports sent to central personal tracker about 7Hz Personal Tracking Process each new report from a sensor colordepth patches people Past locations Predicted location New sensor measurement “Person creation zone” Microsoft
KAIST, AIM Lab HomeLab Philips HomeLab appearance looking and feeling like a regular home for testing its new home technology prototypes in the most realistic possible way WWICE PHENOM EASY ACCESS POGO: an interactive game for children virtual story world interfaced by active tools Intelligent Personal-Care Environment based on measurements from the activity monitor and heart rate sensor Philips Research
KAIST, AIM Lab Role of Wearable Computer in Ubiquitous Computing Environment It easily acquires personal data (personalization). It guarantees safety of personal data (privacy). It enhances user’s interaction with many devices. It reduces network traffic about personal data transmitting. It assists us to work (agent).
KAIST, AIM Lab Background This system should understand a user’s intention or preference. This system should communicate with various electronic media. A system that assists us in interacting with those media in our daily life is required. A system that assists us in interacting with those media in our daily life is required. We will frequently interact with those media. (We will feel much annoyed with this interaction.) We will frequently interact with those media. (We will feel much annoyed with this interaction.) Various electronic media will be scattered around us in the near future (ubiquitous computing environments). Various electronic media will be scattered around us in the near future (ubiquitous computing environments).
KAIST, AIM Lab Research Objective ♦ To Establish Some Concepts - IEM - IWAS ♦ To Establish Some Concepts - IEM - IWAS ♦ To Propose an IWAS prototype and IEM prototypes ♦ To demonstrate interaction of IWAS and IEM
KAIST, AIM Lab IEM Interactive Electronic Media electronic media in ubiquitous computing environment that are not only controlled by a user’s command but that also respond to context or the user’s emotional state... Wireless Control IEM
KAIST, AIM Lab IEM IEM Examples IEM encapsulated electronic appliances such as a TV, a video player, a radio, a computer, and etc. Responsive digital media interactive media artworks All objects with embedded computer chips or sensors an automatic curtain that rises or falls according to a user’s intention or preference a lamp that intelligently controls the intensity of light according to a user’s emotional state IEM Features Wireless control => ultimately, automation (agent system) Unique ID Interaction capability
KAIST, AIM Lab IWAS Intelligent Wearable Assistance System
KAIST, AIM Lab IWAS H/W Design Self-contained System to Wear integrating all components of wearable computer with a suit User-friendly Interface input: speech recognition, key-pad, mouse, etc. output: see-through HMD, small speakers. Various Sensors FSR and postural sensing unit Infra-red tag reading unit Wireless networking Wireless LAN, IEEE 802.11b
KAIST, AIM Lab Functions of IWAS Intelligent User-Assistance local identification using IR sensor device direct control of IEM using IR remote controller communication via wireless LAN or Bluetooth information service such as schedule alert, email check interacting with media
KAIST, AIM Lab Functions of IWAS Home Network (Home RF, IEEE902.11, ···) Home G/W TV PC Audio Player Phone Sensing Control Intelligent Agent IEM identification using IR sensor wireless control using IR remote controller UbiComp Environment providing personalized information service Lamp
KAIST, AIM Lab IWAS H/W Prototype FSR sensor See-through HMD with speech head set IR tag reader & IR remote controller 3-axis postural sensor IWAS suit
KAIST, AIM Lab Interaction with IWAS and IEM CASE 1: Operating a laptop computer CASE 2: Turning on TV
KAIST, AIM Lab Interaction with IWAS and IEM CASE 3: Controlling virtual system