1. Ubiquitous Computing
Lecture 28

2. Topics we shall Cover today
Introduction to Ubiquitous Computing
History
Definition
Need
Phases
Challenges and Researches in Ubiquitous Computing.

3. The Trends in Computing Technology
Late 1990s
Now and Tomorrow ?

4. Pervasive Computing Era

5. Computing Evolution

6. The Major Trends in Computing
4/21/2017
The Major Trends in Computing
Mainframe (Past) :N one computer shared by many people 
Personal Computer (Present) :1 one computer, one person
 
Ubiquitous Nk:1
Computing
N: *Internet - Widespread Distributed Computing*
In the past fifty years of computation there have been two great trends in this relationship: the mainframe relationship, and the PC relationship.
Today the Internet is carrying us through an era of widespread distributed computing towards the relationship of ubiquitous computing, characterized by deeply imbedding computation in the world.

7. Phase I - The Mainframe Era
4/21/2017
Phase I - The Mainframe Era
Computers were a scarce resource run by experts behind closed doors.
The first era we call "mainframe", to recall the relationship people had with computers that were mostly run by experts behind closed doors. Anytime a computer is a scarce resource, and must be negotiated and shared with others, our relationship is that of the mainframe era.

8. 4/21/2017
Phase II - The PC Era
In 1984 the number of people using PCs surpassed that of people using mainframe computers.
PC Era: You have your computer, it contains your stuff, and you interact directly and deeply with it.
The PC is most analogous to the automobile.
Now we are in the personal computing era, person and machine staring uneasily at each other across the desktop.
The second great trend is that of the personal computer. In 1984 the number of people using personal computers surpassed the number of people using shared computers.
The personal computer is most analogous to the automobile - a special, relatively expensive item, that while it may "take you where you want to go", requires considerable attention to operate.
Some people name their PC - many people curse or complain to their PC.
Any computer with which you have a special relationship, or that fully engages or occupies you when you use it, is a personal computer.

9. Transition Phase - The Internet
4/21/2017
Transition Phase - The Internet
The Internet brings together elements of the mainframe era and the PC era. 
Client = PC
Server = Mainframe
It is client-server computing on a massive scale, with web clients the PCs and web servers the mainframes.

10. 4/21/2017
Phase III - The UC Era
The UC era will have lots of computers shared by each one of us.
UC is fundamentally characterized by the connection of things in the world with computation.
Frequently used related terms:
Pervasive computing, Wearable computers,
Intelligent environment, Things That Think (T³),
Wearware, Personal Area Networking (PAN).
Some of these computers will be the hundreds we may access in the course of a few minutes of Internet browsing. Others will be imbedded in walls, chairs, clothing, light switches, cars - in everything.
All kinds of jargon. . Wireless internet. Peripheral computing. Self-configuring, adaptively coordinated Embedded Nets.. Locator Tags. JINI.

11. Ubiquitous Computing Mark Weiser, Xerox PARC 1988
“Ubiquitous computing enhances computer use by making many computers available throughout the physical environment, but making them effectively invisible to the user.”
Source: Weiser, 1993a

12. Pervasive (Ubiquitous) Computing Vision
“In the 21st century the technology revolution will move into the everyday, the small and the invisible…”
“The most profound technologies are those that disappear. They weave themselves into the fabrics of everyday life until they are indistinguishable from it.”
Mark Weiser (1952 –1999), XEROX PARC
Small, cheap, mobile processors and sensors
in almost all everyday objects
on your body (“wearable computing”)
embedded in environment (“ambient intelligence”)

13. Related Topics Several terms that share a common vision
Pervasive Computing
Sentient computing
Ubiquitous Computing
Ambient Intelligence
Wearable Computing
Context Awareness
...

14. What is Ubiquitous Computing?
Ubiquitous computing (ubicomp) integrates computation into the environment, rather than having computers which are distinct objects.
The idea of ubicomp enable people to interact with information-processing devices more naturally and casually, and in ways that suit whatever location or context they find themselves in.
~from Wiki

15. Goals of Pervasive (Ubiquitous) Computing
Ultimate goal:
Invisible technology
Integration of virtual and physical worlds
Throughout desks, rooms, buildings, and life
Take the data out of environment, leaving behind just an enhanced ability to act

16. 4/21/2017
What UC is NOT
It is not science fiction (SF), though it relies a great deal on it.
It is not impossible.
It is not Virtual Reality (VR).
It is not a Personal Digital Assistant (PDA).
It is not a personal agent (PA).
It cannot be mistaken for VR, ever
They are not “multimedia computers”
Ubiquitous computing is roughly the opposite of virtual reality
It is not a Personal Digital Assistant (PDA) such as Apple's Newton. It is not a personal or intimate computer (IA)with agents doing your bidding.
UC vs. VR
Virtual reality puts people inside a computer-generated world.
Ubiquitous computing forces the computer to live out here in the world with people.
Virtual reality is primarily a horse power problem
Ubiquitous computing is a very difficult integration of human factors, computer science, engineering, and social sciences.
Unlike virtual reality, ubiquitous computing endeavers to integrate information displays into the everyday physical world. It considers the nuances of the real world to be wonderful, and aims only to augment them.
Diametrically opposed to our vision is the notion of "virtual reality," which attempts to make a world inside the computer. Users don special goggles that project an artificial scene on their eyes; they wear gloves or even body suits that sense their motions and gestures so that they can move about and manipulate virtual objects. Although it may have its purpose in allowing people to explore realms otherwise inaccessible -- the insides of cells, the surfaces of distant planets, the information web of complex databases -- virtual reality is only a map, not a territory.
The opposition between the notion of virtual reality and ubiquitous, invisible computing is so strong that some of us use the term "embodied virtuality" to refer to the process of drawing computers out of their electronic shells.
UC vs. PDA
Unlike PDA's, ubiquitious computing envisions a world of fully connected devices, with cheap wireless networks everywhere
Unlike PDA's, it postulates that you need not carry anything with you, since information will be accessable everywhere.
UC vs. IA
Unlike the intimate agent computer that responds to one's voice and is a personal friend and assistant, ubiquitous computing envisions computation primarily in the background where it may not even be noticed. Whereas the intimate computer does your bidding, the ubiquitous computer leaves you feeling as though you did it yourself.
World Is Not a Desktop:
Take intelligent agents. The idea, as near as I can tell, is that the ideal computer should be like a human being, only more obedient. Anything so insidiously appealing should immediately give pause. Why should a computer be anything like a human being? Are airplanes like birds, typewriters like pens, alphabets like mouths, cars like horses?
Take magic. The idea, as near as I can tell, is to grant wishes: I wish my computer would only show me what I am interested in.
Take virtual reality. The idea, as near as I can tell, is that by moving to full-body-sensing and interaction we'll solve the user interface problem by maximally utilizing all of our body's input and output channels.
Take voice input. The idea, as near as I can tell, is that if I could just talk to my computer it would finally understand me. The problem is, if I could talk to my computer today, I'd have to talk in C or Fortran or CSH, because that is what they understand.
I do think that research on agents, speech recognition, and so on is important; the problem is that they are all in the domain of the conscious interaction.

17. Phases of Ubiquitous Computing

18. Pervasive Computing Phase I
Smart, ubiquitous I/O devices: tabs, pads, and boards
Hundreds of computers per person, but casual, low-intensity use
Many, many “displays”: audio, visual, environmental
Wireless networks
Location-based, context-aware services
Using a computer should be as refreshing as a walk in the woods

19. Smart Objects
Real world objects are enriched with information processing capabilities
Embedded processors
in everyday objects
small, cheap, lightweight
Communication capability
wired or wireless
spontaneous networking and interaction
Sensors and actuators

20. Smart Objects (cont.) Can remember pertinent events
They have a memory
Show context-sensitive behavior
They may have sensors
Location/situation/context awareness
Are responsive/proactive
Communicate with environment
Networked with other smart objects

21. Smart Objects (cont.)

22. Pervasive Computing Enablers
Moore’s Law of IC Technologies
Communication Technologies
Material Technologies
Sensors/Actuators

23. Moore’s Law
Computing power (or number of transistors in an integrated circuit) doubles every 18 months

24. Moore’s Law
1965
Computing power (or number of transistors in an integrated circuit) doubles every 18 months

25. Generalized Moore’s Law
Most important technology parameters double every 1–3 years:
computation cycles
memory, magnetic disks
bandwidth
Consequence:
scaling down
Problems:
• increasing cost
• energy

26. 2nd Enabler: Communication
Bandwidth of single fibers ~10 Gb/s
2002: ~20 Tb/s with wavelength multiplex
Powerline
coffee maker “automatically” connected to the Internet
Wireless
mobile phone: GSM, GPRS, 3G
wireless LAN (> 10 Mb/s)
PAN (Bluetooth), BAN

27. Body Area Networks
Very low current (some nA), some kb/s through the human body
Possible applications:
Car recognize driver
Pay when touching the door of a bus
Phone configures itself when it is touched
nA= nanoampere

28. Spontaneous Networking
Objects in an open, distributed, dynamic world find each other and form a transitory community
Devices recognize that they “belong together”

29. 3rd Enabler: New Materials
Important: whole eras named after materials
e.g., “Stone Age”, “Iron Age”, “Pottery Age”, etc.
Recent: semiconductors, fibers
information and communication technologies
Organic semiconductors
change the external appearance of computers
“Plastic” laser
Flexible displays,…
Plastic laser – means technology being so advanced that visual items are flexible or foldable.

30. Interactive Map
Foldable and rollable
You are here!

31. Foldable Cell Phone

32. Smart Clothing Conductive textiles and inks
print electrically active patterns directly onto fabrics
Sensors based on fabric
e.g., monitor pulse, blood pressure, body temperature
Invisible collar microphones
Kidswear
game console on the sleeve?
integrated GPS-driven locators?
integrated small cameras (to keep the parents calm)?

33. Solar Coat – Smart Clothings

34. Smart Glasses
By 2009, computers will disappear. Visual information will be written directly onto our retinas by devices in our eyeglasses and contact lenses -- Raymond Kurzweil

35. Google Glass

36. 4th Enabler: Sensors/Actuators
Miniaturized cameras, microphones,...
Fingerprint sensor
Radio sensors
RFID
Infrared
Location sensors
e.g., GPS
...

37. Example: Radio Sensors
No external power supply
energy from the actuation process
piezoelectric and pyroelectric materials transform changes in pressure or temperature into energy
RF signal is transmitted via an antenna (20 m distance)
Applications: temperature surveillance, remote control (e.g., wireless light switch),...

38. RFIDs (“Smart Labels”)
Identify objects from distance
small IC with RF-transponder
Wireless energy supply
~1m
magnetic field (induction)
ROM or EEPROM (writeable)
~100 Byte
Cost ~$ $1
consumable and disposable
Flexible tags
laminated with paper

39. Past, Present and Future Researches of Ubiquitous Computing

40. Computing with natural interfaces
Ubicomp inspires “off-the-desktop” applications
Needs “off-the-desktop” means of interaction
Speech, gestures, writing
More accessible
Easier to use???

41. Computing with natural interfaces
Error prone interaction
Permit new and numerous mistakes
People do not have perfect recognition
As low as 54%; cursive handwriting 88%; printed handwriting 96.8%
Recognition accuracy == user satisfaction??
Not really: complexity of error recovery dialogues and value-added benefit of any given efforts
Entering a command vs. writing journal entries
Several research areas
Error reduction (about 5-10%)
Error detection
Reusable toolkit for error handling

42. Context aware computing
Current Systems
Generally using position and identification of objects
Still do not provide a complete context
Definition of context is limited
Research areas
Context toolkits
Toolkit for sensing environment
Explicit use of sensed information is up to program
What is context?
How is context represented?

43. What is context? Who What Where When Why
Currently generally tailored to one user
How important are others in determining our behavior
How could this be captured?
What
Attempt to figure out what is currently happening
Sense environment, use calendar software etc.
Where
Location based information, e.g., GPS
Most explored context information
When
Easily obtained information -- Computer is good at remembering time
Although determining when one event stops and another begins is not easy
Why
Even harder than the “what” question, biometric sensors might help (e.g., body temperature, heart rate, etc)

44. Toward context aware computing
Context representation
Requires universal context schemes or toolkits with standard context representations
Context sensing and fusion
How to make context-aware computing “ubiquitous”?
In practice, there are few truly ubiquitous, single-source context services
E.g., GPS does not work indoors; different indoor localization schemes have different characteristics (e.g., cost, range)
Like sensor fusion, context fusion handles seamless handling of sensing responsibility between boundaries of different context services
Combining multiple context sources can increase the accuracy of context information

45. Automated capture and access
Recording information and data as it occurs
Computers are inherently good at recording, people are not
People freed up to summarize and understand
Most work in academic/ classroom settings
Time stamping lectures, digital whiteboards
Challenges in “capture and access”
Sometime we don’t know we want to capture something until after its already happened
How could the computer know that?
If it captures everything then we need a system of sorting and filtering (access)
Access is a problem because capturing of raw data can be burdensome for sifting through; systems need to recognize important events facilitate access

46. Everyday computing
Continuous interactions (i.e., no clear beginning or end)
Both fundamental activities like communication and long-term endeavors do not have predefined starts and ends; information from past can be recycled
Very different traditional HCI design which assumes “closure” with clear goals like spell checking, dialogue, etc.
Interruption is expected:
People are constantly interrupted
Computer systems must recognize interruption and change state
Also computers must appropriately inform users
Multiple activities operate concurrently:
People multitask and rapidly switch task based on external unpredictable environment
Systems need to adapt to this opportunistic behavior and change accordingly

47. Toward everyday computing
Develop continuously present interface
No current model of continuously present interfaces, even people are not continuously present
Create an interface that doesn’t get annoying (e.g., wearable devices)
Determine what information should require my attention and what should be display peripherally
Connect events in the physical and virtual worlds (e.g., face to face vs. , document, webs)
Modify/fuse existing HCI schemes to efficiently support everyday computing (but evaluation is challenging and laborious)

48. System evaluation challenges
Hard to evaluate Ubicomp Systems
Little publish on ubicomp evaluation
Systems often required to be fully connected leading to systems that are hard to build
Lack of development toolkits make system creation difficult
Systems often need to be integrated into peoples lives which using big clunky prototypes does not lead itself well too
Task/Goal centric approaches don’t work in ubicomp

49. Example Projects
Pervasive computing projects have emerged at major universities and in industry:
Project Aura (Carnegie Mellon University)
Oxygen (Massachusetts Institute of Technology)
Portalano (University of Washington)
Endeavour (University of California at Berkeley)
Place Lab (Intel Research Laboratory at Seattle)
For illustration let us look at Project Aura

50. Example Projects : Project Aura (1)
Aura (Carnegie Mellon University)
Distraction-free (Invisible) Ubiquitous Computing.
The most precious resource in a computer system is no longer its processor, memory, disk or network. Rather, it is a resource not subject to Moore's law: User Attention. Today's systems distract a user in many explicit and implicit ways, thereby reducing his effectiveness.
Project Aura will fundamentally rethink system design to address this problem. Aura's goal is to provide each user with an invisible halo of computing and information services that persists regardless of location. Meeting this goal will require effort at every level: from the hardware and network layers, through the operating system and middleware, to the user interface and applications.
Project Aura will design, implement, deploy, and evaluate a large-scale system demonstrating the concept of a “personal information aura” that spans wearable, handheld, desktop and infrastructure computers.
PROJECT WEBSITE:

51. Example Projects : Project Aura (2)
Moore’s Law Reigns Supreme
Processor density
Processor speed
Memory capacity
Disk capacity
Memory cost
...
Glaring Exception
Human Attention
Human Attention
The most precious resource in a computer system is no longer its processor, memory, disk or network. Rather, it is a resource not subject to Moore's law: User Attention. Today's systems distract a user in many explicit and implicit ways, thereby reducing his effectiveness.
Project Aura will fundamentally rethink system design to address this problem. Aura's goal is to provide each user with an invisible halo of computing and information services that persists regardless of location. Meeting this goal will require effort at every level: from the hardware and network layers, through the operating system and middleware, to the user interface and applications.
Project Aura will design, implement, deploy, and evaluate a large-scale system demonstrating the concept of a “personal information aura” that spans wearable, handheld, desktop and infrastructure computers.
PROJECT WEBSITE:
Adam & Eve
2000 AD

52. Example Projects : Project Aura (3)
Aura Thesis:
The most precious resource in computing is human attention.
Aura Goals:
Reduce user distraction.
Trade-off plentiful resources of Moore’s law for human attention.
Achieve this scalably for mobile users in a failure-prone, variable-resource environment.
The most precious resource in a computer system is no longer its processor, memory, disk or network. Rather, it is a resource not subject to Moore's law: User Attention. Today's systems distract a user in many explicit and implicit ways, thereby reducing his effectiveness.
Project Aura will fundamentally rethink system design to address this problem. Aura's goal is to provide each user with an invisible halo of computing and information services that persists regardless of location. Meeting this goal will require effort at every level: from the hardware and network layers, through the operating system and middleware, to the user interface and applications.
Project Aura will design, implement, deploy, and evaluate a large-scale system demonstrating the concept of a “personal information aura” that spans wearable, handheld, desktop and infrastructure computers.
PROJECT WEBSITE:

53. Example Projects : Project Aura (4)
The Airport Scenario
Jane wants to send from the airport before her flight leaves.
She has several large enclosures
She is using a wireless interface
She has many options.
Simply send the
Is there enough bandwidth?
Compress the data first
Will that help enough?
Pay extra to get reserved bandwidth
Are reservations available?
Send the “diff” relative to older file
Are the old versions around?
Walk to a gate with more bandwidth
Where is there enough bandwidth?
How do we choose automatically?
The most precious resource in a computer system is no longer its processor, memory, disk or network. Rather, it is a resource not subject to Moore's law: User Attention. Today's systems distract a user in many explicit and implicit ways, thereby reducing his effectiveness.
Project Aura will fundamentally rethink system design to address this problem. Aura's goal is to provide each user with an invisible halo of computing and information services that persists regardless of location. Meeting this goal will require effort at every level: from the hardware and network layers, through the operating system and middleware, to the user interface and applications.
Project Aura will design, implement, deploy, and evaluate a large-scale system demonstrating the concept of a “personal information aura” that spans wearable, handheld, desktop and infrastructure computers.
PROJECT WEBSITE:

54. Example Projects : Project Aura (5)
The Mobile Task Scenario
Aura saves Scott’s task.
Scott enters office and gets strong authentication and secure access.
Aura restores Scott’s task on desktop machine and uses a large display.
Scott controls application by voice.
Bradley enters room.
Bradley gets weak authentication, Scott’s access changes to insecure.
Aura denies voice access to sensitive application.
Scott has multi-modal control of PowerPoint application.
Aura logs Scott out when he leaves the room.
The most precious resource in a computer system is no longer its processor, memory, disk or network. Rather, it is a resource not subject to Moore's law: User Attention. Today's systems distract a user in many explicit and implicit ways, thereby reducing his effectiveness.
Project Aura will fundamentally rethink system design to address this problem. Aura's goal is to provide each user with an invisible halo of computing and information services that persists regardless of location. Meeting this goal will require effort at every level: from the hardware and network layers, through the operating system and middleware, to the user interface and applications.
Project Aura will design, implement, deploy, and evaluate a large-scale system demonstrating the concept of a “personal information aura” that spans wearable, handheld, desktop and infrastructure computers.
PROJECT WEBSITE:

55. Other Scenarios of Ubiquitous Computing
Buy drinks by Friday (1)
Take out the last can of soda
Swipe the can’s UPC label, which adds soda to your shopping list
Make a note that you need soda for the guests you are having over this weekend

56. Other Scenarios Buy drinks by Friday (2) Approach a local supermarket
AutoPC informs you that you are near a supermarket
Opportunistic reminder: “If it is convenient, stop by to buy drinks.”

57. Other Scenarios Buy drinks by Friday (3)
Friday rolls around and you have not bought drinks
Deadline-based reminder sent to your pager

58. Other Scenarios Screen Fridge Provides: Email Video messages
Web surfing
Food management TV
Radio
Virtual keyboard
Digital cook book
Surveillance camera

59. Other Scenarios The Active Badge
This harbinger of inch-scale computers contains a small microprocessor and an infrared transmitter.
The badge broadcasts the identity of its wearer and so can trigger automatic doors, automatic telephone forwarding and computer displays customized to each person reading them.
The active badge and other networked tiny computers are called tabs.

60. Other Scenarios
The Active Badge

61. Other Scenarios Edible computers: The pill-cam
Miniature camera
Diagnostic device
It is swallowed
Try this with an ENIAC computer!

62. Other Scenarios Artificial Retina Direct interface with nervous system
Whole new computational paradigm (who’s the computer?)

63. Other Scenarios Smart Dust Nano computers that couple:
Sensors
Computing
Communication
Grids of motes (“nano computers”)

64. Summary
Moving our focus of interaction away from the traditional two-dimensional graphical user interface on the desktop presents many exciting and new challenges to the field of HCI.
Weiser’s vision of ubiquitous computing was human-centered, and many years later, it still presents a grand challenge for those who wish to address this new interaction paradigm.