Download presentation
Presentation is loading. Please wait.
Published bySally Guthrie Modified over 10 years ago
1
Interaction Techniques with Mobile Devices Jingtao Wang jingtaow@cs.berkeley.edu March 6th, 2006 Guest Lecture for CS160
2
Agenda Why Mobile Devices Matters Ubiquitous Computing Key Challenges in Designing Mobile Applications Input Techniques for Mobile Devices Output Techniques for Mobile Devices Interact With Other Devices
3
Why Mobile Devices Matters 6.5 billion people in the world 1.5 billion cell phones worldwide 500 million PCs (?) 46 million PDAs 1 million TabletPCs Challenge: How can handheld devices improve the user interfaces of everything else, and not just be another gadget to be learned
4
Agenda Why Mobile Devices Matters Ubiquitous Computing Key Challenges in Designing Mobile Applications Input Techniques for Mobile Devices Output Techniques for Mobile Devices Interact With Other Devices
5
Mark Weiser (1952 – 1999) Introduced the idea of ubiquitous computing
6
Weisers Vision The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it
7
Weisers Vision: Pervasive Mainframe many people share 1 computer PC 1 person with 1 computer Ubicomp many computers server each person
8
Agenda Why Mobile Devices Matters Ubiquitous Computing Key Challenges in Designing Mobile Applications Input Techniques for Mobile Devices Output Techniques for Mobile Devices Interact With Other Devices
9
Key Challenges in Making Mobile Applications Limited Physical Resources CPU, Memory, Screen Size, Input Devices, Battery Life etc Diversified Context of Use Different Activities Limited Attention
10
Limited Physical Resources A mobile device usually has 1/100 CPU power, 1/30 Screen resources, 1/20 Memory, and extremely limited input devices when compared with desktops in the same era. Small Screen Geography is different a. Large Screen b. Small Screen
11
Diversified Context of Use
12
Different Activities People use small-screen devices for different activities than desktops; dont assume you understand these activities already
13
Limited Attention Dont assume your applications have peoples full attention; theyre doing something else while using your device.
14
Context, Activity, Attention There is more opportunity for purpose-specific or context-specific devices than for general-purpose solutions that try to work for everyone in any situation.
15
One Sentence Summary Mobilize, Dont Miniaturize ! There is no silver bullet in designing mobile applications, but there is one sentence you should remember -
16
Agenda Why Mobile Devices Matters Ubiquitous Computing Key Challenges in Designing Mobile Applications Input Techniques for Mobile Devices Output Techniques for Mobile Devices
17
Input Techniques for Mobile Devices Pointing Text Input (Virtual) Keyboard Input Handwriting Input Speech Input Marker Based Input
18
Common Pointing/Navigation Techniques iPod Dialpad TrackPoint JogDial Touch Screen Four-directional keypad
19
TinyMotion – Camera Phone Based Pointing Detecting the movements of cell phones in real time by analyzing image sequences captured by the built-in camera. Typical movements include - horizontal and vertical movements, rotational movements and tilt movements.
20
Input Techniques for Mobile Devices Pointing Sensor Augmented Input Text Input (Virtual) Keyboard Input Handwriting Input Speech Input Marker Based Input
21
(Virtual) Keyboard Input
22
How to Make QWERTY Keyboards Portable ? Break
23
Making QWERTY Keyboards Portable Reducing the Absolute Size Reducing the Number of Keys Making Keyboards Foldable Projective Keyboard
24
From http://www.vkb-tech.com
25
Projective Keyboard – Working Mechanisms 1. Template creation 2. Reference plane illumination 3. Map reflection coordinates 4. Interpretation and communication
26
Can We Perform Better Than QWERTY? Originally QWERTY layout is manually optimized for two handed, alternative typing to minimize mechanical jam OPTI ATOMIK OPTI II FITALY
27
Theories Behind Quantitative Keyboard Layout Optimization Fitts Law Digraph Distribution Model in a Language
28
Can We Use Less Buttons than a Full QWERTY? 12-button Keypad 15-button Keyboard Half Keyboard
29
Disambiguation Methods for Reduced Size Keyboard The QWERTY keyboard itself is ambiguous! ( A vs. a, 3 vs. #) Pressing Several Keys together (shift key) Multiple Key Press Multi-Tap (22.5 wpm*) Two-Key Input (25.0wpm*) Dictionary/Statistics Based Disambiguation Methods T9/T15 (45.7 wpm*) LetterWise *Performance Upper Bound Estimation from Silfverberg 2000
30
FingerSense – Button Disambiguation by Fingertip Identification Differentiating a pressing action by identifying the actual finger involved Can be Faster than Regular Tapping When the Adjacent Tapping Involves Different Fingers and Different Buttons (59% on a phone keypad)
31
Input Techniques for Mobile Devices Pointing Sensor Augmented Input Text Input (Virtual) Keyboard Input Handwriting Input Speech Input Marker Based Input
32
Handwriting Input 1938 George Hansel, U.S. Patent 2,143,875, machine recognition of handwriting 1957 T. L. Dimond's stylator - the first on-line handwriting recognizer prototype Newton, Palm Pilot, PocketPC, CrossPad, TabletPC
33
Handwriting Recognition - Terminology Printed Character Recognition (OCR) Relatively mature these days, key challenges – layout analysis, fonts recovery, robust recognition for low quality, low resolution input Major Usage – Digital Library Handwritten Character Recognition Online HWR (With Temporal info) Character, Word, Sentence Level Offline HWR (Using raster image as input, no temporal info) Major Usage : Bank Check Recognition, Postal Automation
34
Word/Sentence Level Recognizer Build on Top of Character Recognizer General Strategy Over Segmentation Call Character/Component Recognizer, Get a List of Candidates with Scores Apply Geometry Spatial Information ( size, component gap ) Language Information (Dictionary, Language Model etc) to Each Sub Path Use Hypnosis Search (Dynamic Programming, A* etc) to Determine the Best Possible Path
35
Challenges in Online Handwriting Recognition Character Set/Dictionary Size (Especially Asian Languages!) Cursive Writing Styles/Broken Strokes/Duplicate Strokes/Omitted Components Stroke Order Variations Limited memory and CPU Power in Small Devices
36
Some Prototype Recognizers from IBM
37
New Form Factors - Anoto Pen Commercial Product is Available In the U.S. Market (Logitech IO Pen) Uses A Camera Mounted Beside the Tip of the Pen and Preprinted Dot Patterns to Detect Pen Movment
38
SHARK – Shorthand Writing on Stylus Keyboard A Combination of Virtual Keyboard and Handwriting Recognition Writing Shape of a Word (Shorthand) is Defined By the on Screen Location of Characters in the Word
39
EdgeWrite Input An EdgeWrite user enters text by traversing the edges and diagonals of a square hole imposed over the usual text input area Faster and More Reliable Than Regular Graffiti Especial Useful for People with Motor and Muscle Disabilities
40
Input Techniques for Mobile Devices Pointing Sensor Augmented Input Text Input (Virtual) Keyboard Input Handwriting Input Speech Input Marker Based Input
41
Input Techniques for Mobile Devices Pointing Sensor Augmented Input Text Input (Virtual) Keyboard Input Handwriting Input Speech Input Marker Based Input
42
Emerging Marker Based Interactions on Camera Phones
43
Towards More Sensitive Mobile Devices
44
Agenda Why Mobile Devices Matters Ubiquitous Computing Key Challenges in Designing Mobile Applications Input Techniques for Mobile Devices Output Techniques for Mobile Devices Interact With Other Devices
45
Peephole Displays (With Demo)
46
Zoomable Interface on Mobile Devices ZoneZoom By Microsoft Take advantage of spatial memory VS.
47
Halo - A Virtual Periphery for Mobile Devices Provding Visual Cue for Objects Located Out of the Small Screen
48
Agenda Why Mobile Devices Matters Ubiquitous Computing Key Challenges in Designing Mobile Applications Input Techniques for Mobile Devices Output Techniques for Mobile Devices Interact With Other Devices
49
Using Mobile Devices with Desktop Computers Pebbles Project at CMU Using a PDA as additional keypad, touch pad, scroll wheel and controller of PointPoint slides for desktop Applications http://www.pebbles.hcii.cmu.edu/
50
Using Mobile Devices with Laptops Wang 2002
51
Using Mobile Devices with Large Displays Ballagas 2005
52
Question and Answer
53
Backup Slides
54
Electromyographic (EMG) Keyboard NASA Virtual Keyboard SenseBoardKeyboard
55
The General Flow of Handwriting Recognition
56
Major Players in this Area (Embedded) English ART - ART Recognizer CIT - Jot IBM - Derived from Multi-lingual version Microsoft - Transcriber ( Licensed version of Calligrapher) & Own Single character recognizer Motorola Paragraph - Calligrapher Chinese/Japanese FineArt - GoGoPen Hanwang - more than 70% PDA market share in mainland China IBM Embedded HWR Motorola Lexicus - DragonPen PenPower - most influencial in Taiwan
57
Sensor Augmented Pointing
Similar presentations
© 2024 SlidePlayer.com Inc.
All rights reserved.