Shift: A Technique for Operating Pen-Based Interfaces Using Touch Daniel Vogel University of Toronto Patrick Baudisch Microsoft Research

Motivation

Small Targets

Advantages of the Pen PenFinger unique contact point remove hand from screen ambiguous contact point finger occludes target

Possible Solutions …

Offset Cursor (Potter et al. 1988) PenOffset Cursor

Offset Cursor (Potter et al. 1988)

Disadvantages 1.no visual feedback until contact, need to estimate offset 2.makes some display areas inaccessible 3.unexpected offset affects walk-up-and-use scenarios Offset Cursor (Potter et al. 1988)

Shift

Benefit 1: Aim for the Target Users expect to click on the target itself.  allows switching between pen and touch  walk-up and use with kiosk

Users expect to click on the target itself.  allows switching between pen and touch  walk-up and use with kiosk Benefit 1: Aim for the Target

Benefit 2: All Areas Accessible Callout is relative to finger, so it can go anywhere.  no edge problems

Callout is relative to finger, so it can go anywhere.  no edge problems Benefit 2: All Areas Accessible

Callout only used when necessary  same speed as unaided touch screen for large targets Benefit 3: Fast For Large Targets

Design Iterations

Performance Model Model

First Prototype

Revision and Visuals

Based on selection ambiguity with fallback to hesitation. S T = Target Size, S F = Finger occlusion threshold  S T << S F  high selection ambiguity  no delay  S T >> S F  no selection ambiguity  long delay  S T ≈ S F  “ambiguous selection ambiguity”  short delay Escalation

Based on selection ambiguity with fallback to hesitation. S T = Target Size, S F = Finger occlusion threshold  S T << S F  high selection ambiguity  no delay  S T >> S F  no selection ambiguity  long delay  S T ≈ S F  “ambiguous selection ambiguity”  short delay

user’s view hardware’s view Perceived Input Point Correction Users expect selection point to be higher. Iterative estimate for a correction vector V using difference between initial contact point P 1 and final lift off point P 2  V t+1 = V t + w(P 2 - P 1 )

Experiment

Experimental Design 3 techniques (Shift, Touch, Offset Cursor) x 2 finger styles (nail, tip) x 3 blocks x 6 target sizes (6, 12, 18, 24, 48, 96px) x 4 target directions (NW, NE, SW, SE)

Error

Time

Corrective Movements

Discussion Able to select small targets reliably (like Offset Cursor) Fast for large targets (like unaided Touch Screen) However, biggest benefit may be simpler mental model:  “Just aim for the target”

High Accuracy Enhancements Added Zooming and CD-Ratio Manipulation

High Accuracy Enhancements

Thanks to members of the ASI and VIBE groups at MSR, special thanks to Raman Sarin, Ed Cutrell, and David Thiel.

Appendix

Estimating Occlusion Threshold Don’t know actual finger size, so estimate it over time  when S T ≈ S F  short delay … means user can choose to use escalation by hesitating or not  if they hesitate and use escalation  make S F larger  if they just click without escalation  make S F smaller

Prototypes