Su-ting, Chuang 1

Outline Introduction Related work Hardware configuration Detection system Optimal parameter estimation framework Conclusion 2

Introduction 3

Outline Introduction Related work Hardware configuration Detection system Optimal parameter estimation framework Conclusion 4

Related Work FTIR (Frustrated Total Internal Reflection) J. Y. Han, “Low-cost multi-touch sensing through frustrated total internal reflection," in Proceedings of the 18th annual ACM symposium on User interface software and technology (UIST '05). New York, NY, USA: ACM Press, 2005, pp

Related Work DI (Diffused Illumination) J. Rekimoto and N. Matsushita, “Perceptual surfaces: Towards a human and object sensitive interactive display," Workshop on Perceptural User Interfaces (PUI'97),

Outline Introduction Related work Hardware configuration Detection system Optimal parameter estimation framework Conclusion 7

Hardware configuration Table setup 8

Hardware configuration Order of diffuser layer and touch-glass layer 9 Diffuser layer IR illuminator IR camera spot IR illuminator IR camera Touch-glass layer IR camera spot IR camera

Hardware configuration Problem: IR rays will be reflected by the touch-glass and resulting IR spot regions in camera views Solution: Use other cameras to recover the regions which are sheltered by IR spots 10

Outline Introduction Related work Hardware configuration Detection system Optimal parameter estimation framework Conclusion 11

Detection system IR cam Pre- processing Image processing Image processing Finger Detection Finger Detection Data Association Data Association Data Transmission Data Transmission IR cam GPU CPU 12

Detection system Pre-processing Image processing Image Fusion (Blend) IR Camera IR camera Undistortion HomoWarp Background Subtraction Normalization Simple Highpass MonoThreshold 13 I (x,y) = a x I 1 (x,y) + (1-a) x I 2 (x,y)

Pre-processing Undistortion Undistort camera image Warp Unify finger size among different position of table Image fusion Increase intuition of vision Simplify foreground object matching among cameras 14

Pre-processing Advantage of implementing on GPU Increase performance High frame rate Preserve CPU for application computation Enable detection system and interactive application on the same computer Reduce unsynchronized problem among different computers 15

Image processing Normalization Motivation Eliminate influence due to ununiform lighting condition Various finger touch response Hard to decide a good threshold Method Model each pixel’s dynamic range Using specific material to simulate foreground Stretch dynamic range to

Image processing Finger Detection Connected component Finger analyzer finger size evaluation 17

Data association Fingertip matching Matching fingertips among frames Using bipartite algorithm Fingertip tracking Smooth detected results and fix lost results Using Kalman filter 18

Outline Introduction Related work Hardware configuration Detection system Optimal parameter estimation framework Conclusion 19

Software architecture 20 Detection system Sample set Training Parameter Set Detection Result Ground Truth Optimal Parameter Set Verify Next Parameter Set Generator Detection Result Ground Truth Error Rate Parameter Set

Optimal parameters estimation framework for finger detection Motivation Parameter set Procedure Collect samples Various finger size Hard press and soft press Search exhaustively Verify performance of all possible parameter combinations 21

Optimal parameters estimation framework for finger detection Task Soft /Hard touch Vertical/Oblique touch Various fingers Sample set Each task has 2x2x5 samples Sample collection Step-by-step instruction Straightforward UI design Finger touch position 5 timer Instructions…. 22

Method Exhaustive search Test various parameter combination in each set Step Each parameter combination Detect finger touch Calculate precision and error rate 23

Outline Introduction Related work Hardware configuration Detection system Optimal parameter estimation framework Conclusion 24

25

Sample collection Hard/Soft vertical touch Finger touch position 5 timer 26

Background Subtraction Normalization Simple Highpass MonoThreshold 27

Image Fusion (Blend) IR Camera IR camera Undistortion HomoWarp 28

Detection Module Verify Next Parameter Set Generator Detection Result Ground Truth Error Rate Parameter Set Parameter Set’ 29

30

31 Detection system Sample set Training Parameter Set Detection Result Ground Truth Optimal Parameter Set

32 Verify Next Parameter Set Generator Detection Result Ground Truth Error Rate

33 Detection system Sample set Training Parameter Set Detection Result Ground Truth Optimal Parameter Set Verify Next Parameter Set Generator Detection Result Ground Truth Error Rate Parameter Set

34