Su-ting, Chuang 1

Outline Introduction Related work Hardware configuration Finger Detection system Optimal parameter estimation framework Conclusion 2

Outline Introduction Related work Hardware configuration Finger Detection system Optimal parameter estimation framework Conclusion 3

Introduction Motivation Evaluate components in finger detection systems Verify and improve performance of finger detection systems Method Develop an optimal parameter estimation framework Use most prevalent finger detection system as testbed Touchlib 4

Outline Introduction Related work Hardware configuration Finger Detection system Optimal parameter estimation framework Conclusion 5

Related Work FTIR (Frustrated Total Internal Reflection) J. Y. Han, “Low-cost multi-touch sensing through frustrated total internal reflection," in Proceedings of the 18th annual ACM symposium on User interface software and technology (UIST '05). New York, NY, USA: ACM Press, 2005, pp

Related Work DI (Diffused Illumination) J. Rekimoto and N. Matsushita, “Perceptual surfaces: Towards a human and object sensitive interactive display," Workshop on Perceptural User Interfaces (PUI'97),

Related Work TouchLib A multi-touch development kit Finger detection processing flow chart 8 Background Subtraction Simple Highpass ScaleMonoThreshold

Outline Introduction Related work Hardware configuration Finger Detection system Optimal parameter estimation framework Conclusion 9

Hardware configuration Table setup 10

Hardware configuration Order of diffuser layer and touch-glass layer 11 Diffuser layer IR illuminator IR camera spot IR illuminator IR camera Touch-glass layer IR camera spot IR camera

Hardware configuration Problem: IR rays will be reflected by the touch-glass and resulting hot spot regions in camera views Solution: Use other cameras to recover the regions which are sheltered by IR spots 12

Outline Introduction Related work Hardware configuration Finger Detection system Optimal parameter estimation framework Conclusion 13

Detection system IR cam Pre- processing Image processing Image processing Finger Analyzing Finger Analyzing Data Association Data Association Data Transmission Data Transmission IR cam GPU CPU 14

Detection system Pre-processing Image Fusion (Blend) IR Camera IR camera Undistortion HomoWarp 15

Pre-processing Undistortion Undistort foreground objects Warp Unify finger size among different position of table Image fusion Mask hot spots and recover them from the other camera image Finger at border won’t be discard 16

Pre-processing Advantage of implementing on GPU Increase performance High frame rate Preserve CPU for application computation Enable detection system and interactive application on the same computer Reduce unsynchronized problem among different computers 17

Detection system Image processing 18 Background Subtraction Normalization Simple Highpass MonoThreshold

Image processing Normalization Motivation Eliminate influence due to non-uniform lighting condition Various finger touch response Hard to decide a good threshold Method Model distribution of IR illumination Use specific material to simulate foreground Calculate each pixel’s dynamic range Stretch dynamic range to

Finger Analyzing Connected component finger size evaluation 20

Data association Fingertip matching Matching fingertips among frames Using bipartite algorithm Fingertip tracking Smooth detected results and fix lost results Using Kalman filter 21

Outline Introduction Related work Hardware configuration Detection system Optimal parameter estimation framework Conclusion 22

Optimal parameters estimation framework Motivation Find optimal parameters for finger detection system 23

Optimal parameters estimation framework Procedure Define parameters for finger detection system Collect samples Various finger size Various hand gesture Search optimal parameters Verify performance of all possible parameter combinations 24

Optimal parameters estimation framework Collect samples Task Soft /Hard touch Vertical/Oblique touch Various fingers Sample set Each task has 2x2x5 samples Sample collection Step-by-step instruction Straightforward UI design Finger touch position 5 timer Instructions…. 25

Optimal parameters estimation framework Search optimal parameters Exhaustive search Test various parameter combination in each set Step Each parameter combination Detect finger touch Verify detection result Calculate error rate 26

Optimal parameters estimation framework 27 Detection system frame Optimal parameter finder Parameter Set Detection Result Ground Truth Optimal Parameter Set Verify Next Parameter Set Generator Detection Result Ground Truth Error Rate Parameter Set Sample set

Outline Introduction Related work Hardware configuration Detection system Optimal parameter estimation framework Conclusion 28

29