Su-ting, Chuang 2010/8/2

Outline Introduction Related Works System and Method Experiment Conclusion & Future Work 2

Outline Introduction Related Works System and Method Experiments Conclusion & Future Work 3

Introduction i-m-Top interactive multi-resolution tableTop interactive: multi-touch multi-resolution: fovea + peripheral projectors 4 Chia, Yi-Wei, i-m-Top, an Interactive Multi-resolution Tabletop Display System (2007)

Introduction i-m-Top SDK A software development toolkit for rapid prototyping multi-resolution and multi-touch applications 5

Introduction Low computation efficiency Non-uniform lighting problem Various finger touch responses among different positions No tools available for helping users determine parameters automatically 6

Outline Introduction Related Works System and Method Experiments Conclusion & Future Work 7

Related Works DI (Diffused Illumination) J. Rekimoto and N. Matsushita, “Perceptual surfaces: Towards a human and object sensitive interactive display," Workshop on Perceptural User Interfaces (PUI'97),

Related Works FTIR (Frustrated Total Internal Reflection) J. Y. Han, “Low-cost multi-touch sensing through frustrated total internal reflection," in Proceedings of the 18th annual ACM symposium on User interface software and technology (UIST '05). New York, NY, USA: ACM Press, 2005, pp

Related Work TouchLib A multi-touch development kit Finger detection processing flow chart 10 Background Subtraction Simple Highpass Intensity Scaling Thresholding Finger Analysis Parameters to be adjusted manually Image Enhancement a1a2, a3a4a5a6

Related Work DirectShow Filter-based framework GShow GPU-accelerated framework Combination of DirectX and DirectShow 11

Outline Introduction Related Work System and Method Experiments Conclusion & Future Work 12

Hardware Configuration (2) IR Camera (3) IR Illuminator (1) Peripheral Projector 13

Hardware Configuration Order of diffuser layer and touch-glass layer 14 Diffuser layer IR illuminator IR camera spot IR illuminator IR camera Touch-glass layer IR camera spot IR camera

Hardware Configuration Problem: IR rays reflected by the touch-glass will result in hot spot regions in camera views Solution: Two cameras to cover spot regions for each other’s sheltered IR spot zone 15

Software Architecture Detection system Image Stitching Finger Detection Finger Tracking Parameter determination 16 Image Stiching Image Stiching Finger Detection Finger Detection Finger Tracking Finger Tracking

Software Architecture 17 Image Stiching Image Stiching Finger Detection Finger Detection Finger Tracking Finger Tracking Enhance in GPU

Image Stitching Combine multi-camera view into a virtual camera view 18

Image Stitching Remove IR spot effect Unify finger size among different positions of table Easily fit in existing finger detection systems 19

Image Stitching 20 Image Blending IR Camera(L) IR camera(R) Undistortion HomoWarp

Image Stitching HomoWarp

Image Stitching Image Blending 22

Finger Detection TouchLib Our method 23 Normalization Difference of Gaussians Background Subtraction Thresholding Finger Analysis Simple Highpass Scale Background Subtraction Thresholding Finger Analysis

Finger Detection Insert normalization 24

Finger Detection Normalization Method Model distribution of IR illumination Use specific material to simulate foreground Construct normalization map Normalize foreground image Result Before normalization: mean = 75, standard variation = 30 After normalization: mean = 254.8, standard variation =

Finger Detection Approximate Difference of Gaussians (DoG) Modified from simple highpass in TouchLib TouchLib: (Original image – Blurred image ) + median filter 26

Fingertip Tracking Goal Smooth the trajectory of finger Method Kalman filter Use Kalman filter to predict the current position of t n 27

Parameter Determination Requirements of ideal finger detection system High sensitivity  miss ↓ Noise-free  false alarm ↓ Goal Find an appropriate set of parameters for finger detection system fulfilling the requirements 28

Parameter Determination 29 Parameters Determinator Parameter Combination Detection Result Applicable set of Parameters Test Set Touch Data Ground Truth (Trace) Detection System

Parameter Determination Evaluation of parameters Data Collection Depict trace Measurement Minimize # of miss and false alarm 30

Parameter Determination Ideal finger detection Only one fingertip landing on trace Continuity among frames 31

Outline Introduction Related Work System and Method Experiments Conclusion & Future Work 32

Experiments Performance evaluation x 480

Experiments Parameter determination Decide parameters in our system Adopt sampling-based parameter search technique 34 Normalization Difference of Gaussian Background Subtraction Binary Finger Analysis b4 b1: subtraction value b2: kernel size b3: threshold b4: finger size b3 b2 b1

Experiments Parameter determination Exhaustive search Parameter combination 5 (Iteration) *5 (Iteration) *5 (Iteration) *5 (Iteration) = 625 Applicable parameter num 16/625 = 2.56% 35 Subtract value Smooth kernel ThresholdFinger size Low bound 0510 Step55510 High bound

Outline Introduction Related Work System and Method Experiments Conclusion & Future Work 36

Conclusion & Future Work Multi-touch detection system GPU-accelerated Non-uniform lighting problem solved Automatically parameter determination tool proposed Future Work Optimize parameter determination 37

38