By shooting 2009/10/1

outline imTop overview imTop detection Finger Mobile Finger detection evaluation Mobile detection improvement

imTop Architecture Scene Touch/Mobile Detection Touch/Mobile Detection View Application (oForm) Node Other Sensors Projector PTU Speaker Socket Window Message COM Message Parser OpenGL c# Device Control Virtual Director (oDirector) Event queue LED Light CamNode

Background Subtraction Opening Down- scale Integral Image Connected Component Palm Tracking Connected Component PCA Analysis Palm-finger Association Finger Tracking Connected Component PCA Analysis Palm Association Finger Fusion fingerpalm

Camera Synchronize manager Capture camera (Source filter) Capture camera (Source filter) imTopFilter (Transform Filter) Merge finger results Match current result to previous Process un-matched finger points Smooth finger results imTopFilter (Transform Filter) Show detection results (Render filter) Send current results

Peripheral Projector Fovea Projector IR Camera IR Illuminator Mirror Pan-tilt Unit 56” 96 ppi 24 ppi

Undistortion Preprocess Build integral image Smooth image Background subtraciton Finger detection Finger result Detect lighter spot Connected component Finger analyze Kernel scan Sufficient energy Appropriate center imTopFilter (Transform Filter)

Flow chart Load Image Undistotion Pre-process Finger detection Show result Send Result to imTop Calculate Background image by 10 initial frames (initial 10 frames) Calculate Background image by 10 initial frames (initial 10 frames) Background subtraction (exceed 10 frames) Background subtraction (exceed 10 frames)

Finger detection Detect lighter spot Connected component finger analyze Set finger result Image preprocessing

Mobile detection Detect contour mobile analyze Set finger result Image preprocessing

Result management Match current result to the previous Process for un-matched finger points/mobile contours Process for un-matched finger points/mobile contours Smooth the detection results Send results Copy current results from each camera

outline imTop overview imTop detection Finger Mobile Finger detection evaluation Mobile detection improvement

Image preprocessing Smooth current image Smooth Image by Gaussian filter (kernel 3x3) Build integral image Calculate the M+2 by N+2 integral image of current M by N background subtracted image

Detect lighter spot Searching for region where the average intensity of inner part is higher enough then that of outer part Corners & Finger-size regions will be detected

Connected component Using previous finger candidates Using queue Calculate the mean position of each connected candidates Mark each connected component with groupID which is the position of the initial candidate

Finger analyze Calculate region center and energy of each connected component Accept as a Finger input if Sufficient Energy The Center is within its region

outline imTop overview imTop detection Finger Mobile Finger detection evaluation Mobile detection improvement

Mobile detection Image preprocessing Smooth current image Smooth Image by Gaussian filter (kernel 3x3) Detect contour Mobile Analyze Filter contour Not rectangle Inappropriate area

Detection result

outline imTop overview imTop detection Finger Mobile Finger detection evaluation Mobile detection improvement

Single-tapping test Fixed Target Appear sequentially Calculate accuracy Variation between Target center & finger detection point

Finger-moving test Demo a video of moving a target Tell user try to move this target as demo video Record finger detection point Calculate variation between target center & finger detection point +

outline imTop overview imTop detection Finger Mobile Finger detection evaluation Mobile detection improvement

Microsoft’s tag