An Effective & Interactive Approach to Particle Tracking for DNA Melting Curve Analysis 李穎忠 DEPARTMENT OF COMPUTER SCIENCE & INFORMATION ENGINEERING NATIONAL TAIWAN UNIVERSITY

DNA Melting Curve Analysis  Used for the detection of DNA sequence variants  DNA Melting Analysis in Temperature-Gradient Micro-channel Temperature-Gradient Micro-channel Heater Carrier (Bead/Droplet) Thermometer Substrate 1/54

DNA Melting Curve Analysis Temperature Fluorescent Intensity Melting Temperature 2/54

DNA Melting Curve Analysis 3/54

Motivation  People label each particles (carrier) frame by frame  That is time-consuming  We design an annotation tool to reduce human effort 4/54

Related Work  Particle tracking ParticleTracker: An ImageJ plugin for multiple particle detection and tracking [Sbalzarini et al., Journal of structural biology 2005] u-track [Jaqaman et al., Nature Methods 2008]  Interactive video annotation Tracking with active learning [Vondrick et al., NIPS 2011] Interactive object detection [Yao et al., CVPR 2012] 5/54

Proposed System User annotation Detection of bounding circle of the particle Acquisition of labels at other frames by tracking the particle User correction Update of tracker & labels Acquisition of all correct labels 6/54

Detecting Bounding Circle of a Particle Median filter Otsu's method Edge detection Least-squares fitting Dilation Erosion 7/54

Least-Squares Fitting of Bounding Circle 8/54

Least-Squares Fitting of Bounding Circle 9/54

Possible Choices of Trackers  Linear interpolation  Correlation filter based tracker [Zhang et al., ECCV 2014]  Normalized cross-correlation matching 10/54

Linear Interpolation /54

Linear Interpolation: User Correction /54

Linear Interpolation: Update of Labels /54

Linear Interpolation: Update of Labels /54

Linear Interpolation: User Correction /54

Correlation Filter Based Tracker [Zhang et al., ECCV 2014] 16/54

Online Update of Filter Frame 1 17/54

Online Update of Filter Frame 2 18/54

1 2 One-Way Method 19/54

1 2 One-Way Method 20/54

1 2 One-Way Method 3 21/54

1 2 One-Way Method 3 22/54

Two-Way Method 23/54

Two-Way Method 4 24/54

Two-Way Method /54

Two-Way Method /54

Two-Way Method /54

Two-Way Method /54

Two-Way Method 29/54

Normalized Cross-Correlation Matching  Given a image f and template t, normalized cross-correlation (NCC) measures the similarity between each part of f and t: TemplateInput imageOutput NCC 30/54

Normalized Cross-Correlation Matching Template Frame 1 31/54

Normalized Cross-Correlation Matching Frame 2 32/54

1 2 One-Way Method 33/54

1 2 One-Way Method 34/54

1 2 One-Way Method 3 35/54

1 2 One-Way Method 3 Update the template 36/54

Two-Way Method 37/54

Failure in Tracking with Normalized Cross-Correlation Template of particle /54

Combining NCC & Extrapolation Frame t-2Frame t-1Frame t x x x 39/54

Combining NCC & Extrapolation NCCScore of predicted location Combined score 40/54

Experiments  Evaluate how much human effort our system can reduce  Simulate the process of annotating video with our system  Evaluation metric Number of manual annotation  Count a tracked bounding box as a correct label if the distance between the centers of it and the ground-truth bounding box is not more than 10 pixels 41/54

Methods  Interp  CF-1way  CF-2way  NCC-1way  NCC-2way  NCC-Extrap-1way  NCC-Extrap-2way 42/54

The Order of Labeling  For those methods not restricting the order of labeling Always correct the label with maximum center location error  For other methods Same as the video display order 43/54

Video Dataset Name# frames# particles# annotations Droplet Droplet Bead  Video Droplet 1 is for parameter tuning which is performed using brutal force search 44/54

Parameter Tuning for CF- 1way 45/54

Parameter Tuning for CF- 1way 46/54

Parameter Tuning for NCC-Extrap-1way 47/54

Parameter Tuning for NCC-Extrap-1way 48/54

Result Droplet2 (# annotations = 4192) Bead (# annotations = 727) Interp457 (10.90%)88 (12.10%) CF-1way1475 (35.19%)79 (10.89%) CF-2way1973 (47.07%)112 (15.41%) NCC-1way56 (1.34%)11 (1.51%) NCC-2way129 (3.08%)21 (2.89%) NCC-Extrap-1way53 (1.26%)9 (1.24%) NCC-Extrap-2way115 (2.74%)20 (2.75%) 49/54

Error Analysis for NCC-Extrap-1way 50/54

Error Analysis for NCC-Extrap-1way 51/54

Error Analysis for NCC-Extrap-1way 52/54 Target Error

Conclusions  We designed a system for particle annotation in video sequences  Our system can reduce human effort in annotation  Combining NCC and extrapolation achieves the best result  It is better to annotate video in its display order  Future work Use polynomial curve fitting to predict the location of particle in the next frame 53/54

Thank you for listening