Dynamic Time Warping for Automated Cell Cycle Labelling A. El-Labban, A. Zisserman University of Oxford Y. Toyoda, A. Bird, A. Hyman Max Planck Institute of Molecular Cell Biology and Genetics

Objectives Segment and track mitotic cells Label mitotic phases Fully automated system Interphase Prometaphase Anaphase Prophase Metaphase Telophase

Data 3D time lapse image stacks Use max intensity z-projections 1-5 minute temporal resolution 0.2 micron xy-resolution

Approach Existing approaches (e.g. Harder et al. 2009, Held et al. 2010 [CellCognition]): Track cells Label cell cycle phase frame-by-frame Smooth result with HMM (CellCognition) Our Approach: Label all frames by using temporal signals of features

Temporal signals of features

Temporal signals of features Interphase Prometaphase Anaphase Prophase Metaphase Telophase

Overview Part I Track cells in videos Part II Label mitotic phases

Part I – Tracking

Tracking Tracking by detection Detect first, then associate objects Here we use detection by classification.

Segmentation: Our approach Logistic regression classifier Graph Cuts Logistic regression classifier Graph Cut Input image Probability map Binary map

Logistic Regression Classifier Feature: 10 bin intensity histogram in 5x5 window around pixel Non-uniform bins Get local neighbourhood information as opposed to single pixel Histogram gives rotational invariance

Logistic Regression Gives a probability map:

Uses local neighbourhood information to make decisions Graph Cuts   Probability from Logistic Regression Classifier Gradient dependent pairwise term Uses local neighbourhood information to make decisions Pairwise term penalises different labels for adjacent pixels

Graph Cuts

Tracking Associate objects based on distance between centroids in consecutive frames. Easy given segmentation and slow movement of cells.

Tracking Associate objects based on distance between centroids in consecutive frames. Easy given segmentation and slow movement of cells.

Tracking Associate objects based on distance between centroids in consecutive frames. Easy given segmentation and slow movement of cells.

Tracking

Part II – Phase Labelling

Simple features Maximum Intensity: Interphase

Simple features Maximum Intensity: Interphase Prophase

Simple features Maximum Intensity: Interphase Prometaphase Prophase

Simple features Maximum Intensity: Interphase Prometaphase Prophase

Simple features Maximum Intensity: Interphase Prometaphase Anaphase Prophase Metaphase

Simple features Maximum Intensity: Interphase Prometaphase Anaphase Prophase Metaphase

Simple features Maximum Intensity: Interphase Prometaphase Anaphase Prophase Metaphase Telophase

Reference signal Average over training set (±1 standard deviation shaded):

Dynamic time warping Stretch signal onto labelled reference:

Dynamic time warping Stretch signal onto labelled reference:

Dynamic time warping Interphase Prometaphase Anaphase Interphase Prophase Metaphase Telophase

Dynamic time warping Find a cost matrix of pairwise distances between points on the two signals Find minimum cost path through matrix Test Signal Reference Signal

Features Use 3 features and their gradients at two different scales: Maximum intensity Area Compactness ( 𝑎𝑟𝑒𝑎 𝑝𝑒𝑟𝑖𝑚𝑖𝑡𝑒𝑟 2 )

Hidden Markov Model Hidden states, x Observations, y Mitotic phases Observations, y Features Transition probabilities, a From one phase to the next Emission probabilities, b Of features having a given value in a given phase Image: http://en.wikipedia.org/wiki/Hidden_Markov_model

Hidden Markov Model DTW essentially a special case of HMM Easy to extend approach Can add other classes e.g. cell death Split phases into sub-phases to account for variation

Experiments and Data 54 movies 119 mitotic tracks 27 movies (61 tracks) training, 27 movies (58 tracks) testing

Results Interphase Prophase Prometaphase Metaphase Anaphase Telophase

Results

Outputs

Outputs Synopsis video1 of mitotic cells Aligned to start of anaphase 1Rav-Acha et al., 2006

Conclusions Novel approach to cell cycle phase labelling Utilises temporal context Extendable with HMM

Questions?