Tracking Multiple Animals in Laboratory Experiments with Mices: Prelimary Results João Bosco O. Monteiro Dr. Hemerson Pistori (Advisor) Dr. Albert S. de Souza (Co-Advisor) GPEC - Research Group in Engineering and Computing

Summary Context and Motivation Context and Motivation Open Field Experiment Open Field Experiment Very Simple Tracking Techniques Very Simple Tracking Techniques Tools and First Experiments Tools and First Experiments Preliminary Results Preliminary Results Particle Filters Particle Filters Final Considerations Final Considerations

Context and Motivation TOPOLINO: Low cost, computer vision based, automation of animal behaviour analysis TOPOLINO: Low cost, computer vision based, automation of animal behaviour analysis Most of Current Solutions are Expensive and Platform Dependent: Ethovision, 2020 Plus, Any Maze Most of Current Solutions are Expensive and Platform Dependent: Ethovision, 2020 Plus, Any Maze Many interesting open problems (tracking similar deformable objects) Many interesting open problems (tracking similar deformable objects)

Open Field Experiment Anxiety-provoking situation: mices move around less and tend to stay near the walls Anxiety-provoking situation: mices move around less and tend to stay near the walls Variables: speed, distance travelled, location (center, wall) Variables: speed, distance travelled, location (center, wall)

Very Simple Tracking Thresholding and Binarization Thresholding and Binarization Fixed threshold choosen empirically Fixed threshold choosen empirically

Image Subtraction Image Subtraction Background = First Frame Background = First Frame Very Simple Tracking

Tools and First Experiments ImageJ (Java Based DIP Framework) ImageJ (Java Based DIP Framework) ImageJ Plugins: ImageJ Plugins: ParticleAnalyzer (Simple Connected Components – Blobs – Segmentation) ParticleAnalyzer (Simple Connected Components – Blobs – Segmentation) MultiTracker (ParticleAnalyser applied to multiple frames) MultiTracker (ParticleAnalyser applied to multiple frames) Three mices in Open Field Three mices in Open Field

Results Easy Situation – Simple Tracking Successful Simple Connected Components Analysis Failures

Predictive Filters Combine dynamic and observation models Combine dynamic and observation models Kalman – Linear and Gaussian Kalman – Linear and Gaussian Particle Filter – Less restrictive Particle Filter – Less restrictive PDF of Random State Variable = Weighted Samples (Non-parametric)

Particle Filters Deterministic Drift Stochastic Diffusion Observation Weighting Correction

Final Considerations Animal tracking – interesting challenges Animal tracking – interesting challenges Very simple techniques don’t work Very simple techniques don’t work Work in progress - Predictive filters applied to animal tracking Work in progress - Predictive filters applied to animal tracking Free-software based implementation Free-software based implementation Results will be incorporated by the SIGUS project: ML+DIP for Vision Based HCI Results will be incorporated by the SIGUS project: ML+DIP for Vision Based HCI Project Website (in Portuguese): Project Website (in Portuguese):