1. Vision System for Wing Beat Analysis of Bats in the Wild 1 Boston University Department of Computer Science 2 Boston University Department of Biology Mikhail Breslav 1, Nathan W. Fuller 2, and Margrit Betke 1 ComputerScience

2. Motivation Behavior Aerial Vehicles

3. Past Work Study Kinematics (Hubel. 2012) – Wind Tunnel – High Resolution Cameras 3D Tracking (Wu. 2009) – Outdoor environment – Model Bats as Points Behavior and Trajectory Analysis (Theriault. 2010, Fisher. 2010)

4. Goal Estimate Wing Beat Frequencies – Potential to improve tracking T Wing Beat Frequency: 1/T Hubel et al. 2012

5. Challenging Data Unpredictable Motions Relatively Low Resolution In FOV for a Short Time

6. Segmentation and Tracking Foreground / Background Estimation Kalman Filter

7. Shape-time Signals Output of Tracker Define “Shape”

8. Prototype Shapes Assumption – There are shapes that uniquely identify 3D poses for a given camera Example Currently chosen manually “up” “down” “neutral”

9. Main Idea A prototype shape is equal to a 3D pose Repeating prototype shapes in a shape-time signal Estimate Wing Beat

10. Shape Comparison Shape Distance – Shape Context Descriptor ( Belongie et al. 2002 ) Invariant to translation, scale, and optionally rotation – Hungarian Algorithm Establish Correspondences Estimate Wing Relative to Body with feature W

11. Shape Similarity Scores Use Shape Distance and Ratio W to assign similarity score – Also consider the ‘none’ hypothesis ‘None’.24.16.43.17

12. “up” “down” “neutral” Process Shape-Time Signal Find confident matches to prototype shapes

13. “up” Process Shape-Time Signal “down” “neutral” Time Axis

14. Fast Fourier Transform Time Axis “up” “down” “neutral” FFT

15. Fast Fourier Transform Periodicity Estimate of 9.76 Hz

16. Experimental Results 20 Bats Both Automatic and Manual estimates

17. Discussion Reasonable Estimates – Deviates from manual annotations by 1.3 Hz on average, standard deviation 1.8 Hz – Falls within 10-15 Hz as reported in biology literature (Foehring. 1984) Main Contribution – System for using shapes to estimate wing beat – First to do this for bats in the wild Vision based system

18. Future work Choosing prototype shapes – Automatically and Intelligently Understand the mapping between 2D shapes and 3D poses for a given model – Generalize across datasets Try more robust shape comparison measures

19. Thank You for your Attention!

20. Questions? Holding a bat!