24/10/02AutoArch Overview Computer Vision and Media Group: Selected Previous Work David Gibson, Neill Campbell Colin Dalton Department of Computer Science University of Bristol

24/10/02AutoArch Overview Duck: The Automatic Generation of 3D Models Generating 3D computer models is difficult Put object on turntable Take 8 pictures of it from different angles Crank the handle… No skilled user or expensive equipment Make avatars by spinning person on chair

24/10/02AutoArch Overview

24/10/02AutoArch Overview Cog and Stepper Automatically inject life into computer animations 3D swathe through 4D space time Where space is 3D computer model Or just to make things look strange!

24/10/02AutoArch Overview

24/10/02AutoArch Overview

24/10/02AutoArch Overview Casablanca: Motion Ripper Computer animation driven by film Animator labels a small number of points System then tracks these points over all frames Motions are extracted and used to drive animation

24/10/02AutoArch Overview

24/10/02AutoArch Overview Laughing Man Motion Ripper Part 2 Automatic video creation Points are marked and tracked System learns the motions System generates new motions which are different but correct Forever!

24/10/02AutoArch Overview

24/10/02AutoArch Overview AutoArch: The Automatic Archiving of Wildlife Film Footage David Gibson, Neill Campbell David Tweed, Sarah Porter Department of Computer Science University of Bristol

24/10/02AutoArch Overview Motivation BBC Natural History Unit Manual archiving/meta data generation Reuse problematic –Inefficient/time consuming –Expensive –Limited access Obvious need to automate

24/10/02AutoArch Overview Objectives Generate efficient visual representations –Video segmentation –Visual browsing/summarisation –Visual searching Generate as much meta data automatically –Camera motions/effects –Scene structure –Scene content

24/10/02AutoArch Overview System Overview Shot Segmentation Visual Summarisation Motion Analysis Colour/Texture Analysis Meta data extraction algorithms Catalogue Entry Visualisation based algorithms Visualisation and Searching

24/10/02AutoArch Overview Video Segmentation

24/10/02AutoArch Overview Visual Summarisation Key frame extraction

24/10/02AutoArch Overview Visual Summarisation Tree Entire shot Level of detail

24/10/02AutoArch Overview Visual Searching Layered 2D representation of high D clip space

24/10/02AutoArch Overview Motion Analysis using point tracking Camera Motion Estimation Event/Area of Interest Detection Gait Analysis Foreground/Background Separation Combine with Colour and Texture for Classification See cheetah track avi

24/10/02AutoArch Overview Camera Pan BCD _0085.eps lines = 47, curls = 98, shorts = 5 long lines = 47, mode = 95.00, mean = 95.21, std = 4.15 zoom centre = (603.01, 63.65), val = zoom residual per line = zoom residual #2 per line = Average line vector: pan/tilt angle: 94.30, vector: ( ) pan/tilt residual per line = pan/tilt residual #2 per line = percentage of lines within 5% of mode: 89.36

24/10/02AutoArch Overview Camera Zoom BCD _0067.eps lines = 142, curls = 1, shorts = 7 long lines = 134, mode = , mean = , std = zoom centre = (182.97, 55.52), val = zoom residual per line = 4.86 zoom residual #2 per line = 6.90 Average line vector: pan/tilt angle: , vector: ( ) pan/tilt residual per line = pan/tilt residual #2 per line = percentage of lines within 5% of mode: 17.16

24/10/02AutoArch Overview Tracking Failure This could be an interesting event in its self: flocking, herding, close up of lots of activity, shot grouping, etc.

24/10/02AutoArch Overview Event/Area of Interest Detection

24/10/02AutoArch Overview Frequency Analysis: Gait Detection FFT After trajectory segmentation

24/10/02AutoArch Overview Foreground/Background Extraction Feature space #1 Feature space #2 Foreground model Background model Which pixels are foreground?

24/10/02AutoArch Overview Animal Identification Give models a name: = cheetah = elephant = zebra = lion

24/10/02AutoArch Overview Some Problems Noise in images Noise in measurements Camouflage Occlusion Answer: Need higher level models See next few slides

24/10/02AutoArch Overview Model Based Tracking

24/10/02AutoArch Overview Lion Tracking Synchronise horse model with lion points Move and deform horse model to lion points See avi To do: Improve spatial deformation, especially for legs, using colour and texture

24/10/02AutoArch Overview Multiple Object Tracking

24/10/02AutoArch Overview Conclusions Visualisation is very powerful Combined with text is even better! Assists searching and communication Lots of meta data can be auto generated Assists archiving Help to prioritise manual archiving Can be applied to any visual media