1. Tracking-dependent and interactive video projection (Big Brother project)

2. Team Pierre Bretéché Matei Mancas Jonathan Demeyer Thierry Ravet Donald Glowinski Gualtiero Volpe

3. System Architecture

4. Summary Robust tracking Modeling human motion attention of a visual scene

5. Summary Robust tracking Tracking should be robust to severe illumination changes ! Modeling human motion attention of a visual scene

6. Robust tracking: data acquisition Color tracking of red hats IR tracking of lights on the top of the red hats

7. Robust tracking: data fusion Projective transform

8. Robust tracking: data fusion Selected points (Matlab) Transformation matrix (EyesWeb) Image warping (EyesWeb)

9. Robust tracking: data fusion RED: color tracking / GREEN : IR tracking / BLUE : Final fused tracking

10. Summary Robust tracking Modeling human motion attention of a visual scene

11. Summary Robust tracking Modeling human motion attention of a visual scene Regions of interest should be dynamically highlighted and visual effects corresponding to outstanding events should be displayed

12. Attention: in space (blob speed) HOT RED : most important / DARK RED : less important Global contrast !

13. Attention: in time (blob speed) HOT RED : really interesting / BLACK : really boring Rarity on 4 seconds

14. Attention: in time (quantity of motion) HOT RED : really interesting / BLACK : really boring Rarity on 4 seconds

15. Conclusion Summary : Multi-blob real time tracking (on IR images and color images) Image registration using projective warping Data fusion using a weighted combination of blob positions based on confidence levels for each modality Demonstration of motion attention in space (instantaneous) and in time (short- time memory) -> motion is not necessarily salient : it depends on the context Use of blob speed and silhouette quantity of motion features So we worked hard...

16. Conclusion... but we will work even harder : Refinement of confidence level for each modality Better hardware is needed: adapted optics for the cameras, and a color camera with the same characteristics than the IR camera: easy modality registration A computer with several firewire ports: easy modality synchronization Use of other features for attention for tracked paths (direction, acceleration, direction variation, trajectory curvature, …) and for gesture expressivity (energy, internal/external quantity of motion, …) Long-time memory attention (flickering lights, commonly used paths, …) to get higher level motion segmentation More efficient instantaneous attention (space) which works also with moving cameras and for surprising behaviors in crowds …

17. Thank you !