1. International Conference on Image Analysis and Recognition (ICIAR’09). Halifax, Canada, 6-8 July 2009. Video Compression and Retrieval of Moving Object Location Applied to Surveillance William R. Schwartz, Hélio Pedrini, Larry S. Davis

2. International Conference on Image Analysis and Recognition (ICIAR’09). Halifax, Canada, 6-8 July 2009. 2 Introduction Proposed Method Experimental Results Conclusions and Remarks Organization

3. International Conference on Image Analysis and Recognition (ICIAR’09). Halifax, Canada, 6-8 July 2009. 3 Motivation:  Surveillance cameras capture hours of data that need to be store and analyzed. Introduction  Analysis is based mostly on moving objects (interaction between people in the scene).  Reduce redundancy + store objects.

4. Introduction Proposed method Experimental results Conclusions 4 Goals:  Develop a video compression technique that takes advantage of static cameras (surveillance).  Provide information regarding the location of moving objects (higher-level computer vision tasks). Key ideas:  Estimate eigenspaces for non-overlapping blocks.  Use a second encoding scheme to encode regions not well modeled by the eigenspaces. Introduction

5. Introduction Proposed method Experimental results Conclusions 5 Use of eigenspaces allow high compression ratio.  Store the projection vectors (obtained using PCA).  For a new frame, save only projection coefficients (in case of no changes or linear changes in the block). Advantages of Eigenspaces Block size Compression ratio (assuming float coefficients) 1 coefficient2 coefficients3 coefficients4 coefficients 8x81685.334 16x16643221.3316 32x3225612885.3364 64x641024512341.33256

6. Introduction Proposed method Experimental results Conclusions 6 Eigenspaces do not model non-linear changes (i.e. moving person). Use a second encoding method for these cases. Two-Stage Scheme

7. Introduction Proposed method Experimental results Conclusions 7 Segment the image area into non-overlapping blocks. Estimate eigenspaces for each block. For new frames, project blocks onto the eigenspaces and compute the reprojection error. If the reprojection error is acceptable, save the scores, otherwise set the image block to be encoded using MPEG-4. Proposed Method

8. Introduction Proposed method Experimental results Conclusions 8 For each block, sample frames free of non-linear transformations (≈ 200 frames). Estimate projection vectors P = {p 1,…, p k }. Estimate reconstruction error distribution δ p for each pixel in the block (used to locate moving objects). Learning the Eigenspaces

9. Introduction Proposed method Experimental results Conclusions 9 Compression Algorithm

10. Introduction Proposed method Experimental results Conclusions 10 The reprojection error is high when there are non- linear transformations within a block (moving objects / non-linear local illumination changes). Given that a block was compressed using MPEG- 4, check the pixels that do not satisfy the error distribution δ p to decide if there is a moving object on that block. Location of Moving Objects

11. Introduction Proposed method Experimental results Conclusions 11 Our method was tested on four video sequences. Compared to MPEG-4 and H.263 using MEncoder. Experiments:  Compression with a constant PSNR.  Moving object location accuracy. Experimental Results

12. Introduction Proposed method Experimental results Conclusions 12 Frames were converted to YCbCr. Using blocks of 16 × 16 pixels. Initial 200 frames were used to learn the eigenspaces. Number of PCA coefficients kept is estimated for each block (bounded by a maximum). Video Compression

13. Introduction Proposed method Experimental results Conclusions 13 Video Compression video sequence Size & # frames PSNR (dB) Compression Ratio MPEG-4H.263proposed camera 1768x288x269539.0033.7834.3337.27 camera 2720x528x533338.0040.4440.2745.06 station720x576x237039.5051.2749.94105.64 robbery720x480x332038.0034.7034.6661.14 camera 2 camera 1 robbery station

14. Introduction Proposed method Experimental results Conclusions 14 Easy to retrieve object location due to the encoding scheme used. Evaluation compares the location obtained by the method to the ground truth location. At a false positive rate of 0.025 obtained a false negative rate of 0.051. Moving Object Location

15. Introduction Proposed method Experimental results Conclusions 15 High compression rates:  Robust to linear transformations in illumination.  No need for saving key-frames. Fast to encode (≈ 5 fps in MATLAB). Useful for higher level computer vision tasks due to the storage of moving object locations. The use of multiple size blocks might increase the compression ratio. Conclusions and Remarks

16. Introduction Proposed method Experimental results Conclusions 16 Removal of Foreground Blocks Estimate initial eigenspace. Compute reprojection error for each frame. Compute the median and standard deviation. Remove frames with high error (∆ > m + cσ).