1. Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 21, NO. 6, JUNE 2011

2. Outline Background Model Construction – Block-Based Background Subtraction – Pixel-Based Background Subtraction Hierarchical Foreground Detection Background Models Updating with the Short-Term Information Models Experimental Results

3. Background Model Construction This method involves two types of codebooks(CBs), block-based and pixel-based CBs. The modeling of two CBs is similar to the former CB[14] [14] K. Kim, T. H. Chalidabhongse, D. Harwood, and L. Davis, “Real-time foreground- background segmentation using codebook model,” Real- Time Imaging, vol. 11, no. 3, pp. 172–185, Jun. 2005.

4. Background Model Construction

5. There are two different time intervals for training (x t ). (1 ≤ t ≤ T) and (t > T) for training the background models and foreground detection. The updating algorithms are separated into two parts for different time zones.

6. The Features Used in Block-Based Background Subtraction A frame x t of size P x Q is divied into multiple non-overlapped blocks of size M x N. The former block truncation coding(BTC) reduce the frame into two means,high-mean and low-mean. In this paper,we have four means to represent a frame, high-top mean (μ ht ), high- bottom mean (μ hb ), low-top mean (μ lt ), and low-bottom mean (μ lb ).

7. The Features Used in Block-Based Background Subtraction

8. Each means have three colors(RGB),so each codebook have 12 dimensions.

9. Updating Block-Based Background Models (CBs) in the Training Phase a specific block can be represented as a vector V b = {v b t |1 ≤ t ≤ T }. A CB for a block can be represented as C = {c i |1 ≤ i ≤ L}, consisting of L codewords An additional weight w i is geared for indicating the importance of the ith codeword. Codebook size is (P/M)x(Q/N)

10. Updating Block-Based Background Models (CBs) in the Training Phase

12. Updating Pixel-Based Background Models (CBs) in theTraining Phase The same as block-based method. Codebook size is P x Q. Each codebook is 3 dimensions (RGB)

13. Hierarchical Foreground Detection After the background models training as indicated before the time point T, the two CBs are applied to the proposed hierarchical foreground detection. The foreground is obtained by background subtraction.

14. Foreground Detection with the Block- Based CB the input vector (v b t ) extracted from a block is compared with the ith block-based codeword (c i ) to determine whether a match is found When a v b t is classified as background, the corresponding block is also used to update the pixel-based CB.

15. Foreground Detection with the Pixel- Based CB This subsection introduces how to classify a pixel in a block to foreground or background. The foregrounds are classified into one true foreground and two fake foregrounds (shadow and highlight).

16. Foreground Detection with the Pixel- Based CB

18. Background Models Updating with the Short-Term Information Models an additional variable time i cs is involved to store the updated time for estimating whether the corresponding ith codeword (c i s ) has been updated for a specific period or not. If the duration is longer than a predefined parameter D s delete, the corresponding c i s is simply a temporary foreground.

19. Background Models Updating with the Short-Term Information Models When c i s, is favor to strong stationary ( w i cs ≥ D add ), the short-term information model can be considered as a part of the true background model. This additional value is employed for filtering out c i which meets the states of eventually moving as foregrounds with the predefined parameter D delete.

20. Experimental Results λ = 5 for block-based, λ = 6 for pixel-based, η = 0.7, θcolor = 3, β = 1.15,γ = 0.72,D update = 3, and α = 0.05, D add = 100, D s delete = 200, and D delete = 200

21. Experimental Results [9]MOG [5]color model [11][25] hierarchical MOG [14]CB [9] C. Stauffer and W. E. L. Grimson, “Adaptive background mixture models for real-time tracking,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2. Jun. 1999, pp. 246–252. [5] R. Cucchiara, C. Grana, M. Piccardi, and A. Prati, “Detection moving objects, ghosts, and shadows in video streams,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 10, pp. 1337–1342, Oct. 2003. [11] Y.-T. Chen, C.-S. Chen, C.-R. Huang, and Y.-P. Hung, “Efficient hierarchical method for background subtraction,” Pattern Recognit., vol. 40, no. 10, pp. 2706–2715, Oct. 2007. [25] C.-C. Chiu, M.-Y. Ku, and L.-W. Liang, “A robust object segmentation system using a probability-based background extraction algorithm,” IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 4, pp. 518–528, Apr. 2010. [14] K. Kim, T. H. Chalidabhongse, D. Harwood, and L. Davis, “Real-time foreground-background segmentation using codebook model,” Real- Time Imaging, vol. 11, no. 3, pp. 172–185, Jun. 2005.

22. C)MOG d)Color model e)CB f)g) hierarchical MOG

23. C)MOG d)Color model e)CB f)g) hierarchical MOG

24. C)MOG d)Color model e)CB f)g) hierarchical MOG

26. Experimental Results

29. Conclusion The block-based stage can enjoy high speed processing speed and detect most of the foreground without reducing TP rate. Pixel-based stage can further improve the precision of the detected foreground object with reducing FP rate. Short-term information is employed to improve background updating