1. Temporal Video Denoising Based on Multihypothesis Motion Compensation Liwei Guo; Au, O.C.; Mengyao Ma; Zhiqin Liang; Hong Kong Univ. of Sci. & Technol., Clear Water Bay Circuits and Systems for Video Technology(CSVT), IEEE 2007

2. Outline Introduction Video Signal Model With Multihypothesis MC Multihypothesis Motion Compensated Filter (MHMCF) – The Proposed Linear Temporal Filter – MHMCF – Implementation Issues – Performance Analysis Experimental Results Conclusions

3. Introduction Spatial correlation denoising: 2-D Kalman filter [1], 2-D Wiener filter [2], wavelet shrinkage [3], non-local means [4] etc. Until now there are few temporal denoising methods presented in the literature. These temporal predictions, defined as its motion-compensated hypotheses for the current pixel.

4. Video Signal Model With Multihypothesis MC We present a novel model of residue(z m ) for multihypothesis MC: Let the mean and the variance of be and respectively. We propose a linear model for this relationship: f : the current pixel of F k P m : the motion compensated prediction of f from F k-m

5. Video Signal Model With Multihypothesis MC

6. For video with large motion, the correlation tends to decrease faster than small motion. Large b implies video with large motion. Large a implies texture regions.

7. Multihypothesis Motion Compensated Filter (MHMCF) The Proposed Linear Temporal Filter – MHMCF Implementation Issues – Motion Estimation – Parameters Estimation Performance Analysis

8. -The Proposed Linear Temporal Filter – MHMCF Assumptions: – Video sequence is contaminated by additive zero- mean random noise. – The noise source is stationary over spatial and temporal domain, and independent of residue(z m ). The noise-corrupted video signal f’ and p’ m : We propose MHMCF to estimate the current pixel f :

9. For simplicity, we rewrite (3) as: We define the objective function of MHMCF: Minimizing is equal to Minimizing [16, p. 273]. -The Proposed Linear Temporal Filter – MHMCF Random varianbles Let

10. By = 0 and = 0, the optimal w and d that minimize are: As z m and n m are independent, and g m is independent with each other: -The Proposed Linear Temporal Filter – MHMCF

11. The optimal w and d that minimize are: Large implies low temporal correlation. When, then d = 0, w 0 = 1, w m = 0, and no filtering will be applied. -The Proposed Linear Temporal Filter – MHMCF

12. -Implementation Issues Motion Estimation: – MHMCF needs to perform ME with respect to every reference frame. – Fast ME algorithm, PMVFAST [17], is employed. – Experiments show that PMVFAST compared to full search, about 1% denoising error is increased. Parameters Estimation: ?

13. -Implementation Issues Parameters Estimation: – : We select the minimum 3% out of the total block variances (their average is ) : – and : Let be the noisy residue. Then ( ), since n 0 and n m are all zero-mean: As g m and n 0 are independent:

14. -Performance Analysis The estimation error : MHMCF is an unbiased estimator leading to and error variance. Combining (3), (6), (9),(11), and (12): – We have the model of residue variance. – The remaining noise in the reference frame is the estimation error: ?

15. -Performance Analysis Small motion Smooth regions

16. -Performance Analysis

17. Experimental Results Denoising Performance: JNT[9]STVF[10] WRSTF[11]

18. Experimental Results

20. Computational Complexity : – In terms of the number of ADD and MUL performed to process a frame in CIF resolution (352 288).

21. Conclusions A temporal denoising filter MHMCF is developed for the removal of noise in video. MHMCF has very good noise suppression capability while using fewer inputs than other proposed filters. MHMCF is a purely temporal filter, spatial blurring is avoided and most spatial details could be preserved.