1. When Does a Camera See Rain? Department of Computer Science Columbia University Kshitiz Garg Shree K. Nayar ICCV Conference October 2005, Beijing, China Sponsors: NSF, DARPA

2. When is Rain Visible? Rain [Garg and Nayar, ICCV 05] Visibility of Rain Depends Strongly on Camera Parameters ( Exposure time, Aperture size, Focus distance )

3. Defining the Visibility of Rain Pixel Intensity Intensity Time (Frames) Visibility: Variance due to a Volume of Rain

4. Intensity of a Rain Streak background Background zmzm Rz m Fog like Rz m zmzm Noise Level Distance Change in Intensity Dependence on Defocus Dependence on Exposure Time Rain Region [Garg and Nayar, ICCV 05] Streak’s width Radius of blur circle Dependence on Distance

5. Visibility and Volume of Rain Background Number of drops Intensity Change Streak Width Variance Layer of Rain: Volume of Rain: Rain Properties Camera Properties Scene Properties [Garg and Nayar, ICCV 05]

6. Experimental Verification of Rain Visibility Standard Deviation Standard deviation 0246810 1 2 3 F-number (N) Exposure time Visibility 01020304050 1 2 Distance of Focus Plane z 0 (m) (ms) 04080 120 1 4 8 Visibility Theoretical Visibility Measured Visibility (with error bars) Camera Parametes F-num (Aperture) Exposure Time Focus Distance zoom

7. Scene Near Depth Exposure F-number Motion Distance Range (m) Time (ms) slow close large 66 14 small 33 4.4 far large 33 6 small 33 2 fast close large X X small X X far large 8 6 small 8 2.4 Optimal Settings for Rain Removal Slow: Motion< 15 pixels/secClose: z < Rain Visible RegionSmall: D z < Rain Visible Region

8. Scene with Fast Motion Default camera settings F-number =12 Exposure time = 8 ms Focus plane = 100 m

9. Scene with Fast Motion F-number decreased from 12 2.4 F-number = 2.4 Exposure time = 8 ms Focus plane = 100 m

10. Comparison: Scene with Fast Motion Default camera settingsF-number : 12 2.4

11. Scene with Large Depth Default camera settings F-number =14 Exposure time = 16 ms Focus plane=10 m

12. Scene with Large Depth Exposure time Increased from 16 66 ms F-number =14 Exposure time = 66 ms Focus plane=10 m

13. Comparison: Scene with Large Depth Default camera settingsExposure time: 16 66ms

14. Panning Video in Rain Default camera settings Exposure time =16 ms F-num=14 Optimal camera settings Exposure time =33 ms F-num=6

15. Implications for Vision Algorithms Default camera settings Exposure time =16 ms F-num=14 Optimal camera settings Exposure time =33 ms F-num=6 [KLT feature tracker code from Stan Birchfield]

16. Enhancing the Visibility of Rain: Rain Gauge Moderate Rain VideoHeavy Rain Video Type of RainfallCamera parameters (f, N, z0, T) Measured rain rate mm/hr Reported rain rate* mm/hr (a) Light(7095, 3.8, 3, 16)1.340.763 (b) Moderate(5148, 4.4, 3, 4)3.2152.28 (c) Heavy(3300, 1.8, 3, 4)14.2811.17

17. Summary Contributions Future Work Camera Based Rain Gauge Analysis of Visibility of Rain Camera Parameters for Removal of Rain Limitations A Camera-Lens system that automatically detects rain in videos and sets optimal camera parameters for reducing rain during image acquisition Not as effective in very heavy rain and for scenes with close by and fast-moving objects