Removing Weather Effects from Monochrome Images Srinivasa Narasimhan and Shree Nayar Computer Science Department Columbia University IEEE CVPR Conference December 2001, Hawaii, USA Sponsors DARPA HID, NSF

How does scene contrast degrade in bad weather ? How can scene contrast be restored from bad weather images ? Contrast Degradation in Bad Weather RainFog

Weather Effects are Depth Dependent Image Processing Does Not Suffice Histogram Equalized Images

Prior Methods for Contrast Restoration Yitzhaky, Kopeika [98] Oakley, Tan, Satherley [98,01] Nayar, Narasimhan [99] Narasimhan, Nayar [00] Scene Depth Weather Information Required Predicted Weather PSF Required Computed Wavelength Independent Scattering Not Required Method Clear-day Scene Intensity/Color Computed Gaussian distribution assumed Computed (Color Images Required) OUR GOAL : ComputedNot Required Computed

Scattering Models : Attenuation and Airlight Object Observer d Attenuation Sunlight Diffuse Skylight Diffuse Ground Light Airlight

Contrast Degradation in Bad Weather Irradiance = Attenuation + Airlight + = Horizon Brightness Depth Reflectance Scattering Coefficient Contrast Decay : Exponential in Scene Depth Contrast between Iso-Depth points, P and P : (1)(2)

Mild Fog Denser Fog Depth Edge Reflectance Edge Depth Edges vs. Reflectance Edges Normalized SSD of Reflectance Edge Neighborhood Normalized SSD of Depth Edge Neighborhood

Edge Classification from Weather Changes Mild Fog Denser Fog Edge Classification Reflectance Edge : Depth Edge :

Scene Structure from Weather Changes Irradiance under versus Irradiance under : Linear Scaled Depth : All Scene points at Depth 1 All Scene points at Depth 2

Depth Map from Two Weather Conditions Mild Fog, 5 PM (Input) Denser Fog, 5: 30 PM (Input) Computed Depth Map (Output) Comparing with Prior Methods: Color Images Not Needed Works for Wider Range of Weather Conditions

Weather Removal Using Scene Structure Contrast Restored Image (Output) Dense Fog, 5:30 PM (Input) Computed Depth Map (Input) Histogram Equalized Image (For comparison)

Different amounts of Fog removed from different Depths

A De-Weathering System Detect Significant Weather Change Video Frame (Weather 1) Video Frame (Weather 2) Scene Structure System Initialization : Computing Scene Structure Continuous De-Weathering Using Scene Structure Scene Structure Contrast Restored Video Frame