Shedding Light on the Weather Srinivasa Narasimhan and Shree Nayar Computer Science Department Columbia University IEEE CVPR Conference June 2003, Madison, USA Sponsors : DARPA Human ID, NSF

Computer Vision in Bad Weather Mild Haze Dense Haze Computed Dehazed Image 3D Visualization Removing Weather Structure from Weather Color Contrast Polarization Models Algorithms (Narasimhan , Nayar ’00) (Narasimhan , Nayar ’01) (Narasimhan , Nayar ’02) (Schechner et. al. ’01)

Light Sources in Bad Weather Mist Fog

Multiple Scattering in the Atmosphere Phase Function Incident Beam Particle Light Source A T M O S P H E R E Imaging Plane Glow Pinhole

Radiance Rate of Change Radiative Transfer Infinitesimal Scattering Volume : Direction Exiting Beam Radiance dR Incident Beam Radiance Extinction Radiative Transfer Equation : Radiance Rate of Change Source Function Phase Function Optical Thickness

Light Source in a Spherical Medium Spherical Radiative Transfer Equation : Phase Function Light Field Cosine of Angle Optical Thickness [ Chandrasekhar 1960 ] Scattered Isotropic Source Homogeneous Medium

Axially Symmetric Phase Functions Exiting Direction Incident Direction Legendre Polynomial Expansion : [ Ishimaru 1997 ] [ Henyey et al., 1941 ] Legendre Polynomial Forward Scattering Parameter

Light Source in a Spherical Medium Scattered Light Field Isotropic Source Homogeneous Medium Spherical Radiative Transfer Equation : [ Chandrasekhar 1960 ] Cosine of Angle Optical Thickness Light Field Phase Function

Analytic Multiple Scattering Solution Scattered Light Field : Legendre Polynomial Phase Function Parameter Optical Thickness Exponential Coefficients : Source Radiant Intensity

Highlights of the Model Single and Multiple Scattering Absorbing and Purely Scattering Media Isotropic and Anisotropic Phase Functions 1.02 1.2 1.4 1.6 1.8 T m 160 120 60 30 10 Small Number of Coefficients (m) :

Scattered Light Field vs. Weather Condition Angular PSF : Scattered Light Field at a Point Mild Weather (T = 1.2) Dense Weather (T = 4)

Validation : Multiple Scattering in Milk Image acquired With No Milk Original Milk Images Increasing Milk Concentrations Rendered Milk Images

Model Fit Accuracy Low Milk Concentration High Milk Concentration Number of Milk Concentrations : 15 Model Fitting Error : [ 1 % to 3 % ] Diffusion Fitting Error : [ 20 % to 50 % ]

Rendering Glows using Convolution Increasing Fog Rendered Images Original Image Joint work with Ramamoorthi (submitted to TOG)

Single versus Multiple Scattering Single Scattering Original Image Multiple Scattering (Mild Condition) Multiple Scattering (Dense Condition) Joint work with Ramamoorthi (submitted to TOG)

Inverse RTE : Weather from APSF Measured APSF : Objective Function : Meteorological Visibility : [ Middleton 1952] Weather Condition : [ Van de Hulst 1957] 1 Pure Air Small Aerosols Haze Mist Fog Rain 0.1 0.4 0.7 0.9 0.8

Atmospheric Visibilities A Camera-based Weather Station 45 images of a light source (WILD Database ECCV 02) Computed Atmospheric Visibilities Estimated Ground Truth Computed Weather Conditions Estimated Ground Truth

Active Visibility Meter for Ground Truth

Summary Analytic Multiple Scattering Model Validation using Milk Volume Rendering as Convolution Shedding Light on the Weather

Prior Work on Radiative Transfer [ Chandrasekhar 1960 , Ishimaru 1997 ] Sun Distant Source Plane Parallel Medium Scattered Light Field Our Problem : Divergent Source Inside Medium

Scattering Space of Weather Conditions Phase Function Parameter, q Single Scattering 1 Diffusion Pure Air Small Aerosols Haze Mist Fog 10 Optical Thickness, T

Axially Symmetric Phase Functions Exiting Direction Incident Direction Legendre Polynomial Expansion : [ Ishimaru 1997 ] Legendre Polynomial Henyey – Greenstein Function : [ Henyey, Greenstein 1941 ] Forward Scattering Parameter

Effect of Source Visibility 315 240 180 90 30 o Increasing Milk Concentrations Observed Milk Images

Scattering is Everywhere… Computer Vision Astronomy Oceanography Medical Imaging Computer Graphics