artificial PErception under Adverse CONditions: The Case of the Visibility Range LCPC in cooperation with INRETS, France Nicolas Hautière Young Researchers Seminar 2007 Brno, Czech Republic, 27 to 30 May 2007

Overview ADAS and adverse visibility conditions Visibility range under daytime fog  Modelling  Measurement methods  Experimental validation Under way applications Discussion and future works PErception under Adverse CONditions Nicolas Hautière, LCPC

ADAS and Adverse Visibility Conditions PErception under Adverse CONditions Nicolas Hautière, LCPC Sensor aspects To improve the operation range of exteroceptive sensors To qualify / adapt / stop the other driving assistances Human aspects To switch or to adapt the operation of vehicle lights (AFS) To adapt the speed according to the weather conditions (ISA) To detect the visibility conditions allows: Improve the safety ! [Hautière and Aubert, 2005a] Hautière, N. and Aubert, D. (2005). Onboard evaluation of the atmospheric visibility for driving assistance systems, Recherche Transports Sécurité, 87:  Let’s talk about daytime fog

Daylight Visibility Range under Daytime Fog PErception under Adverse CONditions Nicolas Hautière, LCPC Scattering Atmospheric veil Koschmieder’s Law: Let express the contrast of an object against the sky:  Contrast attenuation “the greatest distance at which a black object of suitable dimensions can be recognized by aday against the horizon sky” (CIE, 1987) For a black object (C 0 =1) and a visibility contrast threshold of 5%: Direct transmission

Exploitation of the Atmospheric Veil PErception under Adverse CONditions Nicolas Hautière Extraction of a region of interest Fitting of a measurement bandwidth V met = 50m Estimation of the meteorological visibility distance Measurement and derivation of intensity curve Extraction of the inflection point B&W camera z x f d  S X Y Z C y v u vhvh H M Road plane Image plane Assuming a flat road: v h horizon line, camera parameters Method: Instanciation of Koschmieder’s Law [Hautière et al., 2006a] Hautière, N., Tarel, J.-P, Lavenant, J. and Aubert, D. (2006). Automatic Fog Detection and Measurement of the visibility Distance through use of an Onboard Camera. Machine Vision Applications Journal, 17(1):8-20

Exploitation of Contrast Attenuation PErception under Adverse CONditions Nicolas Hautière, LCPC Estimation of the so-called mobilized visibility distance Range map obtained by “v-disparity” stereovision approach Visible objects are those having a local contrast above 5% Method: computation of the range to the most distant visible object f(x 1 ) f(x) C x,x1 (s) s x x1x1 F(s)  Evaluated contrast on F(s) is equal to 2C(s 0 ) where Local contrast measurement based on a binarisation method: [Hautière et al., 2006b] Hautière, N., Labayrade, R. and Aubert, D. (2005). Real-Time Disparity Contrast Combination for Onboard Estimation of the Visibility Distance. IEEE Transactions on Intelligent Transportation Systems, 7(2):

Video samples PErception under Adverse CONditions Nicolas Hautière, LCPC Daytime fog Twilight fog

Experimental Validation PErception under Adverse CONditions Nicolas Hautière, LCPC Development of a validation site Objectives :  To estimate V met thanks to the targets:  To obtain a ground truth,  To compare it with the in- vehicle methods  Triangle based pattern  Constant solid angle  5 fixed targets: d=65m 1mx1m d=98m 1.5mx1.5m d=131m 2mx2m d=162m 2.5mx2.5m d=195m 3mx3m  1 mobile targe: 0.5mx0.5m

Experimental Validation Content PErception under Adverse CONditions Nicolas Hautière, LCPC Sunny weather V met = 5000 m Light rain V met = 3400 mHaze V met = 2130 m Snow fall V met = 1000 m Fog V met = 255 m Thick fog V met = 61 m Sample images of the validation site

Experimental Validation PErception under Adverse CONditions Nicolas Hautière, LCPC Meteorological visibility estimation Mobilized visibility distance estimation Quantitative results [Hautière et al., 2006c] Hautière, N., Aubert, D., Dumont, E. and Tarel, J.-P. (2008). Experimental Validation of Dedicated Methods to In-Vehicle Estimation of Atmospheric Visibility. IEEE Transactions on Instrumentation and Measurement, 57(10),

Principle: reversal of Koschmieder’s law Assuming a flat world, we have: Under Way Applications PErception under Adverse CONditions Nicolas Hautière, LCPC Improved Road Departure Prevention  Enhancement of road markings extraction under adverse visibility conditions [Hautière and Aubert, 2005b] Hautière, N. and Aubert, D. (2005). Contrast Restoration of Foggy Images through use of an Onboard Camera, IEEE Conference on Intelligent Transportation Systems (ITSC’05), Vienna, Austria d 1 =28m d 2 =62m

Contrast enhancement of the road scene Iterative contrast restoration Under Way Applications PErception under Adverse CONditions Nicolas Hautière, LCPC Improved obstacle detection [Hautière et al., 2007] Hautière, N., Tarel, J.-P., Aubert, D. (2007). Towards Fog-Free In-Vehicle Vision Systems through Contrast Restoration. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’07), Minneapolis, USA

Objective: to compute an adequate speed according to the weather conditions based solely on a digital map and a camera One of the requirements: a mapping function between the driver and the sensor visions: PErception under Adverse CONditions Nicolas Hautière, LCPC Under Way Applications ISA and “risk mitigation” or “safety margin” [Hautière and Aubert, 2006d] Hautière, N. and Aubert, D. (2006). Visible Edges Thresholding: a HVS based Approach, International Conference on Pattern Recognition (ICPR’06), Hong-Kong, China

Discussion, Current and Future works We have presented two methods, issued from the ARCOS French project, to estimate the visibility range and some applications, Methods are being extended in the REACT project by the Mines de Paris to develop probe vehicles, Currently, we are adapting the methods to the use of fixed CCTV cameras in the SAFESPOT IP, In the future, we would like to apply our scientific processes to other adverse visibility conditions, like: PErception under Adverse CONditions Nicolas Hautière, LCPC Rain GlareNocturnal Fog  This is the heart of the FP7 PEACON proposal lead by LCPC/INRETS !

Acknowledgments PErception under Adverse CONditions Nicolas Hautière, LCPC I would like to acknowledge the contributions of my colleagues Didier Aubert, Jean-Philippe Tarel, Raphaël Labayrade, Benoit Lusetti, Eric Dumont from LCPC and INRETS, of Michel Jourlin from the University of Saint-Etienne, and of Clément Boussard from the Mines Paris.