Quality management, calibration, testing and comparison of instruments and observing systems M. Leroy, CIMO ET on SBII&CM

Quality management, some aspects  Explicit identification of uncertainty objectives  Representativeness of observations  Calibration and maintenance  Selecting instruments : knowing their characteristics –Testing –Intercomparisons  Composite observing systems  Value analysis  ISO is an international frame

Quality management process  Setting and documenting objectives  Defining associated indicators  Regular survey of indicators  Regular survey of users’ (customers) needs and satisfaction.  Correcting and improving

Identification of uncertainty objectives  Identification of users of observation data –Forecasters and climatologists –Few direct users of observation : special case is aeronautic users, with needs clearly defined in Annex 3 of ICAO.  Getting the users’ needs is not always easy –They are not always aware of the possible uncertainty and associated cost. –A value analysis is necessary to get a good compromise.  The CIMO guide (WMO doc n° 8) is a guide. –Annex 1B indicates users’ needs and achievable measurements (best state of the art).  The measurement uncertainty must be demonstrated. So realistic choices must be done.

An example  For his proprietary Radome network, mainly dedicated for real-time observation, Météo- France has set up the following objectives; some are less ambitious than the achievable measurement uncertainty stated by the CIMO guide. –0.5 hPa for pressure –6% for relative humidity –0.5°C for air temperature –10% for wind speed –5-10% for precipitation amount –5% for daily amount of solar radiation.

 For other objectives, additional care can be necessary  Example : US Climate Reference Network –Redundancy and cross check of measurements. –3 independent air temperature measurements. –Dual fence rain gauge. –Associated measurement of wind speed.  Such efforts cannot be made in each station of a climatological cooperative network.

Quality indicators  Availability of data in the (final) data base –Within a given limit of time : example 95% of expected surface observations available within 10 minutes. –Availability of data in the climatological data base : example 98% for the Radome network.  Measurement uncertainty –% of sensors calibrated in a stated delay : example 12 months nominal, 15 months maximum, objective is more than 90%. –% of sensors found outside an Acceptable User Limit : example hygrometers found with one (or more) control point more than 5% from the reference. These sensors are flagged as having drifted. –% of sensors for which the periodicity of preventive maintenance is respected. –% of large errors when compared to numerical models (monitoring)

Some examples

Knowing the characteristics of an instrument  Technological survey : reading publications, participating to conference and instrument exhibitions.  Testing of instruments : please, publish results and make them available.  Intercomparisons, WMO intercomparisons.  Read the manufacturers’ technical documentation, having in mind the common characteristics of instruments, for example, thru the CIMO guide.

Recent WMO intercomparisons  Radiosonde intercomparisons  WMO Laboratory Intercomparison of Rainfall Intensity gauges  WMO Field Intercomparison of RI gauges : May 2007-May 2008, Vigna di Valle, Italy  WMO Intercomparison of Thermometer Screens/Shields in conjunction with Humidity Measurements : 2007, Ghardaïa, Algeria

WMO Field Intercomparison of Rainfall Intensity Instruments  The laboratory Intercomparison was a first step, easier to handle than a field test.  Not all instrument types available on the market were tested in laboratory.  Looking for a possible site, Vigna di Valle, Italy (Italian Meteorological Service, ReSMA) was selected. August 2007  August 2008

Field Intercomparison  Call for instruments (March 2006)  24 instruments of different measuring principles were selected (54 proposed !).  Reference based on a set of high quality devices, inserted in a pit gauge.  Draft Rec. 4.2/2 : Procedure and reference instruments for field rainfall intensity intercomparisons

WMO Combined Intercomparison of Thermometer Screens/Shields, in conjunction with Humidity Measuring Instruments  Looking for a possible site in harsh conditions, Ghardaïa, Algeria was selected.  Desert region  Jan  Jan  Instruments’ calibration before and after

Screens & Hygrometers Intercomparison  Call for instruments (March 2006)  16 screens/shields’ types (29 proposed) and 11 hygrometers’ types(17 proposed) were selected.  Reference : Thygan  Data analysis : –International Organizing Committee –ISO standard (draft)

Conclusion  There are many other subjects and actions concerning quality measurement : –Quality control directly at the level of the measurement systems. –Quality control using spatial and temporal cross-check. –Representativeness of observations, due to the nearby environment of the measuring site. –…  Quality management and ISO certification force us to explicit the objectives of observing networks and the proofs demonstrating their achievement.