1. Page 1© Crown copyright 2004 The use of an intensity-scale technique for assessing operational mesoscale precipitation forecasts Marion Mittermaier and Clive Wilson

2. Page 2© Crown copyright 2004 Outline 1.An intensity-scale technique (Casati et al. 2004) 2.Model output and data description 3.Value added by higher resolution for a severe flooding event (Boscastle, August 2004) 4. A modified sign-test statistic for highlighting persistent/prevalent errors at the monthly time scale. 5.Radar vs gauge as “truth” 6.Concluding remarks

3. 1. An intensity-scale technique ….. best illustrated with an example …. (from Casati, 2004)

4. Radar Model forecast from Casati (2004) Radar > 1 mmForecast > 1 mmBinary error image X > uX < u Y > u Hits a False Alarms ba+b Y < uMisses c Correct Rejections d c+d a+cb+da+b+c+d=n

5. Page 5© Crown copyright 2004 MSE skill score 1 0 -2 -3 -4 threshold (mm/h) spatial scale (km) [from Casati (2004)] Axes multiples of 2

6. Page 6© Crown copyright 2004 2. Model output and data description  Mesoscale version of the Unified Model (MES) runs 4 times a day at ~12 km over the UK (for Unified Model description see Davies et al., QJRMS, 2005)  Newly implemented 4-km model now runs twice a day over the UK (see Bornemann et al, this conference)  Radar-rainfall accumulations available on a 5 km x 5 km national grid  ~2700 rain gauges have been used to produce a daily gridded rainfall product also on a 5 km x 5 km grid

7. Page 7© Crown copyright 2004 3. Boscastle: the benefit of higher resolution? How does one assess added benefit?  Output from the MES and 4 km model isn’t directly comparable  Basis of comparison should ideally be the same. Solution: Average the 4 km model output to the 12 km grid and compare against the same 12-km averaged radar rainfall product. …. consider 6-hr rainfall between 12-18Z from the 00Z run … On 16.08.2004 over 180 mm were recorded by one gauge in a 5-hr period during a highly localised flooding event.

8. 12-18Z 4 km 00Z 6 hr rainfall MES 00Z 6 hr rainfall4 km 00Z avg 6 hr rainfall Error scale (km) 2x2x 16  x 1 mm 64 mm 2x2x 16  x Max radar = 44 mm 68 mm 7 mm 46 mm Rainfall threshold (mm)

9. Page 9© Crown copyright 2004  Distribution-free test as normality of errors can’t be assumed.  B = number of +ve skill scores for a given scale and intensity during a given time interval, e.g. 1 month.  Hypotheses:  H 0 : SS >= 0 (implicit positive and skillful)  H 1 : SS < 0 (less skill than a random forecast)  H 0 is rejected if b <= b n,   where B ~ bi(n, 0.5) for small samples (n < 40),  = 0.025  The value of (n – B) / n is shaded in intensity-phase space for each scale and intensity where H 0 is rejected. 4. A modified sign-test statistic

10. Page 10© Crown copyright 2004 Added benefit: comparison of prevalent errors at the monthly time scale  (sub-)“grid” scale errors are more prevalent at trace rainfall totals for the 4 km model  prevalent errors at twice and four times the MES grid length for thresholds > 16 mm are less for the 4 km model (captures large totals better) May 2005 MES vs radarMay 2005 4 km avg vs radar X X X X X X X X X X X X X X X X 48 km 32 mm

11. Page 11© Crown copyright 2004 5. Radar vs gauge as “truth”  Slight shifts in the distribution of prevalent errors at the monthly time scale  Overall pattern very similar  Radar-rainfall fields preferred as they are truly spatial with a greater observation frequency August 2004 MES vs radarAugust 2004 MES vs gauge X X X X X X X X

12. Page 12© Crown copyright 2004 6. Concluding remarks 1.The 4 km model contains much more detail (even when averaged to 12 km) 2.Detail does not necessarily equal accuracy! Raw model output needs to be averaged 3.Scale-intensity analyses show that the need for averaging is (almost) independent of grid length (there is always grid noise, regardless) 4.The difference between error analyses produced using radar (true spatial) and gauge (point-interpolated) fields is minimal. Recommend that radar fields are used also because of the high observation frequency.

13. Questions?

14. 12-18Z Radar 12Z 6 hr rainfall MES 12Z 6 hr rainfall4 km 12Z avg 6 hr rainfall Error scale (km) Rainfall threshold (mm) 19 June 2005 Flash flooding caused by thunderstorms over North Yorkshire

15. Haar Wavelet filter deviation from mean value mean value + + mean value on all the domain Casati et al., 2004, Met Apps

16. Page 16© Crown copyright 2004 Wavelets are locally defined real functions characterised by a location and a spatial scale. Any real function can be expressed as a linear combination of wavelets, i.e. as a sum of components with different spatial scales. Wavelet transforms deal with discontinuities better than Fourier transforms do Haar mother wavelet  1 0 1 2 4 n n+1 An intensity-scale technique using wavelets

17. wavelet decomposition of the binary error 1 0 Scale from Casati (2004)