1. Intensity-scale verification technique
B. Casati, G. Ross and D.B. Stephenson (2004) “A New intensity-scale verification approach for the verification of spatial precipitation forecasts”, Meteorol Appl, vol 11, pp
Evaluate the forecast skill as a function of the precipitation intensity and the spatial scale of the error
NOTE: scale = single band spatial filter  features of different scales  feedback on different physical processes and model parameterizations
In the neighborhood based (fuzzy) verification, the scale is the neighborhood size (low band pass filter): as the scale increases the exact positioning requirements are more and more relaxed

2. Nimrod case study: intense storm displaced
Gridded data, square domain with dimension 2n
It can be applied to any meteorological field … however, it was specifically designed for spatial precipitation forecasts …

3. Intensity: threshold to obtain binary images (categorical approach)
Binary Analysis
Binary Error Image u=1mm/h 1 -1
Binary Forecast

4. Scale: wavelet decomposition of the binary images
Scale l=8 (640 km)
Scale l=1 (5 km)
mean (1280 km)
Scale l=6 (160 km)
Scale l=7 (320 km)
Scale l=5 (80 km)
Scale l=4 (40 km)
Scale l=3 (20 km)
Scale l=2 (10 km) 1 -1

5. Intensity-scale skill score
For each threshold and scale component: skill score associated to the MSE of binary images ( = HSS)
Skill versus random chance, equally partitioned across the scales 1 -1 -2 -3 -4
SSu,l

6. Links with categorical verification
Binary Analysis
Binary Rec.Forecast
Overlapping
X > u
X < u
Y > u
Hits a
False Alarms b
a+b
Y < u
Misses c
Correct Rejections d
c+d
a+c
b+d
a+b+c+d=n

7. Strenghts
Categorical approach  robust and resistant
Wavelets  cope with spatially discontinuous fields characterized by the presence of few sparse non-zero features
 suitable for spatial precipitation forecasts
Weaknesses
need gridded data on a square domain with dimension 2n : work in progress …

8. Intensity-scale verification technique summary
evaluate the skill as function of precipitation intensity and spatial scale of the error
it is capable of isolating specific IS errors (e.g. displaced storm  negative minimum in skill for 160 km scale)
in general small scales have negative skill (small scale displacements) and large scales have positive skill
bridges categorical approaches (joint distribution) and scale verification approaches (physical properties)
is suitable for spatial precipitation forecasts: wavelets (discontinuities and features) + categorical approach (robust and resistant)

9. Spring 2005, 13 May case

10. Intensity-Scale Skill Score

11. Future work Confidence intervals, p-values Under-sampling
Energy and bias on each scale
Random chance can be partitioned across the scales in proportion of magnitude and number of events characterizing the scale …
THANK YOU !

12. Intense storm displaced
Intensity-scale verification technique Casati et al. (2004), Met App, vol. 11
The intensity-scale verification approach measures the skill as function of precipitation intensity and spatial scale of the error
intensity: threshold  binary images (categorical approach)
scale: 2D Wavelets decomposition of binary images
For each threshold and scale: skill score associated to the MSE of binary images = Heidke Skill Score
Skill
threshold (mm/h) 1 -1 -2 -3 -4
640
320
160 80 40 20 10 5
scale (km)
0 1/16 ¼ ½
Intense storm displaced
threshold = 1mm/h
Casati et al. introduces an intensity-scale verification technique which measure the skill a a function of the precipitation intensity and on the spatial scale of the error. Forecast and observation fields are transformed into binary images by thresholding for different precipitation rates. A 2D wavelet decomposition of the error image, obtained as the difference between forecast and observation fields, is used to separate different scale components. A skill score, based on the MSE of binary images and related to the HSS, is then evaluated for each scale component and precipitation threshold. Note that the technique links categorical approaches with scale-verification approaches.
Note that the technique uses a categorical approach (robust and resistant, therefore suitable for precipitation). Moreover uses wavelets, which cope well with discontinuous on and off fields such as precipitation. Figure example is an intense storm badly advected so that it is displaced of almost entirely its length. The displacement is well detect by a minimum negative skill corresponding at the 160 km scale, for intensities of 0.5 to 4 mm/h.