1. COSMO-LEPS Verification
Chiara Marsigli
ARPA-SMR

2. Available italian stations

3. Verification on station points
Bi-linear interpolation (4 nearest points) X

4. Verification on super-boxes
Average value – maximum value – frequency
OBS.
PRED. X

5. OVERLAPPING BOXES X X X X

6. ROC area Need for LM verification at these fc ranges COSMO-LEPS vs
observations
station points
weighted
Need for LM
verification
at these fc ranges
ROC area
Nov 02 – Dec 02 – Jan 03

7. COSMO-LEPS vs
observations
station points
not weighted
ROC area

8. w
+48 h nw

9. w
+120 h nw

10. COSMO-LEPS vs
observations
station points
weighted
Brier Skill Score

11. Brier Skill Score COSMO-LEPS vs observations station points
not weighted
Brier Skill Score

12. Brier Skill Score

13. Weighting procedure It is possible to decide (in real-time)
if it is better to weight or not to weight?
Dependence from ensemble spread?
Flow dependence?

14. Brier Skill Score
station points
average values on super-boxes

15. Brier Skill Score
station points
average values on super-boxes

16. Brier Skill Score
Nov 02 only

17. Brier Skill Score

18. ROC area contingency table Observed Yes No Forecast a b c d
A contingency table can be built for each probability class (a probability class can be defined as the % of ensemble elements which actually forecast a given event). For the k-th probability class:
The area under the ROC curve is used as a statistic measure of forecast usefulness

19. Brier Skill Score oi = 1 if the event occurs
Brier Score
oi = 1 if the event occurs
= 0 if the event does not occur 
fi is the probability of occurrence according to the forecast system (e.g. the fraction of ensemble members forecasting the event)
BS can take on values in the range [0,1], a perfect forecast having BS = 0
The forecast system has predictive skill if BSS is positive, a perfect system having BSS = 1.
Brier Skill Score
= total frequency of the event
(sample climatology)