Bookmaker or Forecaster? By Philip Johnson. Jersey Meteorological Department.

What was wrong with the “Old” methods. Can categorical forecasts be improved? Do the public understand percentages? How can we produce the odds? Do we know if we are getting it right. Why use probabilities?

The product that started us in Probability forecasting.

Producing the forecast. Various model outputs from different centres MOS (model output statistics) Climatology Local knowledge Feedback of forecast assessment

UKMO forecast chart

American MRF model

American MRF model

MSL pressure from an American ensemble forecast.

Verification To assess the output To provide quality assurance Feedback of results to improve forecast

Verification Techniques Reliability Diagrams Brier Scores Skill Scores

Reliability Diagrams Number of forecasts Brier % score Under and over forecasting

Brier and Skill Scores Brier Scores use same information as reliability diagrams. Shows a mean square error of the forecast. 0 is perfect, 1 is worst possible score. Skill Score compares two Brier scores.

Brier and Skill Score v Sample Climate Brier score

Hedging your bets. What if you only forecast the climate value? All forecasts around 28% and you must be right!

Only 25% forecast. Reliable forecast. No sharpness.

Single value forecast. Brier goes up. Skill goes down.

Over the last 6 years we have assessed our forecasts. These have been compared with MOS output. Sample and fixed Climatology has been used for evaluation. What results have we had?

Brier Results. These show that the forecaster used a good range of % values, and good skill levels especially up to T +78 From T+90 the forecasts showed an increase in Brier Scores inline with model variability.

Skill scores The forecast skill when compared with climate shows good positive values Model and forecast skill deteriorate noticeably from T+90

Brier and Skill Score v Climate Brier score for fixed 25% forecast

Did the forecaster make it as a successful Bookmaker? YES

What next? Continued feedback of results Improved modeling More use of Ensemble Forecasts Neural networks

Bookmaker or Forecaster? By Philip Johnson. Jersey Meteorological Department.