1. Ensemble forecasts and seasonal precipitation tercile probabilities
Michael Tippett, Tony Barnston, Andy Robertson
IRI

2. Motivation PDF fitting: Wilks (2002) NWP, Kharin & Zwiers (2003)
Reduce sampling error in tercile probabilities;
2-tier seasonal forecasts
Force GCMs with predicted SST.
Compute tercile probabilities from frequencies.
Post-process.
Characterize predictability.
Changes in probabilities related to:
Ensemble mean?
Ensemble variance?
Both?
The general topic of this work is computing tercile probabilities from fitted distributions.
There are two motivations.
First, seasonal forecasts use model-based tercile probabilities as a key input.
Second, changes in tercile probabilities from equally likely indicates predictability.
Parametric descriptions allow us to associate those changes in with changes in the pdf.

3. Outline Quantify sampling error Fit parametric forecast PDFs
Analytical estimates
Counting.
Gaussian fit.
Sub-sampling from large ensemble. Perfect model.
ECHAM 4.5 T42 79 members
DJF precipitation over North America
Fit parametric forecast PDFs
Gaussian
Constant variance vs. Variable variance
Generalized linear model
Ensemble mean vs. Ensemble mean and variance.
Two main results.
First, quantify sampling error analytically and sub-sampling from a large ensemble.
Second, use two fitting methods. Is sampling error reduced?
What parameters are useful to characterize changes in tercile probabilities.

4. Sampling error How to measure sampling error?
Compare the sample tercile probability with true tercile probability.
Problem: Don’t know the true probability.
Compare two independent samples.
Error variance between two samples is twice true error.
One way of measuring sampling error is to compare the sample probability with the true probability.
Another is to compare two independent sample.
The variance of their difference is twice the true error.
Do this in a Monte Carlo fashion.
Average over many samples.

5. S = Signal-to-noise ratio N = ensemble size
Converges like
S = Signal-to-noise ratio
N = ensemble size
DJF North America precipitation. Sup-sampling from ensemble of size 79.
Sampling error when you calculate tercile probabilities by counting.
Error depends on sample size and signal-to-noise ratio.
Signal to noise is model dependent.

6. Fitting with a Gaussian
Two types of error:
PDF not really Gaussian!
Sampling error
Fit only mean
Fit mean and variance
Compare the error of counting with fitting a Gaussian.
There’s also sampling error when fitting a Gaussian.
Two sources.
First, the real PDF is not Gaussian. Problem dependent.
Second, sampling error estimating mean and variance. Treat analytically.
Conclusion, ff the PDF is really Gaussian, FIT for better tercile probabilities!
Expression for no signal, additional terms when there is a signal
Error(Gaussian fit N=24) = Error(Counting N=40)

7. Generalized Linear Model
Logistic regression
Regression between tercile probabilities and
Ensemble mean
Ensemble mean and variance
Why GLM?
Relation is nonlinear—probabilities bounded.
Errors are not normal.
An empirical approach.

8. Results Randomly select 24 members
Compute DJF ( ) precipitation tercile probabilities by
Counting (theory predicts average rms error = 10.9)
Fitting Gaussian
Constant variance
Interannually varying variance
GLM
Ensemble mean
Ensemble mean and variance
Compare with frequency probability from independent 55 member ensemble.
Adding more parameters better fits the 24 but not the 55.
Show results comparing counting, fitting a Gaussian and GLM.
Precipitation is not Gaussian, use square-root of precipitation.
This approach allows use to determine which parameters are useful.

9. Gaussian (square-root)
Counting
Below
Above
N=24
Gaussian (square-root)
9.25 = rms error of sampling with 40 member ensemble.
Gaussian fitting gives a reduction in error that is equivalent to going from 24 to 40 members.
Some problems due to the PDF not really being Gaussian.
Time-varying Gaussian

10. Regression with mean (square-root)
Counting
Below
Above
N=24
Regression with mean (square-root)
GLM results are similar, on average.
Some indication that GLM is slight better in regions where the model is more confident and worse elsewhere.
Lots of Monte Carlo draws, so results are robust.
Regression with mean and variance

11. 1996 N=24 Below Above Sample Regression
Illustrate the character of the tercile probabilities based on fitted distributions.
Both use the same 24 members, fitted is spatially smoother.

12. 1998 N=24 Below Above Sample Regression
With strong SST forcing, still have strong shifts.

13. Summary
Used a large ensemble to look at sampling error in perfect model tercile probabilities.
Error well-described analytically
Error depends on sample size and S/N ratio.
Parametric fitting reduces tercile probability sampling error.
For Gaussian fitting and GLM, most of the useful information is associated with ensemble mean.
Future: include model error in fitting process.
Inflate variance of Gaussian.