1. Transition plan for MAPP/CTB Funded Proposal: Improved probabilistic forecast products for the NMME seasonal forecast system Anthony Barnston (PI), Huug van den Dool, Emily Becker, Michael Tippett, Shuhua Li Demonstration The project aims to improve the NMME probabilistic forecasts by addressing systematic biases in both forecast anomalies and categorical probabilities. The corrected NMME forecasts will have improved reliability and accuracy. The improvements will come about due to (1) corrections in pattern placement and pattern characteristics (developed at IRI), and (2) refinements to local probability anomalies (developed at CPC). The development work toward (1) and (2), currently in progress, will be done during year 1 and the first half of year 2. Operational Deployment During the last quarter of the last year, the software associated with the NMME corrections/improvements will be amalgamated at NOAA/CPC in order that they be implemented in real-time on a monthly basis there. Part of this software consolidation may require rewriting some codes in a different programming language, such as converting Matlab scripts to Fortran code.

2. The CPC component of the proposal Probabilistic forecast products for the NMME seasonal forecast system Emily Becker and Huug van den Dool also Li-Chuan Chen

3. THIS IS WHAT WE DO IN REAL TIME

4. In the same way that traditional MSE can be minimized by a regression, we here attempt to minimize the BS, i.e. the MSE for probability forecasts. The meaning/interpretation of traditional anomaly correlation (AC) By extension the meaning/interpretation of the probability anomaly correlation (PAC)

5. Unanswered question: To what extent should we be ruled by verification metrics?

6. Brier ScoreANB unadj0.1220.1540.132 adj0.1160.1400.124 Brier Skill Sc.ANB unadj0.146-0.0680.115 adj0.1920.0270.164 CFSv2 JanIC SST forecasts for February, Northern Hemisphere Hindcasts 1982-2010.The PAC trick works : lower BS = higher accuracy.It cleans up –ve skill in N class, no embarrassment..It tones down too bold forecasts, particularly in A&B class.The gain is modest, but is the impact small?

7. BS = Reliability minus Resolution plus Uncertainty

8. Brier ScoreANB unadj0.1220.1540.132 adj0.1160.1400.124 Brier Skill Sc.ANB unadj0.146-0.0680.115 adj0.1920.0270.164 CFSv2 JanIC SST forecasts for February, Northern Hemisphere

9. Brier ScoreANB unadj0.1220.1540.132 adj0.1160.1400.124 Brier Skill Sc.ANB unadj0.146-0.0680.115 adj0.1920.0270.164 CFSv2 JanIC SST forecasts for February, Northern Hemisphere BS = Reliability minus Resolution plus Uncertainty

10. In conclusion (CPC part) Probability Anomaly Correlation approach yields a lower BS (as expected) for any model and even for NMME collection. This can be implemented! PAC approach is being at CPC compared to ensemble regression, to have some reference. PAC approach has a large impact on the reliability-resolution diagram. Puzzling actually. PAC has an interesting application in verification of ENSO composites (Li-Chuan Chen) An adjustment to our most beloved conclusion

11. .How much improvement to be expected by (any) calibration?.Correctibility? Inherent skill>=threshold.Individual Models may improve more than the NMME collection. Managing expectations

12. Quad Chart to Track Progress/Issues of MAPP-CTB Demonstration Project in FY16 Improved probabilistic forecast products for the NMME seasonal forecast system Issues/Risks Issues: 1)**. Mitigation: Finances Scheduling Project Information and Highlights Funding Sources: FY14: $271K FY15: $91K Status: Management Attention Required Potential Management Attention Needed On Target Updated on 11/06/2015 Lead: Columbia University: Anthony Barnston, Michael Tippett, Shuhua Li NCEP/CPC: Huug van den Dool, Emily Becker Scope: Improve NMME-based probabilistic forecast products Estimated Benefits: Improved NMME probabilistic forecasts by addressing systematic biases in both forecast anomalies and categorical probabilities. Software will be deployed at CPC MilestoneDateStatus Corrections in pattern placement and pattern characteristics FY16Q1 Refinements to local probability anomalies (developed at CPC). FY16Q3 Post-project reviewFY16Q4 GYR GG G Y

13. (A. Barnston, M. Tippett, S. Li at IRI) Use of multivariate statistical methods to correct systematic errors in placement of forecast patterns of each model. Calibration steps: Spatial corrections by IRI may precede local probability adjustments by CPC. Uncorrected model forecast Corrected model forecast Example: Precipitation forecast during El Nino, DJF, from one atmospheric model

14. Disruption of progress at IRI in 2015 During year 1 of this CTB project, Mike Tippett took a 75%-time teaching position at Columbia University, leaving only summers to do research. So much of his funding was transferred to Shuhua Li. Mike has been training Shuhua to use Matlab in the intended context of this project. Learning has been steady but slow. In the last month, Shuhua Li gave notice to leave IRI, taking an operational forecasting job at BoM in Australia. So his training in this project will not bear any fruit. Due to the above delay and then unexpected loss of Shuhua Li, IRI presently has no concrete results to show regarding model spatial corrections. Possible fixes for these setbacks: (1) Barnston will do all of the hands-on work using CPT instead of Matlab (a quicker, but possibly less permanent, software package); (2) We will request a 1-year no-cost extension of the project, and find a new person to do the hands-on model correction trials.