1. MOS and Evolving NWP Models Developer’s Dilemma: Frequent changes to NWP models… Make need for reliable statistical guidance more critical Helps forecasters adapt to changing model performance Can help remove systematic bias, “downscale” output Forecasters need updated guidance with model changes Make development of reliable statistical guidance more difficult Operational statistical systems can be compromised Long, stable dependent samples not available MODELS OBS MOS

2. Parallel evaluation Run MOS…new vs. old NWP model Assess impacts on MOS skill Do nothing? OK if impacts are minimal But, often they aren’t! (GFS wind / temps) OK, now what? Bias characteristics significantly different Undesirable effects on MOS performance Limited sample available from newest version Responding to NWP Model Changes

3. GFS: Hybrid EnKF parallel evaluation – Cool Season

4. 2009 - 2011 GFS MOS Wind Bias 2009 2010 May-Jul 2011 Jan-Apr 2011

5. MOS Parallel Evaluations Single-station MOS elements most affected by changes in model bias, i.e. temperatures, winds Effects vary regionally, seasonally Bias correction evaluated, some improvement in temperatures, not sufficient for winds Heavy use of boundary layer model predictors in MOS Regionalized elements generally show less degradation PoP, QPF, Clouds, Tstm, Ceiling, etc. Regionalized equations less sensitive to model changes Not all model changes degrade the MOS guidance Improved data assimilation, improved timing, reduction of random errors MOS parallel evaluations have identified bias problems in upgraded models Lessons Learned

6. 1. Consistent archive grid Smoothing of fine-scale detail, constant mesh length for grid sensitive calculations 2. Enlarged geographic regions Larger data pools help to stabilize equations, less sensitive to boundary layer 3. Use of “robust” predictor variables Fewer boundary layer variables, offer variables less sensitive known model changes 4. Multivariate MOS equations Can compensate for sudden changes in a single predictor 5. Mixed samples Increase sample size by blending latest version of model with previous model version output Mitigating the effects on development To help reduce the impact of model changes and small sample size, we rely upon...

7. GFS MOS Wind Verification Results* 5/10/2011 – 9/30/2011 Using even just a little data from the new NWP model version can be helpful! 0.00 * 2-season dependent sample (4/2010 – 9/2011) Windspeed - Southwest CONUS MAE Bias

8. MDL Summary Retrospective Runs or Reforecasts Minimize time lag between significant model changes and availability of quality guidance NMM and GFS development work demonstrated usefulness of 2-3 year sample of “new” model mixed with previous version for temperatures and winds Has to be representative – across all seasons, synoptic regimes Allows calibrated guidance to take advantage of model improvements Representativeness improved by reruns every Nth day (MDL would defer to others for ideal N) but MUST be run on the operational model configuration National Model Blender Project will face challenges gathering samples sufficient for development