1. Update on Dropout Team Work and Related COPC Action Items Bradley Ballish NOAA/NWS/NCEP/PMB Co-Chair JAG/ODAA 15-16 April 2009 CSAB Meeting

2. Outline COPC Action Items NCEP dropout team findings NCEP dropout team plans Joint COPC center dropout work Ideas for COPC Action Item 2008-1.5 Possible forecast divergence measures Satellite radiance temperature plots Summary

3. COPC Action Items COPC Action Item 2008-1.5: Develop a monitoring system to analyze differences between the NCEP and FNMOC global models and the ECMWF global model in real-time and make this real-time system available to OPCs as a daily tool. COPC Action Item 2008-2.14: CSAB will oversee the work of the “Dropout Team” to continue examining the performance dropout issue and how lessons learned can be applied to numerical model performance improvements. This activity shall include establishing periodic interface with ECMWF to exchange QC methodologies/best practices. Recommendations for QC of data, acquisition of new data sources and any other relevant issues shall be presented to the Spring 2009 COPC. Both Priority: (M)

4. NCEP Dropout Team Findings Previous NCEP dropout team findings were presented in reports at past COPC meetings and two papers at the AMS 2009 annual meeting For almost all dropouts studied, no QC smoking gun has been found Most dropouts seem to involve subtle analysis differences, that are not easily explained, in sensitive areas that grow with time in the forecasts The team concludes that making operational implementations to improve the analysis as best we can is a higher priority than just diagnosing dropouts Tests show that the total data in the NCEP model prepbufr file (non-radiance data) is sometimes hurting forecast skill in the Southern Hemisphere and is being researched We have had a number of information exchanges with the ECMWF and are waiting further replies

5. Dropout Team Plans Data impact tests, where different data are deleted, will continue Aircraft thinning and temperature bias correction tests will be performed NMAP graphic tools will be utilized to help diagnose the analysis starting points for dropouts Various data bias correction and QC changes are planned for the future

6. Joint COPC Center Dropout Work Rolf Langland has met with the NCEP dropout team and given them useful information and ideas The NCEP team is using Rolf’s website with adjoint estimates of how analysis differences impact NOGAPS 24 hour forecasts Rolf and others from the Navy have made useful comments and suggestions on NCEP dropout team plans

7. Ideas for COPC Action Item 2008-1.5 Researchers have been working for years at trying to understand poor forecasts as well as analysis/forecast differences – often with no success However, we have three ideas for real-time analysis/forecast differences to the ECMWF: New measures for warnings of large forecast divergence may give an early warning of dropouts Baroclinic instability rates and ensemble tools will be used to help analyze model sensitivity NMAP graphic tools will be tested to help diagnose the analysis starting points for dropouts New graphic tools have been developed to show temperature differences of satellite radiances versus the model background and analysis

8. Possible Forecast Divergence Measures The next slide shows a sample where the current forecast divergence exceeds expected limits Forecast divergence can be an anomaly correlation like score or RMS differences Scores can be for limited areas such as the Northern Hemisphere or North American area Scores could be for different variables and different pressure levels Normal limits could be derived from a 31 day running mean that excludes poor forecast cases

9. Prototype Forecast Divergence Plot Shows divergence of two similar forecasts compared to normal limits Forecast Divergence Limits (red) Current Divergence (blue) Time

10. Large Forecast Divergence Actions When we have a warning that different forecasts such as ECMWF/GFS have large divergence: Forecasters can be alerted Divergence in other forecasts like 18Z GFS vs 00Z GFS or FNMOC vs GFS will be checked Staff will check for possible data problems Forecast maps will be analyzed to find problem areas in the analysis

11. Satellite Radiance Plots New codes have been developed at NCEP to plot temperature differences between the model background or analysis and the analysis estimate of observational temperatures from a given channel for a particular satellite This is useful for analyzing dropout cases, but involves so many satellite types and channels that it would be time consuming to use More work is needed to get a simpler summary such a height differences

12. Summary The NCEP dropout team agrees that implementing analysis improvements is a higher priority than analyzing dropouts, which will continue More work will be done with data impact tests, data bias correction and improved QC We at NCEP will start to use NMAP tools to better analyze forecast differences The development of forecast divergence measures and tools are needed for early detection of dropouts New graphic tools show satellite radiance temperature differences to model but need a simpler summary