Presentation is loading. Please wait.

Presentation is loading. Please wait.

Tom Hopson, Jason Knievel, Yubao Liu, Gregory Roux, Wanli Wu

Published byGinger Johnson Modified over 5 years ago

Similar presentations

Presentation on theme: "Tom Hopson, Jason Knievel, Yubao Liu, Gregory Roux, Wanli Wu"— Presentation transcript:

1 Tom Hopson, Jason Knievel, Yubao Liu, Gregory Roux, Wanli Wu
Ensemble-4DWX update: focus on guidance for configuring ensemble members Tom Hopson, Jason Knievel, Yubao Liu, Gregory Roux, Wanli Wu National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, April

2 Reminder of status in Aug 2009
Improving scripting to manage the computational resources Increased calibration stations from 26 to 34 (regional) Calibration hindcast increased from roughly 600 to over 1000 points Increased hindcast size allows calibration of 2 additional lead-times (36hr, 42hr) Included new skill measure for ensemble skill-spread relationship Corrected over-dispersion of calibration process Automated calibrating 2-m T Testing automated calibration of 2-m dew point T National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, April

3 Progress since August 2010 Adding new graphics for DPG’s easier access to post-processed results Continue to improve scripting and potential use of new computational resources to improve post-processing times Wind speed calibration development Developed tool for testing utility of ensemble number to optimize DPG’s reduced model configuration over potential new locations Work continuing on scientific papers Cad - clear air damming study National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, April

4 Recap on Calibration and verification tools
Ensemble calibration to correct predicted distribution Calibration is needed for users capable of decision making with probabilistic guidance. Will be needed for foreseeable future Verification of different ensemble characteristics is easily completed when performing calibration National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, April

5 Calibration: review and recap
“bias” obs Forecast PDF Probability Probability Forecast PDF obs “spread” or “dispersion” calibration Temperature [K] Temperature [K] Calibration (and verification) is now fully-automated Utilizes “persistence” if available 34 sites in and near DPG Full calibration for all sites ~ 1X per week for each weather variable Using lookup tables ~ every hour (8 hrs, was 1hr) National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, April

6 Probability/°K T [K] Probability/°K T [K]
Step 2: For each quan, use forward step-wise cross-validation to select best regress set Selection requires: a) min QR cost function, b) binomial distrib at 95% confidence If requirements not met, retain climatological “prior” Step I: Determine climatological quantiles Probability/°K climatological PDF 1. Regressor set: 1. reforecast ens 2. ens mean 3. ens stdev 4. persistence 5. LR quantile (not shown) 3. T [K] 2. 4. Temperature [K] observed forecasts Time Step 3: segregate forecasts based on ens dispersion; refit models (Step 2) for each range Final result: “sharper” posterior PDF represented by interpolated quans forecasts posterior Forecast PDF I. II. III. II. I. Probability/°K prior T [K] Temperature [K] Time National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, April

7 3-hr dewpoint time series
Station DPG S01 Before Calibration After Calibration National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, April

8 42-hr dewpoint time series
Station DPG S01 Before Calibration After Calibration National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, April

9 PDFs: raw vs. calibrated
Blue is “raw” ensemble Black is calibrated ensemble Red is the observed value Notice: significant change in both “bias” and dispersion of final PDF (also notice PDF asymmetries) obs ATEC-4DWX IPR, April

10 Troubled rank histograms
Counts Counts Ensemble # Ensemble # ATEC-4DWX IPR, April

11 3-hr dewpoint rank histograms
Station DPG S01 National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, April

12 42-hr dewpoint rank histograms
Station DPG S01 National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, April

13 RMSE of ensemble members
Station DPG S01 3hr Lead-time 42hr Lead-time National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, April

14 Significant calibration regressors
Station DPG S01 3hr Lead-time 42hr Lead-time National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, April

15 Assessing ensemble utility
Goal: provide guidance in ensemble member selection Caveat: results only applying for T and Td surface variables – potential different ensemble usage for different variables and levels Approach: Starting with complete ensemble set, rank model ensembles in terms of their usage in the calibration procedure by assessing over all weather variables, locations, and lead-times Iteratively remove least-used ensembles from calibration, and use verification statistics to determine the cost of reduction What is the “optimal size” that balances forecast skill and computational expense? National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, April

16 Single value to summarize performance.
Skill Scores Single value to summarize performance. Reference forecast - best naive guess; persistence, climatology A perfect forecast implies that the object can be perfectly observed Positively oriented – Positive is good National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, Aug ATEC-4DWX IPR, April

17 Preliminary Results … Brier referenced to Persistence
RMSE referenced to Persistence National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, April

18 … more theoretically Probability/°K Temperature [K] Probability/°K
42hr Lead-time Uncorrelated ensembles Probability/°K Temperature [K] Correlated ensembles Probability/°K More ensembles usefully define mean of PDF if error growth less than linear *and* uncorrelated For perfectly-correlated ensembles, addition of *any* ensemble degrades skill Temperature [K] National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, April

19 Future plans Continue work on refining calibration algorithm to decrease computation time (required as new weather variables are added) Implement for wind speed and wind direction Continue development of guidance tool for optimal ensemble selection for operational use Develop scheme for model points without surface observations over whole model domain National Security Applications Program Research Applications Laboratory ATEC-4DWX IPR, April

Download ppt "Tom Hopson, Jason Knievel, Yubao Liu, Gregory Roux, Wanli Wu"

Similar presentations

Chapter 13 – Weather Analysis and Forecasting

Chapter 13 – Weather Analysis and Forecasting
Model Evaluation Tools MET. What is MET Model Evaluation Tools ( MET )- a powerful and highly configurable verification package developed by DTC offering:

Model Evaluation Tools MET. What is MET Model Evaluation Tools ( MET )- a powerful and highly configurable verification package developed by DTC offering:
A Brief Guide to MDL's SREF Winter Guidance (SWinG) Version 1.0 January 2013.

A Brief Guide to MDL's SREF Winter Guidance (SWinG) Version 1.0 January 2013.
Statistical post-processing using reforecasts to improve medium- range renewable energy forecasts Tom Hamill and Jeff Whitaker NOAA Earth System Research.

Statistical post-processing using reforecasts to improve medium- range renewable energy forecasts Tom Hamill and Jeff Whitaker NOAA Earth System Research.
Comparing and Contrasting Post-processing Approaches to Calibrating Ensemble Wind and Temperature Forecasts Tom Hopson Luca Delle Monache, Yubao Liu, Gregory.

Comparing and Contrasting Post-processing Approaches to Calibrating Ensemble Wind and Temperature Forecasts Tom Hopson Luca Delle Monache, Yubao Liu, Gregory.
Details for Today: DATE:3 rd February 2005 BY:Mark Cresswell FOLLOWED BY:Assignment 2 briefing Evaluation of Model Performance 69EG3137 – Impacts & Models.

Details for Today: DATE:3 rd February 2005 BY:Mark Cresswell FOLLOWED BY:Assignment 2 briefing Evaluation of Model Performance 69EG3137 – Impacts & Models.
Instituting Reforecasting at NCEP/EMC Tom Hamill (ESRL) Yuejian Zhu (EMC) Tom Workoff (WPC) Kathryn Gilbert (MDL) Mike Charles (CPC) Hank Herr (OHD) Trevor.

Instituting Reforecasting at NCEP/EMC Tom Hamill (ESRL) Yuejian Zhu (EMC) Tom Workoff (WPC) Kathryn Gilbert (MDL) Mike Charles (CPC) Hank Herr (OHD) Trevor.
1 Kalman filter, analog and wavelet postprocessing in the NCAR-Xcel operational wind-energy forecasting system Luca Delle Monache Research.

1 Kalman filter, analog and wavelet postprocessing in the NCAR-Xcel operational wind-energy forecasting system Luca Delle Monache Research.
JEFS Status Report Department of Atmospheric Sciences University of Washington Cliff Mass, Jeff Baars, David Carey JEFS Workshop, August 28-29.

JEFS Status Report Department of Atmospheric Sciences University of Washington Cliff Mass, Jeff Baars, David Carey JEFS Workshop, August
Department of Meteorology and Geophysics University of Vienna since 1851 since 1365 TOWARDS AN ANALYSIS ENSEMBLE FOR NWP-MODEL VERIFICATION Manfred Dorninger,

Department of Meteorology and Geophysics University of Vienna since 1851 since 1365 TOWARDS AN ANALYSIS ENSEMBLE FOR NWP-MODEL VERIFICATION Manfred Dorninger,
MOS Performance MOS significantly improves on the skill of model output. National Weather Service verification statistics have shown a narrowing gap between.

MOS Performance MOS significantly improves on the skill of model output. National Weather Service verification statistics have shown a narrowing gap between.
Ensemble Post-Processing and it’s Potential Benefits for the Operational Forecaster Michael Erickson and Brian A. Colle School of Marine and Atmospheric.

Ensemble Post-Processing and it’s Potential Benefits for the Operational Forecaster Michael Erickson and Brian A. Colle School of Marine and Atmospheric.
W ildland F ire D ecision S upport S ystem Overview April, 2008.

W ildland F ire D ecision S upport S ystem Overview April, 2008.
A Regression Model for Ensemble Forecasts David Unger Climate Prediction Center.

A Regression Model for Ensemble Forecasts David Unger Climate Prediction Center.
Chapter 13 – Weather Analysis and Forecasting. The National Weather Service The National Weather Service (NWS) is responsible for forecasts several times.

Chapter 13 – Weather Analysis and Forecasting. The National Weather Service The National Weather Service (NWS) is responsible for forecasts several times.
Multi-Model Ensembling for Seasonal-to-Interannual Prediction: From Simple to Complex Lisa Goddard and Simon Mason International Research Institute for.

Multi-Model Ensembling for Seasonal-to-Interannual Prediction: From Simple to Complex Lisa Goddard and Simon Mason International Research Institute for.
Probabilistic forecasts of (severe) thunderstorms for the purpose of issuing a weather alarm Maurice Schmeits, Kees Kok, Daan Vogelezang and Rudolf van.

Probabilistic forecasts of (severe) thunderstorms for the purpose of issuing a weather alarm Maurice Schmeits, Kees Kok, Daan Vogelezang and Rudolf van.
Copyright 2012, University Corporation for Atmospheric Research, all rights reserved Verifying Ensembles & Probability Fcsts with MET Ensemble Stat Tool.

Copyright 2012, University Corporation for Atmospheric Research, all rights reserved Verifying Ensembles & Probability Fcsts with MET Ensemble Stat Tool.
Ensemble Data Assimilation and Uncertainty Quantification Jeffrey Anderson, Alicia Karspeck, Tim Hoar, Nancy Collins, Kevin Raeder, Steve Yeager National.

Ensemble Data Assimilation and Uncertainty Quantification Jeffrey Anderson, Alicia Karspeck, Tim Hoar, Nancy Collins, Kevin Raeder, Steve Yeager National.
ESA DA Projects Progress Meeting 2University of Reading Advanced Data Assimilation Methods WP2.1 Perform (ensemble) experiments to quantify model errors.

ESA DA Projects Progress Meeting 2University of Reading Advanced Data Assimilation Methods WP2.1 Perform (ensemble) experiments to quantify model errors.

Similar presentations

Ads by Google