1. Concept of Uncertainty - Resources and NRC’s report
Yuejian Zhu
Ensemble Team Leader
EMC/NCEP/NWS/NOAA
Presents for NWP Forecast Training Class
March 30, 2015, Fuzhou, Fujian, China

2. Concept of Uncertainty – Resources and NRC’s Report
Definition of uncertainty
Uncertainty is kind of natural
Uncertainty is kind of error
Errors are from:
Observation instruments
Reading/recording observations
Initial analysis – data assimilation
Model physics and parameterizations
Computations
Model developer:
Using ensemble to simulate the uncertainty maximally/accurately
Find out analysis uncertainty
Find out model uncertainty (dynamic and physic)
Forecasters:
Using ensemble information to make uncertainty forecast
Example 1: probability forecast
Example 2: deterministic forecast with uncertainty level
Example 3: uncertainty forecast
Completing forecast
US NRC’s report – fundamental education material

3. Lorenz Equation ( Chaos in ODE)
X-Z Plane
X-Y Plane
Lorenz Equation ( Chaos in ODE)
The Lorenz model is defined by
three differential equations given
the time evolution of variables X(t), Y(t), Z(t)
Power Spectrum

4. Errors?
Errors?
Errors?
Errors?
Errors?
Errors?

5. Ensemble Forecast System
Describe Forecast Uncertainty Arising Due To Chaos
Buizza 2002

6. Concept of Uncertainty – Resources and NRC’s Report
Definition of uncertainty
Uncertainty is kind of natural
Uncertainty is kind of error
Errors are from:
Observation instruments
Reading/recording observations
Initial analysis – data assimilation
Model physics and parameterizations
Computations
Model developer:
Using ensemble to simulate the uncertainty maximally/accurately
Find out analysis uncertainty
Find out model uncertainty (dynamic and physic)
Forecasters:
Using ensemble information to make uncertainty forecast
Example 1: probability forecast
Example 2: deterministic forecast with uncertainty level
Example 3: uncertainty forecast
Completing forecast
US NRC’s report – fundamental education material

7. Forecast Uncertainty 预报的不确定性 预报误差 不确定性 Instrument 仪器 Assimilation 同化
Observation 观测
Assimilation 同化
Analysis 分析
预报误差
Forecast Errors
Uncertainty
不确定性
Parameterization
模式参数化
Computation
计算过程

8. Earlier Ensemble Forecast System
One day advantage
Due to model imperfection

9. Concept of Uncertainty – Resources and NRC’s Report
Definition of uncertainty
Uncertainty is kind of natural
Uncertainty is kind of error
Errors are from:
Observation instruments
Reading/recording observations
Initial analysis – data assimilation
Model physics and parameterizations
Computations
Model developer:
Using ensemble to simulate the uncertainty maximally/accurately
Find out analysis uncertainty
Find out model uncertainty (dynamic and physic)
Forecasters:
Using ensemble information to make uncertainty forecast
Example 1: probability forecast
Example 2: deterministic forecast with uncertainty level
Example 3: uncertainty forecast
Completing forecast
US NRC’s report – fundamental education material

10. Ensemble Forecasts Deterministic forecast Initial uncertainty
Forecast probability
Verified analysis

11. Real example for bimodality

12. Top: 2-m temperature probabilistic forecast (10% and 90%) verification
2-m temp 10/90 probability forecast verification Northern Hem, period of Dec – Feb. 2008
90%
3-month verifications
10%
Top: 2-m temperature probabilistic
forecast (10% and 90%) verification
red: perfect, blue: raw, green: NAEFS
Left: example of probabilistic forecasts
(meteogram) for Washington DC, every
6-hr out to 16 days from

13. Concept of Uncertainty – Resources and NRC’s Report
Definition of uncertainty
Uncertainty is kind of natural
Uncertainty is kind of error
Errors are from:
Observation instruments
Reading/recording observations
Initial analysis – data assimilation
Model physics and parameterizations
Computations
Model developer:
Using ensemble to simulate the uncertainty maximally/accurately
Find out analysis uncertainty
Find out model uncertainty (dynamic and physic)
Forecasters:
Using ensemble information to make uncertainty forecast
Example 1: probability forecast
Example 2: deterministic forecast with uncertainty level
Example 3: uncertainty forecast
Completing forecast
US NRC’s report – fundamental education material

14. Ensemble forecast is widely used in daily weather forecast
Uncertainties &
disagreements
Ensemble forecast is widely used in daily weather forecast

15. Map of PQPF and Precipitation Types: every 6 hours, 4 different thresholds

16. Example of forecast precipitation (next 24 hours)
(assume all forecasts are bias free – after bias correction or calibration)
100 75 50 50 50 50
80%
80%
mean
mean 25
deterministic
probabilistic
probabilistic
Fcst 1
Fcst 2
Fcst 3
MM/24hrs
Which forecast is relatively easier? Or more confident!
Can we identify all these information?

17. Resolution makes difference for Typhoon Morakot
Ini:
T126 ensemble
T190 ensemble
Most models do not
make right forecasts
Ini:
T126 ensemble
T190 ensemble 17
Is this a good example for uncertainty forecast???

18. Is this a good example to discuss forecast consistency and accuracy?
Typhoon Megi
NCEP
CMC
ECMWF
3-ENS
Is this a good example to discuss forecast consistency and accuracy?

19. Thick blue: ensemble mean
00UTC
Bimodality?
Thick blue: ensemble mean
Opr: T254L42
(55km)
Para: T574L64
(33km)
(8 days)
Red arrow means good forecast
06UTC

20. Concept of Uncertainty – Resources and NRC’s Report
Definition of uncertainty
Uncertainty is kind of natural
Uncertainty is kind of error
Errors are from:
Observation instruments
Reading/recording observations
Initial analysis – data assimilation
Model physics and parameterizations
Computations
Model developer:
Using ensemble to simulate the uncertainty maximally/accurately
Find out analysis uncertainty
Find out model uncertainty (dynamic and physic)
Forecasters:
Using ensemble information to make uncertainty forecast
Example 1: probability forecast
Example 2: deterministic forecast with uncertainty level
Example 3: uncertainty forecast
Completing forecast
US NRC’s report – fundamental education material

21. Completing the Forecast
Characterizing and Communicating Uncertainty
for Better Decisions
Using Weather and Climate Forecasts
Coming soon in Chinese……
US NRC report (174 pages)
The National Academic Press
2006

22. Completing the Forecast
Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts
Board on Atmospheric Sciences and Climate

23. Task
Provide guidance on how to identify and characterize needs for uncertainty information among various users of forecasts. Support this with examples of needs.
Identify limitations in current methods for estimating and validating forecast uncertainty, relating these limitations to users’ needs and recommend improvements or new methods and approaches.
Identify sources of misunderstanding in communicating forecast uncertainty, including vulnerabilities dependent on the means of communication used, and recommend improvements in the ways used to communicate forecast uncertainty.
Recognizing the breadth and depth of this task, NWS advised the committee at its opening meeting to "teach us how to fish as opposed to giving us a fish."

24. Report Contents Summary Introduction Uncertainty in Decision Making
Estimating and Validating Uncertainty
Communicating Forecast Uncertainty
Overarching Recommendations

25. Recommendations
The entire Enterprise should take responsibility for providing products that effectively communicate forecast uncertainty information. NWS should take a leadership role in this effort.
NWS should improve its product development process by collaborating with users and partners in the Enterprise from the outset and engaging and using social and behavioral science expertise.
All sectors and professional organizations of the Enterprise should cooperate in educational initiatives that will improve communication and use of uncertainty information.
NWS should develop and maintain the ability to produce objective uncertainty information from the global to the regional scale.

26. Recommendations
To ensure widespread use of uncertainty information, NWS should make all raw and post-processed probabilistic products easily accessible to the enterprise at full spatial and temporal resolution. Sufficient computer and communications resources should be acquired to ensure effective access by external users and NWS personnel.
NWS should expand verification of its uncertainty products and make this information easily available to all users in near real time. A variety of verification measures and approaches (measuring multiple aspects of forecast quality that are relevant for users) should be used to appropriately represent the complexity and dimensionality of the verification problem. Verification statistics should be computed for meaningful subsets of the forecasts (e.g., by season, region) and should be presented in formats that are understandable by forecast users. Archival verification information on probabilistic forecasts, including model-generated and objectively generated forecasts and verifying observations, should be accessible so users can produce their own evaluation of the forecasts.

27. Recommendations
To enhance development of new methods in estimation, communication, and use of forecast uncertainty information throughout the Enterprise, and to foster and maintain collaboration, confidence, and goodwill with Enterprise partners, NWS should more effectively use testbeds by involving all sectors of the Enterprise.
The committee endorses the recommendation by the NRC “Fair Weather” report to establish an independent advisory committee and encourages NOAA to bring its evaluation of the recommendation to a speedy and positive conclusion.
NWS should dedicate executive attention to coordinating the estimation and communication of uncertainty information within NWS and with Enterprise partners.

28. Key points not to forget
Broad applicability within NOAA and other agencies
This will take time (turn the aircraft carrier)
Let’s get started now
NWS evolution - time is right to incorporate uncertainty information into process
Uncertainty is another component of improving the forecast – it’s not detached from the primary mission of forecast improvement
Uncertainty is not an add-on – it is in the nature of the product