1. May 17, 2006DSST STEP Conf Call #21 DSST Short-Term Ensemble Planning (STEP) Call #2 RFC Presentations and Discussion

2. May 17, 2006DSST STEP Conf Call #22 Agenda Introduction (DJ) RFC presentations and Q/A 1.AB 2.AP 3.CB 4.CN 5.MA 6.MB 7.NC 8.NE 9.NW 10.Comments by other RFCs Discussion (All) –Towards developing (near-) consensus operations concept and identifying overarching needs and issues (service, operations and science) What next? (DJ)

3. May 17, 2006DSST STEP Conf Call #23 ABRFC

4. May 17, 2006DSST STEP Conf Call #24 Operations Concept I see the ABRFC using short term ensembles (STE) as the "official" way to issue almost all of our forecasts. That includes our 27 daily forecast, as well as our forecasts we issue for the other 200+ flood forecast points we have. Only our long term water supply forecasts would be different. If I had our way, I would have implemented the STE as the "AWIPS" requirement fulfillment long ago. 98% of our users do not want long term ESP forecasts! I picture that we would issue the forecasts mainly in a graphical method, much like the examples that this office has provided (see below). Instead of issuing just deterministic forecasts like we do now, I see STE as a replacement/addition to this.

5. May 17, 2006DSST STEP Conf Call #25 Operations Concept (cont.) Another use of short-term ensemble (STE) forecasts is by the WFOs as a basis for triggering a river flood watch –If we got the procedures efficient enough, we could issue our STE forecasts routinely for all of our forecast points, not just for flood situations. This way, with experience and possibly local WFO procedures, the WFOs could use the STE forecasts as the main part of the decision process in determining when to issue flood watches and outlooks.

6. May 17, 2006DSST STEP Conf Call #26 Needs and Issues One thing that NEEDS be to accomplished is for us to determine or define a relationship between probabilistic forecasts and a single deterministic forecast. I think all STE forecasts should also include the office’s best guess of a deterministic forecast, and this deterministic forecast should be located near the 50% probability line. We have to define how deterministic forecasts relate to probabilistic forecasts before we can go any further. Many of our users just want one forecast, i.e. the deterministic forecast, while other more sophisticated users will enjoy the probabilistic forecast. Issuing a STE with both data would be the best bet I think! I also think that the STE info could be presented in a text format, but the exact format could be determined later.

7. May 17, 2006DSST STEP Conf Call #27 Needs and Issues (cont.) Operationally, I see MANY significant things that need to be accomplished. Once you get the science down, we need software that will produce the graphics/text output of this data in a user friendly manner (i.e. xsets/hydrograph creation software). We need training for those producing and using these forecasts. We need a solid verification set of data proving that the science behind STE is reliable and sound. I have yet to see any verification (including that which I have done) that convinces me that the science behind the STE is producing reliable and accurate short term forecasts. As far as the DA goes, I still am not convinced it works. The little I have heard from WGRFC users is that they were not satisfied with it. I think we need some extensive verification of it, and HICS/DOHS/RFCS need to be in the decision to buy off on this. At this point, I have not seen any convincing evidence that it works.

8. May 17, 2006DSST STEP Conf Call #28 Needs and Issues (cont.) Overall, I am excited about STE, but still am not convinced that the science behind our current configuration works. I do however foresee a future where the vast majority of forecasts issued from the ABRFC are STE, which will allow for a great advance in the hydrologic program.

9. May 17, 2006DSST STEP Conf Call #29 APRFC

10. May 17, 2006DSST STEP Conf Call #210 APRFC HAS ConOps Ability to select model(s) of choice Ability to evaluate 6-hour QPF for a set of stations Ability to apply post processing to get pcpn range for each station Ability to review/edit pcpn ranges Ability to apply “Mountain Mapper” functionality to get MAPs Ability to review/edit MAP value ranges Ability to repeat process for temperatures

11. May 17, 2006DSST STEP Conf Call #211 APRFC Hydro ConOps Ability to review forecast and range of MAP and MAT values Ability to run IFP and see resulting hydrographs for range of inputs Ability to adjust MAP/MAT relationships Ability to run post processing to get most likely and range of hydrographs

12. May 17, 2006DSST STEP Conf Call #212 APRFC Summary Comments We think that short term ensembles have equal or greater value than long term ensembles for our users We have concerns about –the ability to quantify the uncertainty in QPF given its significant spatial and temporal variability and individual forecaster bias –the ability to quantify the uncertainty of the interrelated QPF and QTF variables and the resulting impact on the ensemble of hydrographs –the apparent conflicts of wanting control over the output (tweaking the results) vs. maintaining unbiased statistical ensembles

13. May 17, 2006DSST STEP Conf Call #213 CBRFC

14. May 17, 2006DSST STEP Conf Call #214 CBRFC Short-term Ensemble Opportunity: We can improve our short to medium range esp forecasts by using numerical model predictions in place of historical data.

15. May 17, 2006DSST STEP Conf Call #215 But... We can not directly use output from a weather model, even if it is in ensemble form. Our conceptual model does not necessarily want to see reality. It wants to see its own twisted version of reality. We get that twisted version by determining a relationship between what the weather model predicts and what our river model has been calibrated with. Otherwise known as downscaling.

16. May 17, 2006DSST STEP Conf Call #216 Project Area: 27 Segments Above Cameo, Colorado River All recently recalibrated and set up For ESP.

17. May 17, 2006DSST STEP Conf Call #217 Downscaling MRF Variables: 2m air temp Precipitation 700mb Relative Humidity Sea Level Pressure 10m Vector Wind Total Column Precipitable Water Basin Scale Variables: Mean Areal Temperature Mean Areal Precipitation

18. May 17, 2006DSST STEP Conf Call #218 In order to downscale we need several years, if not decades of forecast data. This requires a re-forecast process of the weather model. In addition, for operational use, we need that weather model output to stay consistent with the relationships we have determined. If it is changed to use better physics, for example, we would then need a re-forecast data set and determine new downscaling equations.

19. May 17, 2006DSST STEP Conf Call #219 Downscaling Results MRF is colder than normal in this case.

20. May 17, 2006DSST STEP Conf Call #220 Input into ESP MRF derived MAT/MAPs are attached to historical years (“ensembles”) and ‘fed’ to ESP.

21. May 17, 2006DSST STEP Conf Call #221 ESP peak flow Smaller peaks because MRF is colder for first 14 days causes less melt.

22. May 17, 2006DSST STEP Conf Call #222 Operational challenges –Scale – it may be too large for precipitation. In general, smaller scale should be better –Availability – we need commitment to use this operationally –Re-forecast process – any changes in model physics or scale requires a re- forecast and new downscaling equations. The good news is the weather community is starting to see the value in statistical correction. –Presentation – we should keep these forecasts distinct which will require new data storage and handling techniques

23. May 17, 2006DSST STEP Conf Call #223 CNRFC

24. May 17, 2006DSST STEP Conf Call #224 CNRFC comments on short term ensemble Need customer buy-in, “So how good are you guys anyway?” 50% or median….need a measure of reliability that is easily understood by users 90% 10% bounds….is the 10% exceedance number really exceeded only 10% of the time? A reasonable range of expected values should be very useful to the emergency folks Need to adequately explain how uncertainty increases as you move into the future Conditional uncertainty….Some storms (strong cold front crossing the state) are much easier to forecast that others (cut-off low just off the coast) We should be able to modulate the uncertainty accordingly.

25. May 17, 2006DSST STEP Conf Call #225 A Vision for Operational Hydrologic Short-Term Ensemble Forecasting - Rob Hartman (AHPS Theme Team, June 2005) For many years, NWS customers have benefited from probabilistic long- range water supply forecasts in the Western U.S. The potential benefits of accurate short-term probabilistic flood forecasts are very significant. This becomes obvious when one considers the cost of local emergency management activities. For several years now, OHD has been working on short-term ensemble prototypes. These efforts have been concentrated on developing ensemble inputs through model downscaling or simulation based on the joint distribution of forecasts and observations. Additional activities such as post processing and data assimilation (DA) have been identified. To date, however, the operational environment for short-term ensemble based probabilistic forecasts has not been described. The time-line for AHPS implementation of short-term probabilistic forecasts is such that the forecasting environment will certainly still involve OFS and the IFP. As such, operational ensemble forecasts must function in this environment. This will require several changes to the ESP and OFS architecture. Here is a typical scenario:

26. May 17, 2006DSST STEP Conf Call #226 The RFC is experiencing moderate flooding in several watersheds. A forecast update is due out at 03Z. Data come in after 00Z for the second period of the day, the 5-day QPF is updated by the HAS function. Input data are QC’d and the forecaster starts his/her IFP run to update guidance. The first segment is a flood forecast point. The output includes the single-value forecast as well as shaded regions that depict the probabilistic forecast with user definable regions (10% EP, 25% EP, Ensemble Mean, 75% EP, 90% EP, etc). The output indicates that there is a 40% chance that the river will rise to 3 feet over flood stage by noon tomorrow (mouse tracker interpolation). Three feet over flood stage is a critical stage for local mitigation. But wait, the simulation appears to be a bit screwy. Upon examination, the forecaster sees that bad precipitation data made it through the QC process. The MAP for the 18-00Z period needs to be increased by 50%. A MOD is made. The forecaster reruns the segment. The ensembles are regenerated and included in the display as before. The single- value forecast and probabilities shift slightly. The guidance looks reasonable. The phone rings. It’s the WFO and their local EM needs a forecast update right now. The forecaster selects “Issue guidance” from the pull-down menu and the system initiates the process that generates and issues the single-value and probabilistic guidance for this location.

27. May 17, 2006DSST STEP Conf Call #227 This scenario identifies several issues that are not currently supported with OFS and ESP. These include: 1.The notion of carryover must to be re-examined so that ESP can be run interactively from any point in time and for individual components of a forecast group. Ensemble generation must become interactive rather than a batch process. Performance must support interactive use (rerun small pieces in a few seconds). 2.The ESP process must thoroughly re-examine the notion of MODs and their impacts on short-term ensembles. Some believe that DA and automatic state updating is the only solution to avoid MODs. In the midst of forecasting, this is unrealistic. There will always be times when a forecaster needs to drive the model to the appropriate outcome. That’s why we have forecasters. 3.Statistical post-processing techniques must to be fast and interactive. 4.Visualization tools must to be developed within the IFP framework to support ensemble and probabilistic information. 5.Ensemble and probabilistic information must be managed to facilitate the generation of products and guidance. 6.OFS does not write information back to the processed DB until the segment is exited. This prevents product and information generation while looking at the IFP display. Without doubt, we’ll find lots of other issues as we attempt operational implementation of short-term probabilistic forecasts. As such, it is important to being addressing the issues and developing an operational prototype.

28. May 17, 2006DSST STEP Conf Call #228 MARFC

29. May 17, 2006DSST STEP Conf Call #229 Shorter Term Probabilistic Forecasts 7-day probabilistic river forecasts PQPF/PQTF Demonstration of short-term approach Juniata & Schuylkill Basins 18 points issued daily

30. May 17, 2006DSST STEP Conf Call #230 7-Day Probabilistic River Forecasts Current Basin Conditions Short term probabilistic precipitation and temperatures (PQPF/PQTF) 48-hour PQPF merged with 5 days of climo QPF scenarios based on comparison of historic forecast and observed MAPs 3 graphics generated daily for 18 basins in PA- Juniata (CTP) and Schuylkill (PHI) Basins

31. May 17, 2006DSST STEP Conf Call #231 WFO CTP AHPS Page

32. May 17, 2006DSST STEP Conf Call #232 Frankstown Br. Juniata River at Williamsburg, PA

33. May 17, 2006DSST STEP Conf Call #233 Difficult to Find on AHPS Web Page

34. May 17, 2006DSST STEP Conf Call #234 7-Day Expected Value Plot

35. May 17, 2006DSST STEP Conf Call #235 7-Day PQPF Traces

36. May 17, 2006DSST STEP Conf Call #236 Lessons Learned 7-day PQPF/PQTF forecasts are good contingency forecasts Describe a range of outcomes that help address HSA questions…”What if..” Wide range of potential river responses is not always pleasing…depicts difficulty in forecasting precipitation

37. May 17, 2006DSST STEP Conf Call #237 Lessons Learned Only limited use by WFO’s and cooperators (much less than 30-day ESP forecasts) Difficult for “non-technical” audience Software runs well and can generate graphics in a “hands-off” mode Once rain is on the ground we have considerable modeling uncertainties not incorporated in this method

38. May 17, 2006DSST STEP Conf Call #238 Lessons Learned Await results of OHD verification work to assess validity of products MARFC would like to generate probabilistic forecasts beyond day 2 based on more than climatology Potential collaboration with WFO CTP and SREF ensembles

39. May 17, 2006DSST STEP Conf Call #239 MBRFC

40. May 17, 2006DSST STEP Conf Call #240 Define User Requirements Have user create own on the fly if capable Graphics within hydrograph plot if adequate description provided

41. May 17, 2006DSST STEP Conf Call #241 Vision of Operations Balance rivers without ensemble forecasts Switch on ensemble forecasting and look at rivers again Data assimilation used as guides to mods Displays of historical precip and streamflow to compare distributions

42. May 17, 2006DSST STEP Conf Call #242 Concerns How are segments linked? Is a precip value at one basin linked to a precip value at a basin nearby? Is trace for one location linked to a trace at the downstream location? Users will think uncertainty distribution in hydrograph is really a set of hydrographs Recent historical QPF and MAP are limited and have been dynamic with time

43. May 17, 2006DSST STEP Conf Call #243 Concerns (cont.) Basin boundary changes and addition of new basins require recomputing MAPs and QPFs Inconsistency between 14-day short term and 90-day long term How do you handle forecast locations that are model deficient and require manual override? Blending of regulation modeling? Use of mods in the future?

44. May 17, 2006DSST STEP Conf Call #244 Concerns (cont.) How can distributions be adjusted operationally? What can forecaster view to get some idea of possible needed adjustments to the distribution plots? Are ensemble data out of bounds?

45. May 17, 2006DSST STEP Conf Call #245 NCRFC

46. May 17, 2006DSST STEP Conf Call #246 Operations Concept – Ideas Need the ability to operate and generate forecasts at the sub-basin level (segment) Ensemble generation and viewing should be controlled by an option switch (on/off/default) to allow for interaction with the model for basic tuning before introducing complications of ensembles (assumes integration w/ IFP)

47. May 17, 2006DSST STEP Conf Call #247 Operations Concept – Ideas, cont. System needs full transparency for the forecaster – Summary views of inputs and model internals but with the capability to drill down and view detailed information –Essential for quality control and debugging Smart tools – to assist in diagnosis of the ensemble inputs, data assimilation, and results

48. May 17, 2006DSST STEP Conf Call #248 Operations Concept – Ideas, cont. Probabilistic hydrograph displays should include an overlay of historical ranges Historical perspective of Model inputs Selectable MODS – parameter to designate a MOD for deterministic, ensemble or both Drawing tool to redraw forecasts – necessary for timeliness and expediency in problem cases

49. May 17, 2006DSST STEP Conf Call #249 Issues and Challenges Hydraulic routings – complicates the ability to operate at the segment level Model BIAS correction – error model DA – ability to trace what it did and provide capability to undo or nudge differently Reservoirs – ensembles on autopilot can introduce unrealistic results

50. May 17, 2006DSST STEP Conf Call #250

51. May 17, 2006DSST STEP Conf Call #251 Issues and Challenges, cont. Ability to use alternate model sources (inputs) Need to develop meaningful displays for end-users to illustrate the model inputs (focus group requirement) Hand-off / Receive mechanism for upstream / downstream RFCs

52. May 17, 2006DSST STEP Conf Call #252 NERFC

53. May 17, 2006DSST STEP Conf Call #253 NERFC Requirements At least 3-5 days of precip & temperature data input Generate hydrograph ensembles out 5-7 days using graphics that have been reviewed by customers Generate SHEF output for selected confidence levels (or other means of having data readily available in AWIPS database)

54. May 17, 2006DSST STEP Conf Call #254 NERFC Concerns How do you generate 72+ hours of statistical ensemble input when normal operations never exceed 48 hrs of QPF? WFO Customer likes “what-if” contingency scenarios that are based upon known models Training for NWS staff needs to be included as part of development

55. May 17, 2006DSST STEP Conf Call #255 NWRFC

56. May 17, 2006DSST STEP Conf Call #256 The Shooting Target… Meteorological forcing for the hydrologic models originates from multiple sources. The final product is a combination of these procedures. Sources: HPC, WFO ISC/GFE grids, HAS, the numerical model of the day. Can the methodology be developed to determine the uncertainty and address all the inputs to the hydrologic model? How do we adapt these processes to the change in science and operational procedures?

57. May 17, 2006DSST STEP Conf Call #257 Scale… Meteorological forcing for the hydrologic models arrive at the RFC in many different scales. These are then converted to a base scale used in the hydrologic models. Can the current data networks adequately support the proposed level of downscaling? Will the uncertainty results be beneficial? Do we see a universal methodology to define the uncertainties generated by the downscaling?

58. May 17, 2006DSST STEP Conf Call #258 Transitions are where the money is… Defining the uncertainties is based on understanding the statistics between the forecast and the observation. Are we applying the uncertainties appropriately? Transitions or storms are where short-term forecasts make their money. The statistics and uncertainties during these events tend to shift from the climatology. Can we expect to provide a true picture of the uncertainties during the events that are most beneficial to short-term ensembles techniques?

59. May 17, 2006DSST STEP Conf Call #259 How does it work in a network… How do uncertainties translate from point to point in a hydrologic system?

60. May 17, 2006DSST STEP Conf Call #260 déjà vu… Short- and long-term ESP needs to address the man made influences; such as diversions and regulations. In the NWRFC area, over 90% of the forecast points are influenced by reservoir regulation and diversions. ESP is unable to work on a variable set of rules used by water managers. How do we quantify the uncertainty of forecaster MODS?

61. May 17, 2006DSST STEP Conf Call #261 Benefits… Hydrologic users like uncertainty information to assess the confidence in the forecast. Is there benefit and how is it determined? what is the timeline of short-term ESP being operational? will the uncertainty represent the true knowledge of the forecast error? need a system to develop and keep track of the uncertainty component to the forecast. NWS needs the ability deliver/provide more ESP derived information. requires a verification and validation system.

62. May 17, 2006DSST STEP Conf Call #262 1) 2) Similar displays or information as number 1, but for meteorological forcings. 3) Verification outputs which are converted to layman terms. Products Don wants…

63. May 17, 2006DSST STEP Conf Call #263 End of RFC presentations