Retrospective Evaluation of the Performance of Experimental Long-Lead Columbia River Streamflow Forecasts Climate Forecast and Estimated Initial Soil Moisture Forecast Ensemble Lead time = 8 months ENSO PDO Run Initialized Hydrologic Model Ensemble Streamflow Forecast Select Temperature and Precipitation Data from Historic Record Associated with Forecast Climate Category Climate Forecast Schematic for Streamflow Forecasting Methodology Using VIC Hydrologic Model and Resampled Observed Data Elevation (m) The Dalles Forecasts Effects of the Pacific Decadal Oscillation (warm and cool epochs) and ENSO (red El Niño, blue La Niña, green ENSO Neutral) on Columbia River Summer Streamflows at The Dalles, OR. Cool Warm Interpreting Ensemble Streamflow Forecasts The figures above show a time line for the long range streamflow forecasts, and a climatology based on simulated streamflows from To produce a forecast, a climate forecast and initial soil conditions are estimated on October 1 of each water year. Then meteorological sequences are resampled from the historic record from according to the climate forecast, and the initialized hydrologic model is used to simulate the natural streamflow for each of these resampled driving sequences. The forecasts are informed both by initial conditions and the climate forecasts. The primary objective of the forecast is to predict April-September average runoff, an important quantity in the Columbia’s management scheme. The lead time from the forecast to peak flows in mid summer is about eight months in this case study. Longer lead times of up to 12 months are also possible with very similar results. Overview of Streamflow Forecasting Methods Long-lead forecasts of the Pacific Decadal Oscillation (PDO) and ENSO, global climate phenomena which exert strong controls on Pacific Northwest winter climate, are used to produce streamflow forecasts with 8- month lead times using techniques shown in the schematic below. A long retrospective streamflow forecast was made for water years using the VIC hydrologic model. A perfect categorical forecast of winter average ENSO state (warm, neutral, cool) is assumed, and heuristic methods of predicting relatively uncertain decadal changes in the PDO are used. Simulated soil moisture storage on October 1 is quantified using the hydrologic model driven by observed meteorological data. The climate forecasts contain misclassifications of the PDO for some years, based on these methods ( Hamlet and Lettenmaier, 1999 and 2002). 1998, 1999, 2000 are classified as cool PDO based on the high flow event in 1997, although this classification is uncertain. Schematic and Timeline for Streamflow Forecasts Dept. of Civil Engineering, UW, Box , Seattle, WA Long-Range Streamflow Forecasting JISAO Climate Impacts Group and the Department of Civil and Environmental Engineering, University of Washington Key to Forecast Graphics Year shown on each plot is water year (Oct-September). Y axis is monthly mean streamflow in cms. Gray lines are the ensemble forecast members, light black lines are the highest and lowest values from the climatology, the red lines are simulated natural streamflows (from VIC), blue lines are estimated natural flows from observations. Colored circles in the upper left corner of each plot show the winter ENSO state for each forecast period. Red = warm ENSO, Green = ENSO neutral, Blue = cool ENSO. Forecast Performance Years in which the PDO state is misidentified are 1974, 1977, 1996(?) and 1997(?). Using April-September average streamflow as the forecast variable, the following performance is achieved: Overall reliability, defined as the probability of the observation being contained within the forecast envelope, is 82% Skill Score Quantitative performance of the ensemble forecasts relative to the 50-year simulated climatology is assessed using a Brier- based skill score defined as follows: Skill = 1 - [ 1/N1 *  (obs-forecast) 2 / 1/N2 *  (obs-climatology) 2 ] where the summations in the second term are over all forecast/observation pairs in each case. This performance metric rewards accuracy and punishes higher variability in the forecast ensemble. Using this metric, a perfect forecast would have a skill of 1.0, whereas forecast performance equivalent to using climatological ensemble would have a skill of 0. Skill of the forecast scheme assessed over all years is 0.078, an improvement of 8% over climatology. Conclusions For this fifty year retrospective assessment, the forecasts are shown to be 82% reliable, and improve on ensemble forecasts based on climatology by about 8% using a Brier-based skill metric. The forecasting technique is least effective at capturing unusual droughts that are infrequent or unobserved in the data set used for resampling. In the period from 1950 to 2000, for example, there were no droughts with the same severity as 1973 or 1977 in the same composite PDO/ENSO category, and as a result the observation is outside of the forecast envelope in these years. Increasing the length of the record of observations would improve this aspect of the forecasts, but problems with limited sample size are an inherent weakness in forecasting methods of this type. Streamflow (cms) Alan F. Hamlet Dennis P. Lettenmaier VIC Simulated Climatology