1. Analysis of Blue Mesa Inflow Forecast Errors Tom Pagano, Tom.Pagano@por.usda.gov 503 414 3010 aka: “Wha’ happa’???”

2. Performance of 2008 forecasts

3. NRCS daily forecasts Official coordinated forecasts Daily forecast skill Observed http://www.wcc.nrcs.usda.gov/wsf/daily_forecasts.html

4. 2008 Official historical published outlooks issued April 1

5. Blue Mesa + Taylor Park Reservoir Storage Kac ft Full Aug 1 Jan 1 Forecasts and storage have come up short since 2000

6. 2008 obs Historical NRCS “End of season” daily forecast skill This takes into account all available data including spring/summer precipitation. Even then, this year still falls out as a big anomaly (major over-forecast)

7. April-July 2008 Blue Mesa Inflow Forecast Comparison Forecast issue date official 1971-00 norm April-July inflow kac-ft NRCS daily forecasts based on: Precip/Snow obs

8. April-July 2008 Blue Mesa Inflow Forecast Comparison Forecast issue date official 1971-00 norm April-July inflow kac-ft NRCS daily forecasts based on: Snow only Precip/Snow Precip only obs

9. April-July 2008 Blue Mesa Inflow Forecast Comparison Forecast issue date official NWS ESP 1971-00 norm April-July inflow kac-ft NRCS daily forecasts based on: Snow only Precip/Snow Precip only obs

10. NRCS Daily Forecasts for Apr-Jul Blue Mesa Inflow using snow information available by April 11 Forecast too high Forecast too low 2008 snowpack composite well beyond recent historical variability Daily forecast input is a composite of 7 snotel sites: North lost trail, butte, park cone, brumley, independence pass, porphyry creek, slumgullion

11. 2008 Historical range 1981-2007

12. Park Cone Historical April 1 Snow Data Snow water equivalent (in) 2008… 2 nd biggest in last 42 years 6 th biggest in last 72 years

13. End of season wyprecip End of season snow Apr 11 snow Apr 11 wyprecip

14. Sources of predictability 1950-99 VIC model skill (University of Washington) Explained variance in predicting Apr-July runoff Blue – Snowpack Green – Soil Moisture Red – El Nino Darker colors- more important (courtesy of M Dettinger, Scripps) What basin processes are important? How have they been behaving recently? (using actual and model data) How is runoff driven by: snow and precipitation soil moisture evaporation/temperature Note: Model is not necessarily reality

15. snow and precipitation

16. Snow and precipitation are easily the dominant factor R 2 (% var expl) 1.0 = perfect better worse Blue Mesa April-July Inflow Forecast Skill 1979-2007 Model swe + precip versus Model runoff Univ Washington model Snow + Precip NRCS Daily Forecasts Real world SNOTEL vs Real world runoff Issue date of forecast Snow/precipitation explain upwards of 90% of the year to year variability in runoff

17. Gunnison Basin April-June Precipitation 1920-2008 Percent of 1971-2000 average Data from WESTMAP: http://www.cefa.dri.edu/Westmap/ 2008 72% 2002 30% 1995 206% 2000-08 avg: 75% norm Precipitation after April 1 is important

18. Spring precipitation, especially the sequencing with snowmelt is also important Runoff Snowmelt Rainfall Rainfall mixed with snowmelt “normal” April July

19. Spring precipitation, especially the sequencing with snowmelt is also important Runoff Snowmelt Rainfall Rainfall mixed with snowmelt “normal” Rainfall boosting snowmelt Larger volumes Snowmelt and rainfall separate Not enough “momentum” to produce big volumes All these complex interactions are tough to “cartoonize”; Simulation models can handle this… but still it’s tough to predict beyond 1-2 weeks. April July

20. Spring precipitation, especially the sequencing with snowmelt is also important April July Runoff Snowmelt Rainfall Rainfall mixed with snowmelt “normal” Rainfall boosting snowmelt Larger volumes Snowmelt and rainfall separate Not enough “momentum” to produce big volumes Even then, however, high heat and no rain can lead to “pouring sunshine” All these complex interactions are tough to “cartoonize”; Simulation models can handle this… but still it’s tough to predict beyond 1-2 weeks.

21. 2008 Runoff Blue Mesa natural inflow kac-ft/day Total: Independ Pass Snowmelt Rainfall Schofield Snowmelt Rainfall (inches) Many fits and starts to snowmelt… Almost no spring rainfall

22. 2008 Runoff Blue Mesa natural inflow kac-ft/day Total: Independ Pass Snowmelt Rainfall Schofield Snowmelt Rainfall (inches) 1999 Good mix of rain and snowmelt 1993

23. 2008 Runoff Blue Mesa natural inflow kac-ft/day Total: Independ Pass Snowmelt Rainfall Schofield Snowmelt Rainfall (inches) 1999 Snowpack poor, but “perfect storm” for runoff efficiency

24. Soil moisture/Groundwater Are we still feeling the effects of 2002?

25. Blue Mesa Basin Soil Moisture 2001-2008 (According to the Univ Washington Model- top 2 layers)

26. Blue Mesa Basin Soil Moisture 2001-2008 (According to the Univ Washington Model- top 2 layers) (According to Park Cone Snotel- ~0-30” depth) Snotel does poorly in frozen soils, so that has been censored Model resembles snotel, but also remember we’re comparing basin average with point measurement

27. Blue Mesa Basin Soil Moisture 2001-2008 (According to the Univ Washington Model- top 2 layers) (According to Park Cone Snotel- ~0-30” depth) Snotel does poorly in frozen soils, so that has been censored Univ Washington Model “deep” soil moisture layer

28. Colorado Active Well Level Network Only one Colorado USGS groundwater station in realtime (Pueblo) Gunnison: Period of record 2003-2008 + 1996 Measured 1x/year Crested Butte 8/1996 Taken in Mid-may

29. What influence humans? Does it matter? Blue Mesa For each site, all measurements Jan-Jun, Jul-Dec are averaged by year. Station half-year data then converted into standardized anomaly (o-avg(o))/std(o) vs period of record for the half year. Multiple stations are then averaged.

30. Univ Washington model Blue Mesa inflow basin total soil moisture (mm) January 1 1920-2008 Start of 2008

31. Evapotranspiration/sublimation

32. Butte SNOTEL sublimation (as modeled by NOHRSC) Water year to date cumulative sublimation (inches) Oct-Jul Subl Year Precip Prcp 2004 19.3”30% 2003 19.6” 29% 2005 24.5”15% 2006 22.7”16% 2007 18.1”18% 2008 29.3”10% Long-term average evaporation + transpiration + sublimation = 73% of annual precipitation

33. Gunnison Basin March-May Average Temperature 1920-2008 Departure from 1971-2000 Data from WESTMAP: http://www.cefa.dri.edu/Westmap/ Warm spring temps recently except last year

34. How good are forecasts in general? “Perfect forecasts are all alike; Every bad forecast is bad in its own way.”

35. “Published Official” -Subjective (based on objective guidance) -Humans actively involved -Coordinated by NRCS+NWS

36. “NRCS Daily” -Objective -Statistical model-based -Highly automated -Only uses SNOTEL snow+wytd precip http://www.wcc.nrcs.usda.gov/wsf/daily_forecasts.html

37. http://www.hydro.washington.edu/forecast/westwide/sflow/index.shtml University of Washington -Objective -Simulation model-based -Highly automated -Research grade

38. NWS ESP -Objective -Simulation model-based -Human controlled/vetted (in realtime)

39. Sources Natural flow: (1968-2008) http://www.usbr.gov/uc/crsp/GetSiteInfo Blue Mesa outflow – Taylor Park change in storage Published official: (1971-2008) ftp://ftp.wcc.nrcs.usda.gov/data/water/forecast/ With gaps filled in from other sources NWS ESP: (1981-2002) http://www.nwrfc.noaa.gov/westernwater/database/index.php?id=BMDC2#forecasts University of Washington: (1971-2008) http://www.hydro.washington.edu/forecast/hepex/esp_cmpr/ NRCS Daily: (1979-2008) http://www.wcc.nrcs.usda.gov/wsf/daily_forecasts.html All reforecasts available internally, some available online

40. Blue Mesa April-July Inflow Forecast Skill 1981-2002 RMSE as % 1971-2000 Normal 0 = perfect worse better Published official NRCS Daily Issue Month of Forecast (e.g. January 1) RMSE = sqrt(avg(f-o) 2 ) Period common to all datasets

41. Blue Mesa April-July Inflow Forecast Skill 1981-2002 RMSE as % 1971-2000 Normal 0 = perfect worse better Published official NRCS Daily NRCS Daily (residual) Issue Month of Forecast (e.g. January 1) RMSE = sqrt(avg(f-o) 2 )

42. Model Forecast Observed Model Simulated Forecasting runoff from a start date in June…

43. Model Forecast Observed Model Simulated Forecasting runoff from a start date in June… Is your April-July “forecast” your 1. observed + future only forecast or is it 2. simulated + future only forecast? Forecasting “residuals” (1.) is more accurate but is an “open book” exam

44. Blue Mesa April-July Inflow Forecast Skill 1981-2002 RMSE as % 1971-2000 Normal 0 = perfect worse better Published official NRCS Daily NRCS Daily (residual) Issue Month of Forecast (e.g. January 1) RMSE = sqrt(avg(f-o) 2 )

45. Blue Mesa April-July Inflow Forecast Skill 1981-2002 RMSE as % 1971-2000 Normal 0 = perfect worse better Published official NRCS Daily NRCS Daily (residual) University of Washington Issue Month of Forecast (e.g. January 1) RMSE = sqrt(avg(f-o) 2 )

46. Blue Mesa April-July Inflow Forecast Skill 1981-2002 RMSE as % 1971-2000 Normal 0 = perfect worse better Published official NRCS Daily NRCS Daily (residual) University of Washington NWS ESP Issue Month of Forecast (e.g. January 1) RMSE = sqrt(avg(f-o) 2 )

47. Blue Mesa April-July Inflow Forecast Skill 1981-2002 R 2 (% var expl) 100 = perfect better worse Published official NRCS Daily NRCS Daily (residual) University of Washington NWS ESP Issue Month of Forecast (e.g. January 1) What’s different: This does not penalize for bias

48. Blue Mesa April-July Inflow Forecast Bias 1981-2002 Forecast Bias as % 1971-2000 Normal Forecasts too high Forecasts too low Issue Month of Forecast (e.g. January 1) Published official NRCS Daily University of Washington NWS ESP Bias = avg(f) – avg(o)

49. Blue Mesa April-July Inflow Forecast Bias 2000-2007 Forecast Bias as % 1971-2000 Normal Forecasts too high Forecasts too low Issue Month of Forecast (e.g. January 1) Published official NRCS Daily University of Washington NWS ESP (not available) Bias = avg(f) – avg(o) Note: 2000-2007 were dry years. Comparatively, the 71-00 normal had a “bias” of +36% (!)

50. Conclusions From a snow perspective, 2008 was an epic bust. Water year precipitation matched runoff however. Lack of spring precip and sequencing of snowmelt important All existing models have comparable skill. 30% normal error is typical Jan 1, 20-25% error on April 1 Snow, rainfall, evap, soil moisture important to runoff. Some easier to quantify than others. Recommendations Have local/anecdotal observations feed quantitative historical analyses to build appropriate models. Collect the right data to support creation of objective operational guidance, tempered by reason. Recognize the unknowable versus the unknown. Remember too that higher accuracy is not the only way to improve forecasts

51. Snow sublimation (inches/day) Snow sublimation (inches/day) Daily average wind speed (mph) Daily average relative humidity (%) Butte SNOTEL Site Data from NOHRSC model Nov 2002-Aug 2008 www.nohrsc.nws.gov Note that winds don’t vary from year to year in most hydrologic models (e.g. NWS and UWashington)

52. Blue Mesa Basin Evaporation Losses by Month (according to the Univ Washington Model) Monthly average temperatures (deg F) Evapo-transpiration (kaf/month) April-June: temperatures and ET related July-September: ET moisture driven November-March: ET constant, small jan feb mar apr may jun jul aug sep oct novdec

53. Blue Mesa Basin Water Year Water Balance (according to the Univ Washington Model) Kac-ft/year Long-term average Evapotransporation (ET)/Precipitation = 73% ET doesn’t vary much from year to year. Even then, 65% of its variability depends annual precipitation (at least in the model’s reality…) Precipitation Runoff Evapotranspiration