1. Guidelines for the Development and Maintenance of RTF- Approved Measure Savings Estimates December 7, 2010 Regional Technical Forum Presented by: Michael Baker

2. Why Do We Need Guidelines Define a traceable and transparent decision- making process Achieve reliability commensurate with measure savings and benefits Base decisions on consistent and transparent savings estimates 2 12/7/2010

3. Types of savings estimation methods Deemed Unit Energy Savings (UES)  Program must reliably capture measure identification data, e.g., single family forced-air electric heat  Savings = Delivered units X appropriate UES Standard Protocols  Standardized data collection and analysis  Site-specific savings estimate Custom Protocol  Site-specific data collection, analysis and savings estimate 3 12/7/2010

4. Deemed Unit Energy Savings 4 12/7/2010

5. Measure Specification Eligible Applications Measure Identifiers Savings Baseline  Upgrade New - recent typical choices of eligible end users in purchasing new equipment and services  Upgrade, Replacement and Conversion - typical existing conditions found among eligible end users Implementation Standards Product Standards 5 12/7/2010

6. UES Estimation Procedure Statistical  Experimental (treatment and control)  Quasi – Experimental (pre / post) Meta-Statistical  A number of small studies  Central tendency / lower bound deemed reliable Calibrated Engineering  Required for highly interactive measures  Calibration to baseline and efficient case measure effected end use consumption 6 12/7/2010

7. Calibrated Engineering Procedures Aspects to consider in judging reliability:  Input Parameters  Model Calibration  UES Components  Interactions between Measures Last-In Rule  Heating/Cooling Interactions 7 12/7/2010

8. Provisional - Required Quality of Savings Estimate Reliable baseline consumption for measure- affected end uses Models calibrated to baseline end use consumption Sound engineering and statistical analyses Programs commit to collect efficient case consumption of the measure-affected end uses 8 12/7/2010

9. Deemed - Required Quality of Savings Estimate Statistical or meta-statistical  Findings are determined to be sufficiently reliable Calibrated Engineering  Reliable baseline and efficient-case energy consumption for measure-affected end uses  Sound engineering and statistical analyses  Models calibrated to both baseline and efficient-case end use energy consumption 9 12/7/2010

10. Value-Based Standard for Reliability Many factors contribute to reliability:  Sample design and size  measurement procedure  Validity of the model specification and model calibration RTF collective judgment determines sufficient reliability RTF may require that up to 3% of measure life cycle benefits be spent to achieve sufficiency This is an upper limit on the spending Proposer and RTF should always seek the least cost strategy 10 12/7/2010

11. Development and Maintenance Process Proposed Provisional  Until adequate data for deemed or de-activated Deemed  Sunset criteria Under Review  UES can still be used De-Activated  UES no longer approved 11 12/7/2010

12. Special Status Pilot  Upon request, the RTF will provide a review and comment on pilot projects intended to collect data needed to satisfy the criteria for either provisional or deemed measures Out-Of-Compliance - Measures approved prior to these guidelines  UES estimates continue to be approved if a plan for bringing the measure into compliance is adopted within one year following adoption of these guidelines 12 12/7/2010

13. Transparent Estimates of Savings Summary Sheet – Four Tables  Measure Classification and Properties  Measure Identifiers  Constant Parameters  Unit Energy Savings (UES) Estimation Method, Parameters and Sources See Appendix A for an example 13 12/7/2010

14. Standard Protocols 14 12/7/2010

15. What is a Standard Protocol Document describing precisely the data collection and analysis required to estimate savings. Transparent calculator that implements exactly the savings estimation procedure See Appendix B for an example. 15 12/7/2010

16. Purpose of Standard Protocols Standardization of data collection and analysis for selected measures or closely related groups of measures  Eliminates the need for site-specific measurement planning  Allows for uniform quality control tests  Reduces required skill levels  Reduces costs of savings estimation 16 12/7/2010

17. Requirements for Standard Protocols Precise specification of the measure including eligibility rules Simplest reliable savings estimation procedure Entirely prescriptive data collection and analysis procedure that will work for all eligible projects Independent of program design and delivery method 17 12/7/2010

18. More Requirements for Standard Protocols Skills required are common among the region’s program implementation workforce Protocol is cost-effective across a wide range of project savings Yield savings estimates that are additive, i.e., independent of the savings from other measures implemented at the same site Protocol documentation complies with section 3.5 18 12/7/2010

19. Best Practice Savings Estimate Obtain best practice examples from regional or national sources  Power measurements for affected systems  Measurements of primary determinants  Measurements of less expensive and simpler surrogates for primary determinants  Pre and Post measurements Develop best practice savings estimation model 19 12/7/2010

20. Simplest Reliable Savings Estimation Method Goal is +/- 20% estimation error for each site and unbiased estimate Systematically explore simplification strategies  Trending current  Estimating primary determinants from surrogates  Using performance curves to estimate pre  Minimizing measurement duration Select simplest reliable method 20 12/7/2010

21. Protocol Template Purpose Definition of Key Terms Eligible Projects Required Knowledge and Skills of Practitioner Required Commissioning Data Collection Requirements Savings Estimation Steps 21 12/7/2010

22. More Protocol Template Sampling Procedure Relationship to Other Protocols and Guidelines Estimate of Typical Cost Provisional Data Collection User’s Guide to Savings Calculator 22 12/7/2010

23. Transparent and Accessible Savings Calculator Calculator must accept exactly the data that is required by the protocol Implemented using computer software that is generally accessible and reasonable priced for all practitioners Either inherently transparent, e.g. Excel, or fully documented 23 12/7/2010

24. Provisional Status and Data Requirements Number and quality of best practice examples may not be adequate to definitively support the selection of a simplified savings estimation procedure Provisional version of the protocol with a sunset criteria Additional data collection requirements All data made available to RTF 24 12/7/2010

25. Custom Protocol Outside the scope of SBW’s current contract Prior work by IPMVP, US DOE FEMP, the RTF, BPA, regional utilities and other agencies should be considered in developing guidance for the custom protocol 25 12/7/2010