1. Overview of the Regional Technical Forum Guidelines January 22, 2013

2. After A Decade of Making Decisions the RTF Recently “Codified” Its Decision-Making Process

3. Why Guidelines For Estimating Savings? RTF Mission RTF Mission – Reliable estimates of savings – Transparent methods for estimating savings RTF operated for 10 years transparently but its “rule book” evolved with experience RTF operated for 10 years transparently but its “rule book” evolved with experience RTF needed to capture its current best practices to ensure more consistent decision-making and operations RTF needed to capture its current best practices to ensure more consistent decision-making and operations RESEARCHQUANTIFYDELIVERVERIFY

4. Brief History and Coming Events Guidelines Development – RTF identified need to codify guidance on how it does its work – Developed and tested guidelines for savings estimation methods. – Second version adopted 12-11-2013 – Also, developed guidance on how to estimate measure cost and lifetime Next Steps – Developing an overall Roadmap and integrating all guidelines and supporting tools

5. Guidelines Scope and Intent Describe how the RTF selects, develops and maintains approved methods for estimating savings from the delivery of energy efficiency measures Describe how the RTF selects, develops and maintains approved methods for estimating savings from the delivery of energy efficiency measures Four savings estimation methods: Four savings estimation methods: – Unit Energy Savings (UES) – RTF Decides on – Standard Protocols – RTF Decides on – Custom Measure Protocols – RTF provides guidance on – Program Impact Evaluations – RTF provides guidance on Each method is intended to produce savings estimates of “comparable reliability” at the lowest reasonable cost Each method is intended to produce savings estimates of “comparable reliability” at the lowest reasonable cost

6. Key Concepts – Measure and Savings Measure – …changes in system configuration, equipment specifications or operating practices – …reduces electric power consumption as a result of increases in the efficiency of energy use, production, or distribution Savings – …difference in energy use between the baseline and post periods – … caused by the delivery of a measure – …terms “net” or “gross” are intentionally not used

7. Savings Reliability and Diligent Review Savings Reliability – …sampling, data collection and other errors are sufficiently small and unbiased. – Sufficiency is determined by the collective opinion of the RTF Diligent Review – RTF uses estimates of parameters, e.g., average length of a residential shower or heat/cool interaction factors, from studies performed by other agencies – Diligent review required before use

8. Diligent Review Diligent review includes, but is not limited to understanding – characteristics of the sample studied – study’s data collection methods and analysis methods – variability of parameter estimates – feasibility of normalizing results to this region

9. Best Practice Savings Estimate – …relies on the best practical and reliable data collection and estimation methods – …can be carried out with proven techniques and resources deemed reasonable by the RTF – …may rely on parameter values from studies performed by other agencies, if found to be sufficiently reliable based on diligent review.

10. Two Possible Measure Baselines Current Practice Baseline – …used if the measure affects systems, equipment or practices that are at the end of their useful life – …includes measures delivering new equipment or practices – …baseline is defined by the recent typical choices of eligible end users in purchasing new equipment and services Pre-Conditions Baseline – …used when the measure-affected equipment or practice still has remaining useful life – …defined by typical existing conditions found among eligible end users – For custom measures choices of individual end users define typical current practice conditions

11. Transparency and Accessibility All methods have documentation standards Sources must be cited Data and documents must be accessible Must show your work – Estimation algorithms – Derivation of input parameters – Software must be either inherently transparent, such as a Microsoft Excel ® workbook or fully documented

12. Method 1 - Unit Energy Savings (UES) RTF approves measure methods and savings values RTF approves measure methods and savings values Stable, unitized savings that can be reliably forecast through the period defined by the measure’s sunset date Stable, unitized savings that can be reliably forecast through the period defined by the measure’s sunset date Reduce program delivery costs by simplifying the data that must be collected Reduce program delivery costs by simplifying the data that must be collected – Verified count of delivered units is all that is required – Statistical or meta-statistical analysis used to reliably estimate savings Examples: Examples: CFL Res. Insulation

13. Getting to Unitized Savings (UES)* Statistical or meta-statistical data Statistical or meta-statistical data – Quality judged by relative error of mean savings estimate – Avoid when savings significantly interact with other measures due to large sample needs Calibrated engineering models Calibrated engineering models – Adjusted to individual cases or to the average characteristics and consumption of groups Ex. SEEM heating loads calibrated to billing data from representative sample of SF homes Ex. SEEM heating loads calibrated to billing data from representative sample of SF homes – Savings expected to be regionally applicable – Significant interactions need to be dealt with Planning/Provisional provides a path Planning/Provisional provides a path *The savings formerly known as “deemed”

14. Method 2 - Standard Protocols RTF approves saving estimation method RTF approves saving estimation method Intended for measures with widely varying savings, but where data collection and analysis can be standardized Intended for measures with widely varying savings, but where data collection and analysis can be standardized Site-specific data collection and analysis is the minimum required for a reliable savings estimation Site-specific data collection and analysis is the minimum required for a reliable savings estimation – Known as simplest reliable savings estimation method, +/- 20% error that is unbiased compared to best practice Data collection standardized and savings calculator developed for each protocol Data collection standardized and savings calculator developed for each protocol Examples: Examples: T8 Lighting VFD

15. Method 3 - Custom Protocols RTF provides guidance on appropriate methods and documentation standard Appropriate for measures that require site-specific savings estimation planning, data collection and analysis – Likely numerous components affecting operation, maintenance, and energy use of many separate systems or pieces of equipment Savings report required Savings report required Example: Example: Energy Management System (EMS)

16. Method 4 – Program Impact Evaluation RTF provides guidance on appropriate methods and documentation standard achieved during a program delivery period Based on reliable sample of program participants (and possibly non-participants) to determine the savings achieved during a program delivery period – UES claim & delivery; Standard Protocol faithfully applied Rely on the other three methods, plus guidance provided for “Other UES” measures Examples: Examples: O-POWER

17. Measure Category/Quality Standards Statistical or calibrated engineering data are available and reliable to characterize both the baseline and efficient-case energy consumption for measure-affected end uses Proven RTF approves with special conditions requiring the collection of data from all or a sample of specific measure applications to improve savings estimation Provisional Peer reviewed by RTF for measures with regional applicability and reasonable expectation that data will be collected to bring measure through an RTF path in the future Planning RTF approves based on sound engineering analysis and applicability to the region and because cost to obtain quality data outweighs expected regional savings potential Small Saver

18. Guidelines Measure Status Active Measure meets all requirements set forth in GuidelinesMeasure meets all requirements set forth in Guidelines Under Review Errors need to be correctedErrors need to be corrected Calculation updates with new identified data sourcesCalculation updates with new identified data sources Calculation updates with data to be developed or soughtCalculation updates with data to be developed or sought Out of Compliance Lack of data sources identified that can bring the savings estimations into guideline complianceLack of data sources identified that can bring the savings estimations into guideline compliance 1 year to bring research plan to RTF for review1 year to bring research plan to RTF for review De-activated Inadequate data exists to bring into compliance with GuidelinesInadequate data exists to bring into compliance with Guidelines Federal/State codes require target efficiency levelFederal/State codes require target efficiency level

19. Sunset Date As part of approving a UES or standard protocol measure, the RTF will establish a sunset date for the measure Beyond this date, the RTF may decide that the measure’s savings estimation method is no longer RTF-approved. …RTF records factors that justify each sunset date… – revisions to energy codes or federal standards – completion of RTF approved research plans – shifts in current practices of consumers

20. Provisional Category and Research Planning Defined for UES and Standard Protocol RTF approves plans for research needed to advance provisional measure to the proven category Provisional UES measures need data on efficient-case energy use Provisional standard protocol measures need data to prove simplest reliable method

21. Development of Application of RTF- Savings Estimation Methods

23. Cost and Lifetime Guidelines RTF also approved guidelines for measure cost and lifetime estimation in 2011 Capital and O&M costs along with a measure’s expected useful lifetime are used by the RTF in evaluating regional cost-effectiveness using the ProCost model Program operators may use these values in performing their own cost-effectiveness analyses