National Association of Regulatory Commissioners – February 2008 Page 1 of 13 National Association of Regulatory Utility Commissioners Winter Committee Meeting _________________________________________________________ Energy Efficiency Program Performance Evaluation Issues from the States Nick Hall TecMarket Works

National Association of Regulatory Commissioners – February 2008 Page 2 of 13 5 Topics Covered Today 1.Changing Importance of Evaluation –Need for accuracy – reliability - objectivity 2.Reliability & Budgeting for Evaluation 3.Evaluation Approaches Used 4.Net-to-Gross Issues –Definitions change from state to state –Net savings are difficult to assess –Approaches for crediting savings 5.Changing way EE is viewed / assessed

National Association of Regulatory Commissioners – February 2008 Page 3 of Changing Importance of Evaluation Evaluation provides decision-support information in a changing environment… –More people understand the true cost of fossil fuel & nuclear power supply and are demanding change. –Energy Efficiency represents the single largest impact potential for greenhouse gas reduction. –Energy efficiency is moving to be the resource of choice. –Public resistance to new plants, including some renewable energy facilities. –ISO’s, Commissioners, ALJs and others are asking for more and more reliable evaluations. Accurate information is essential for Policy Decisions.

National Association of Regulatory Commissioners – February 2008 Page 4 of Reliability & Evaluation Budgeting Several state are worried about the accuracy and reliability of evaluation results –Accuracy & reliability are controlled by the evaluation budget & the evaluation timeline. –States are struggling with evaluation budget decisions. –Some are legislating budgets without understanding the effects. –Currently some states are setting evaluation budgets below the level of accuracy desired.

National Association of Regulatory Commissioners – February 2008 Page 5 of Reliability & Evaluation Budgeting Relationship between funding and results Strong Funding – Right Approach Weak Funding – Right Approach Weak Funding – Wrong Approach Strong Funding – Wrong Approach

National Association of Regulatory Commissioners – February 2008 Page 6 of Budgeting for Evaluation NY2%Moved from 1% ILL3%Set in legislation WI4%Moved up from 3% MI3-5%Proposed - Now being discussed KY5%Proposed - Now being discussed OH5%Proposed - Now being discussed NC5%Proposed - Now being discussed SC5%Proposed - Now being discussed IN5%Proposed - Now being discussed CA8%Moved up from 4.25%

National Association of Regulatory Commissioners – February 2008 Page 7 of Evaluation Approaches Used Impact Evaluation – Energy Impacts Majority of Studies Fall Under 5 General Approaches *. States pick and choose according to budget and reliability needs. 1.Engineering Estimates 2.Statistically adjusted Engineering Estimates 3.Building Meter/Billing Analysis 4.Measure Level Metering & Monitoring 5.Building Modeling and Simulations * See California Evaluation Framework/Protocols for others.

National Association of Regulatory Commissioners – February 2008 Page 8 of Evaluation Approaches Used Impact Evaluation – Energy Impacts Reliability Issues 1.Identifying and setting the baselines 2.On-site confirmation / metering / assessments 3.Sample Sizes 4.Representativeness of the Sample 5.Type of metering or monitoring used 6.Accuracy of self-reports

National Association of Regulatory Commissioners – February 2008 Page 9 of Net to Gross Issues Gross savings tend to be defined the same way. Ex Ante - Gross Ex ante savings x tracking system measure installs. Ex ante savings x measure installs x verification rate. Ex Post - Gross Ex post savings x measure installs x verification rate. Net savings tend to be defined in different ways. Net Savings = Gross Savings +/- Net Adjustments

National Association of Regulatory Commissioners – February 2008 Page 10 of Net to Gross Issues Net savings definitions and measurement approaches are not consistent across states, and sometimes across studies within a state. Different definitions from state to state Freeriders (10% to 60% of the program-claimed savings) Participant spillover (5% to 40% of the program-claimed saving) Short term non-participant spillover - market effects (5% to 20% of the program-claimed savings) Long-term non-participant spillover market effects (40% to >500% of the program-claimed savings) Study results are inconsistent and not comparable.

National Association of Regulatory Commissioners – February 2008 Page 11 of NTG Definitions Used NYGross – freeriders Can adjust for PS & ME ILLGross – freeriders + part spilloverNot yet agreed WIGross – freeriders + MEAs documented in eval MINot yet determinedNot yet agreed KYGross – freeriders + adjustments As documented in eval OHGross – freeriders + adjustments As documented in eval NCGross – freeriders + adjustments As documented in eval SCGross – freeriders + adjustments As documented in eval INGross – freeriders + adjustments As documented in eval CAGross – freeridersNow looking at options

National Association of Regulatory Commissioners – February 2008 Page 12 of Changing way EE is viewed / assessed State are considering changes that will impact EE evaluation and cost effectiveness assessments. 1. Elimination of policy-set life-cycle periods (EUL) 2. Valuing energy saved at market prices 3.Valuing greenhouse gas at total societal costs, not traded values 4.Setting renewable energy as the cost alternative to EE in cost comparison approaches (i.e. avoided costs = RE) 5. Discontinuing discounting of future savings. 6. EE as a profitable energy supply choice.

National Association of Regulatory Commissioners – February 2008 Page 13 of 13 Thank You Nick Hall TecMarket Works 165 West Netherwood Road Suite A, 2 nd Floor Oregon, Wisconsin