1. TOPICS 2013 Custom Impact Overview 2013 Custom Impact Elements Evaluation Results Gross Impact Findings Net Impact Findings Project Practices Assessment (PPA) Findings Selected IALC 2013 Findings and Recommendations Q&A/Discussion Emphasis on new sampling and evaluation approaches, tailored scope allowing for measure- and program-level results, and new recommendations 1 2013-14 INDUSTRIAL, AGRICULTURAL AND LARGE COMMERCIAL (IALC) ROADMAP DRAFT REPORT WEBINAR PY2013 CUSTOM IMPACT EVALUATION April 20, 2015

2. IALC CUSTOM IMPACT EVALUATION AGENDA »IALC Evaluation Overview and Objectives »Data Collection and Approach »Evaluation Sample by IOU »Aggregation of Sample Results »Gross Impact Results Discrepancy Factor Impacts Discrepancy Factor Assessment »Net Impact Results »Project Practices Assessment (PPA) Results »IALC 2013 Recommendations 2

3. 2013 IALC CUSTOM IMPACT OVERVIEW 3 »Evaluation of Custom (non-deemed measures) across multiple programs and the four major California IOUs Core Calculated Programs Third Party Programs Local Government Partnerships and Universities / Colleges And More!! Over 100 programs within the IALC 2013 scope »Why is Custom Impact Evaluation Important? Custom measures account for 19% of electric [499 GWh] and 56% of statewide natural gas savings claims [36 Million therms] History of low realization rates for savings claims - How do we address the disconnect between ex-ante and ex- post project treatment?

4. IALC EVALUATION OVERVIEW »IALC Roadmap is composed of nonresidential, non-deemed claims: Non-deemed lighting measures moved to Commercial RM »PY2013 IALC savings claims: 4

5. CUSTOM EVALUATION OBJECTIVES »Custom Priorities and Researchable Issues Estimate achieved gross impacts  identify factors affecting gross realization rates (GRRs)  identify opportunities for improvement Estimate free ridership  identify factors affecting free ridership  identify opportunities for improvement Evaluate PA conformance with CPUC policies, decisions, and best practices Collect data and information to assist with other research or study areas 5

6. DATA COLLECTION AND APPROACH »The gross impact evaluation developed gross realization rates (GRRs) for statistically representative samples of custom projects The gross impact evaluation utilized project-specific measurement and verification (M&V) to estimate GRRs Greater M&V rigor was used for strata 1 and 2 projects - Analysis involved on-site metering and extensive engineering analysis to calculate the ex-post impact of sampled strata 1 and 2 projects - Smaller projects received less rigor 190 PY2013 M&V points were targeted and 189 were achieved 6

7. DATA COLLECTION AND APPROACH 7

8. »The NTG evaluation used an interview-based approach for a representative sample of selected projects, yielding NTG ratios »Similar to the gross impact approach, a mix of rigorous professional interviews and basic rigor CATI surveys were conducted »153 PY2013 points were targeted and 146 were achieved »Gross and net impact sampling by PA Electric and gas savings were collapsed Sampling and analysis used source energy equivalents* * Conversion rates obtained from “2001 Energy Efficiency Standards for Residential and Non-residential Buildings, California Energy Commission,” June 2001: 1 kWh = 10,239 Btu source energy; 1 Therm = 100,000 Btu source energy. 1 MMBtu =1,000,000 Btu 8

9. AGGREGATION OF SAMPLE RESULTS 9

10. GROSS IMPACT RESULTS Lifecycle Gross Realization Rates – All IOUs 10

11. GROSS IMPACT RESULTS Mean Lifecycle Gross Realization Rates by IOU and Energy Metric (MMBtu and kW) 11

12. GROSS IMPACT RESULTS Mean Lifecycle Gross Realization Rates by IOU and Energy Metric (MMBtu and kW) »Mean lifecycle RR by PA and energy metric all less than 0.70 (except GRR for SDG&E kW = 0.79) »All weighted lifecycle RRs lower than first year RRs »Primarily due to adjustments in EUL (overstated for 71 of 189 sampled projects) »All PAs had projects with negative or zero GRRs (30 of 189) 12

13. GROSS IMPACT RESULTS Discrepancy Factors – Downward and Upward Adjustments 13

14. GROSS IMPACT RESULTS Most Influential Discrepancy Factors »Largest downward impacts on FY MMBtu (by degree of savings impact) »Differences in operating conditions »Calculation methods »Inappropriate baselines or baseline conditions »Ineligible measures »Others include: inoperable measure, measure count, tracking discrepancies »For zero-savers, most frequent discrepancies are: »Inappropriate baselines »Ineligible measures 14

15. GROSS IMPACT RESULTS Most Influential Discrepancy Factors 15

16. GROSS IMPACT RESULTS Most Influential Discrepancy Factors by IOU – Example 16

17. NET IMPACT RESULTS 17

18. NET IMPACT RESULTS »Corporate standard practice and corporate environmental policies »Factors contributing to low program influence Project included in capital or operating budget Measure installed elsewhere by same company without rebate Measure automates existing manual practice Measure associated with environmental compliance »Factors contributing to higher program influence Timing of decision relative to discussions with program staff First-time installation of measure Energy efficiency is the sole or primary motivator Program rebate is a high percentage of project cost (> 25%) Most Influential Factors across IOUs 18

19. PPA Overview PROJECT PRACTICES ASSESSMENT (PPA) FINDINGS »Builds upon Lower Rigor Assessment (LRA) in ‘10/12 WO33 »Systematically examines PA ex-ante savings development practices »More targeted than LRA with goal of targeting areas for improvement »Provides quantitative feedback based on qualitative information »Analysis segregated for pre-2013 and 2013+ project agreement date »Capture effects of 2012 EAR policy guidance »Clear improvement between periods not observed for any PA except SDG&E (only 9 2013+ sample points) 19

20. PPA Overview PROJECT PRACTICES ASSESSMENT (PPA) FINDINGS » Includes examination of: »Project eligibility »Baseline selection »Project EUL, RUL, costs and incentives »Calculation methods, inputs and assumptions » Rating-based examination of: »Comprehensiveness of documentation »Quality and appropriateness of ex-ante results and conclusions » General rating scale: »1 – Does not meet basic expectations »2 – Minimal effort to meet expectations »3 – Meets expectations »4 – Exceeds expectations »5 – Consistently exceeds expectations 20

21. Project Eligibility PROJECT PRACTICES ASSESSMENT (PPA) FINDINGS » No clear improvement in appropriateness of project eligibility or quality of documentation » Primary reasons for eligibility failure: »CPUC decisions / guidance »Requirement that measures exceed code / ISP baseline »Previous EAR guidance 21

22. Project Types – All PAs PROJECT PRACTICES ASSESSMENT (PPA) FINDINGS » Generally good agreement  ER and ROB most frequently overturned project types 22

23. Project Baselines – All PAs PROJECT PRACTICES ASSESSMENT (PPA) FINDINGS » Generally good agreement  Existing equipment most commonly overturned project baseline (to ISP or code) 23

24. Project Documentation PROJECT PRACTICES ASSESSMENT (PPA) FINDINGS » Overturned ER, NR, ROB project types due to lack of documentation of: »Age and condition of existing equipment »EULs/RULs »Capabilities of existing / baseline equipment to perform service »Available efficiency levels » Overturned existing equipment baseline often due to: »Lack of PA review of ISP and applicable codes and standards » Instances of regressive baseline and subsequent ineligibility due to: »Lack of PA review of normal facility practices 24

25. Documentation Assessment for Selected Parameters PROJECT PRACTICES ASSESSMENT (PPA) FINDINGS » EUL documentation is maintained in less than 50 percent of custom application files. Evaluation EUL 2 years shorter, on average  downward effect on LC GRR » Where directly relevant, such as for ER claims, RUL is generally not documented in custom application files. » Where indirectly relevant, incremental cost is only documented in custom application files about 40 percent of the time. Incremental cost values for applicable measures were especially deficient and were populated only for 45 of 104 measures (43 percent). 25

26. OVERARCHING FINDINGS AND RECOMMENDATIONS » Lifecycle MMBtu GRRs remain low (0.44 to 0.63) » Relative to 2010-2012 LC GRRs in 2013 were: »PG&E – higher by 5% »SCE – higher by 3% »SCE – lower by 28% »SDG&E – lower by 20% » For all PAs, NTG ratios showed moderate improvement over 2010- 2012 »Range from 0.55 to 0.66 in 2013, approximately 20% improvement 26

27. OVERARCHING FINDINGS AND RECOMMENDATIONS » To more accurately estimate ex-ante savings, the PAs should: Improve documentation and reporting of project EUL, including a review of evaluation EUL conclusions/rationale, Improve quality control of determining project operating conditions, ex- ante baseline determinations, savings calculations, and eligibility rules Ensure adjustments to project savings based on post-installation inspections and M&V. »To achieve sufficient quality control, PA project documentation needs to significantly improve. SDG&E appears to be leading the other PAs in this area, given the level of improvement noted in 2013+ applications. »To reduce continued moderate free ridership, PAs should test changes to program features designed to increase program-induced savings. 27

28. KEY RECOMMENDATIONS BY TOPIC AREA » Increase focus on: Accuracy of operating conditions, Use of pre- and post-installation data and information, and Keeping project documentation and tracking claims up to date with field information »The evaluation team recommends that the PAs ensure that savings calculations are based on actual equipment-use schedules and reflect any changes to the post-installation operating parameters Operating Conditions 28

29. KEY RECOMMENDATIONS BY TOPIC AREA »Increase efforts to ensure conformance with CPUC baseline policies and make a greater effort to examine existing equipment RUL »Clearly identify project events in terms of natural replacement, replace on burnout, early replacement, new construction, and add- on equipment, and set the appropriate baseline accordingly »Disseminate information on baseline selection to ensure best practices across program staff, implementers and customers »Ensure that baseline equipment specifications are capable of meeting post-installation operating requirements, that the baseline selected is consistent with the project type, and that regressive baseline considerations are examined »The requirement that measures exceed the ISP / code baseline is a first order consideration for project eligibility Baseline Conditions 29

30. KEY RECOMMENDATIONS BY TOPIC AREA »Continue to review and improve impact methods and models through review of evaluation results, industry best practices, and the CPUC’s ex-ante review process »Ex-ante savings estimates and calculation methods should be more thoroughly reviewed and approved by PA technical staff prior to finalization of incentives and savings claims »Conduct periodic due diligence to ensure programs adhere to PA and CPUC impact estimation policies, guidelines, and best practices »Continue to work closely and collaboratively with the CPUC’s ex- ante review process Calculation Methods 30

31. KEY RECOMMENDATIONS BY TOPIC AREA »Adopt procedures to screen for and increase efficiency levels for high likelihood free ridership projects. Advise customers to move to a higher level of efficiency undertake a bundled retrofit to ensure deeper savings move to another EE project under consideration »Carefully review the list of qualifying measures; eliminate eligibility for standard practice measures »Make changes to the incentive design. Tier incentives by technology class. Consider adopting a payback floor. »Under development for 2014: modified NTGR framework to address projects involving replacement of existing equipment Early replacement/dual baseline projects Normal replacement/ROB projects Program Influence/NTG 31

32. KEY RECOMMENDATIONS BY TOPIC AREA »Improve PA program requirements, manuals, training, and quality control procedures in order to screen out ineligible projects »It is recommended that a statewide document, similar to the PPA form, be developed for use by all PAs for custom claims; this form has been provided to the PAs for their use »Increase communication and coordination efforts with entities responsible for disseminating and implementing CPUC guidance »Better ex-ante documentation is needed supporting key project parameters identified in the PPA analysis Cross-Cutting and Other Gross Impact-Related 32

33. THANK YOU www.itron.com CONSULTING & ANALYSIS GROUP 1111 Broadway Oakland, CA

34. GROSS IMPACT RESULTS Most Influential Discrepancy Factors by IOU - SCE 15

35. GROSS IMPACT RESULTS Most Influential Discrepancy Factors by IOU – SDG&E 16

36. GROSS IMPACT RESULTS Most Influential Discrepancy Factors by IOU – SCG 17