1. 2013-2014 Multifamily Programs Focused Impact Evaluation March 10, 2016 1

2. Multifamily Programs Impact Evaluation –Program Overviews –Evaluation Overview –Evaluation Methods and Findings –Conclusions and Recommendations 2014 Water Kits ESPI Review Agenda 2

3. Reporting Schedule 3 12/4/15:Draft report released 12/10/15:Webinar to present findings 12/18/15:Public comments due 1/11/16:Draft Final report released - additional dbase adjustments added and opened for comments 2/12/16:Public comments due - on dbase adjustments, only 2/29/16:Final report released 3/10/16:Webinar

4. Program Overview Multifamily Energy Efficiency Rebate Program 4

5. Multifamily Energy Efficiency Rebate Program (MFEER) –Statewide, Core Program – IOU Administered –Self-Install and DI measures –Prescriptive rebate amounts $1.50 - $1,400 –Including CoolingPool Equipment AppliancesSpace Heating LightingShell Water Heating Equipment (Large DHW) Faucet Aerators, Low Flow Showerheads (Small DHW) Program Overview - MFEER 5

6. Rebated >500,000 measures in 2013-2014 SCE claimed most rebates (68%) and electricity savings (87%) SoCalGas claimed most natural gas savings (51%) Program Overview - MFEER 6 IOUMeasures Savings (Ex Ante Gross) kWhkWTherms SCE341,61528,510,3732,083— SDG&E64,5242,145,98714216,112 PG&E77,3432,055,0161,216662,903 SoCalGas21,3288,7406710,589 Totals504,81032,720,1163,4471,389,604

7. Similarly, IOUs also spent less than budgeted Program Overview - MFEER 7 IOUBudgetSpent% Spent % Savings achieved (% of goal) kWhkWTherms SCE $ 23,495,962 $ 13,656,15458%34%16%0% SDGE $ 3,402,589 $ 2,302,76768%52%30%10% PG&E $ 5,189,025 $ 3,847,57874%45%145%201% SoCalGas $ 2,767,910 $ 1,720,68862%70%87%56% Totals $ 34,855,486 $ 21,527,18862%35%24%79%

8. Measure Contributions: Electric measures: lighting dominated the ex ante electricity savings (kWh), –84% for PG&E, 83% for SCE, and 100% for SDG&E Gas Measures: Large DHW contributed most for PG&E (71%) & SoCalGas (94%). –Small DHW contributed most for SDG&E (59%) Program Overview - MFEER 8

9. Program Overview Multifamily Whole Building Program 9

10. Multifamily Whole Building Program (MF-WB) IOU and REN implemented First years of program –PG&E in pilot stage Offers technical and financial assistance –Target larger retrofit projects Savings based on building simulations –> flexibility with measures –Must achieve >10% of savings to qualify Program Overview – MF-WB 10

11. Program Overview – MF-WB 11 IOUProjects Tenant Units Savings (ex ante gross) kWhkWTherms SDGE410826,56015.54,002 PG&E7‡568,382136.119,067 BayREN955,6931,590,268198.1169,807 SoCalREN2384385,25579.814,650 Totals1086,1852,570,465429.6207,526 Completed 108 projects statewide, representing 6,185 tenant units

12. Program Overview – MF-WB 12 Most spent less than planned Also fell short of goals, achieved –28% of kWh, –11% of kW, and –35% of Therm goals. –Exception: BayRen exceeded Therm and kWh goals IOUBudgetSpent% Spent % Savings achieved (% of goal) kWhkWTherms SCE $ 2,000,000 $ 300,67615%0% SDGE $ 2,501,496 $ 1,446,35158%13%9% PG&E $ 5,630,116 $ 1,598,39428%NA SoCalGas $ 1,000,000 $ 43,8444%0% BayREN $ 7,293,750 $ 13,202,593181%117%18%111% SoCalREN $ 9,543,801 $ 2,629,97828%6% 5% Totals $ 27,969,163 $ 19,221,83769%28%11%35%

13. Program Overview 13 MFEER program savings considerably larger than MF-WB

14. Program Overview 14 MF-WB program also more expensive MF-WB: $651 spent per MMBTU saved MFEER: $86 spent per MMBTU saved PA MF-WBMFEER Spending$/MMBTUSpending$/MMBTU SCE $300,676 NA $13,656,154 $140 SDGE $1,446,351 $2,947 $2,302,767 $258 PG&E $1,598,394 $416 $3,847,578 $52 SoCalGas $43,844 NA $1,720,688 $24 BayREN $13,202,593 $589 NA SoCalREN $2,629,978 $946 NA Totals $19,221,837 $651 $21,527,188 $86

15. Evaluation Overview 15

16. Evaluation Overview 16 Evaluation Activities Database Assessment Engineering Desk Review Consumption Analysis Baseline Assessment Freeridership (FR) Estimate Task Name Target Programs Program Administrators MF-WBMFEERIOUsRENs Database Review Engineering Review Consumption Analysis Baseline Assessment FR Estimate

17. Evaluation Overview – Data Collection 17 IOU & REN provided project level backup data CPUC claim savings database Consumption (billing) data Participant Survey UtilityProgram Complete Surveys Total252 PG&EMFEER43 SCEMFEER152 SDGEMFEER26 SoCalGasMFEER31 SDGEMF-WB0

18. Evaluation Methods and Findings: Database Assessment 18

19. Methods and Findings – Database 19 Database Assessment Objectives (MF-WB only) Assess completeness of databases Reasonable ranges Tracking necessary fields RENs and IOUs Lifecycle savings and EULs Specific Fields Assessed Participant contact information Measures installed (quantity, location, efficiency) Preexisting conditions Types of and fuels for property hot water, cooling, and space heating systems Utility account numbers for each property and unit

20. Methods and Findings – Database 20 The RENs and IOUs databases contain varying levels of completeness: ●: data provided were completely populated; ◑ : some of the data were populated; ◌ : most or all of the data were missing or inaccessible.

21. Evaluation Activity Details – Database 21

22. Methods and Findings – Database 22 MF-WB: Difficulty identifying MF projects within whole building programs –CPUC claimed database –PIPs (budgets/goals) MFEER Not all IOUs had clear way of grouping measures into projects –Addresses / account id are not direct link Inaccurate contact info in CPUC claim data –Had to re-request contacts to survey

23. Evaluation Methods and Findings Engineering Desk Review 23

24. Methods and Findings– Engineering Review 24 Objective (MF-WB only) Compare ex ante savings claims (simulation models), to savings from engineering algorithms. Method Calculate measure level savings –Inputs from MF-WB backup data –Savings from DEER or TRM algorithms Findings Inconclusive –Only 4 projects with backup data One did not have quantity of windows Two installed non-standard measure (floor insulation)

25. Evaluation Methods and Findings Consumption Analysis 25

26. Methods and Findings– Consumption Analysis 26 Objective (MF-WB only) Ensure claimed savings were reasonable (10- 20%+) relative to project annual billing (gas and/or electric). Method Three primary steps: –Linking billing data to project data –Validating comprehensiveness of billing and savings data –Comparing the claimed savings to actual pre- program billing data 12 months of pre-installation billing All 4 SDG&E MF-WB projects analyzed

27. Methods and Findings– Consumption Analysis 27 Findings Electricity: –Two projects with high savings One >30%, one >90% –Two projects <10%, but gas OK Gas: –One project >20% –Others 10%-20%

28. Evaluation Methods and Findings Baseline Assessment 28

29. Methods and Findings– Baseline Assessment 29 Objective (MFEER) Assess consistency and accuracy of IOU baseline values (ER / ROB) –Review and compare tracking database across IOUs –Survey participants to identify baseline for installed equipment Compare against overall tracking baselines Compare against same participants tracking assignments

30. Methods and Findings– Baseline Assessment 30 Findings – Tracking Review (MFEER) Current designations vary Measure Category PG&ESCESoCalGasSDG&E ShellROBER NA Lighting ROB ER (84%); ROB (16%) N/A ER (34%); ROB (66%) Small DHWROBERN/AROB Large DHWROB ER (89%); ROB (11%) ER All OthersROB ER (86%); ROB (14%) ROBNA

31. Methods and Findings– Baseline Assessment 31 Survey Method: Participant phone survey responses to: –Equipment working status –Equipment age –Equipment expected remaining life –Retrofit was part of regularly scheduled/government-mandated upgrade

32. Methods and Findings– Baseline Assessment 32 Findings (MFEER only) Shell and small DHW measures – survey vs overall tracking Large DHW and other measures – survey vs overall tracking Measure Category Measures % Survey ER % Tracking ER Shell (n = 26) Windows, Insulation (Attic, Wall, Floor, Crawlspace) 66%31% Small DHW (n = 30) Faucet Aerator, Low-Flow Showerhead 90%15% Measure CategoryMeasures % Survey ER % Tracking ER Large DHW (n = 27) Storage/Tankless/Boiler Water Heaters, Hot Water Boiler Control 21%48% All Others (n = 29) Appliances, Space Heating, Pool 72%63%

33. Evaluation Methods and Findings Freeridership Estimate 33

34. Methods and Findings– Freeridership Estimate 34 Objective (MFEER) Assess influence of program on decision to install measures Capture complex decision-making process associated with MF programs –Explored company policy –Based on the CPUC ED methodology

35. Methods and Findings– Freeridership Estimate 35 Method: Participant phone survey responses to three equally weighted components (program attribution index, PAI): –PAI-1: influence of the program and non- program related factors –PAI-2: perceived importance of the program relative to non-program factors –PAI–3: likelihood of actions the customer might have taken if program had not been available (counterfactual)

36. Methods and Findings– Freeridership Estimate 36 Method: Only freeridership (FR), did not include spillover Option to differ by measure Consistency checks FR estimated individually for each respondent, then MMBtu savings weighted for composite value

37. Methods and Findings– Freeridership Estimate 37 Findings (MFEER): Overall NTFR: 51.6% –PAI-1 (Influence): 47.1% More respondents rated non-program > program influences –PAI-2 (Relative Importance): 56.1% 29% indicated learned of program after deciding to install equipment. –PAI-3 (Install Same Equip): 51.6% High likelihood of installing same efficiency equipment in absence of program

38. Conclusions and Recommendations 38

39. Conclusions and Recommendations 39

40. Conclusions and Recommendations 40 Database Assessment (MFEER) Projects lacked ID to group records –Assign SiteID for all measures installed at site project Inaccurate contact information –Need for accurate transfer of information from application to claim database

41. Conclusions and Recommendations 41 Engineering and Consumption Analysis (MF-WB) Inconclusive engineering review –Simulation model or billing analysis more effective approach High level of uncertainty linking billing data to tracking –Meter numbers should be tracked to link billing Electric savings ratios from consumption analysis were varied, gas were consistent –Providing aggregate project-level billing to PAs will help calibrate models

42. Conclusions and Recommendations 42 Baseline and Freeridership (MFEER) Inconsistent baselines & differ from study findings –Intake survey to determine baseline FR estimates represent 2013-2014 measure mix –Continue to research and update FR levels Inconsistently assigned NTG in tracking –Correctly assign and document DEER based NTG IDs

43. Savings Adjustments (MF-WB) Gross Savings: –PG&E savings removed due to insufficient measure level information Net Savings: –SDGE received NTFR of 58% as determined in the 2013-2014 REN evaluation Conclusions and Recommendations 43 IOU Savings (ex ante gross)Savings (ex post gross) NTFR % Savings (ex post net) kWhkWThermskWhkWThermskWhkWTherms SDG&E26,56015.54,00226,56015.54,00258%15,40592,321 PG&E568,382136.119,067---58%--- Totals594,942151.623,06926,56015.54,00258%15,40592,321

44. Savings Adjustments (MFEER) Gross Savings: –No gross savings adjustments Net Savings: –Based on decision maker survey findings at measure level Conclusions and Recommendations 44 IOU kWh (net)kW (net)Therms (net) ex anteex postNet RRex anteex postNet RRex anteex postNet RR SCE17,752,79315,224,53986%1,1631,13397%-45,995-42,44992% SDGE1,245,2241,261,840101%8586101%8,2557,54191% PG&E1,171,5961,165,19499%90957964%405,815338,74383% SoCalGas4,8073,32169%3267%380,694270,02471% Totals20,174,42017,654,89488%2,1601,80083%794,764616,30878%

45. 2014 Water Kits ESPI Review 45

46. Measures: Faucet Aerators and Low Flow Showerheads Programs: Description of ESPI Measure 46 Program IDProgram Name Measure Name Faucet Aerators Low Flow Showerheads SCG 3761Multifamily Home Tune-up SCG 3764Livingwise (School Kits) SCG 3763Multifamily Direct Therm Savings SDGE 3207MFEER SDGE 3279 Comprehensive Manufactured and Mobile Home Program SDGE 3226Commercial Direct Installations SDGE 3223Commercial Deemed Incentives

47. February 22, 2013 Energy Division Work paper Disposition for Water Fixtures Uses SCG and SDG&E field research (flow rates), DEER values (hot water usage, climate zone multipliers), READI (ISR) Allowed stratification by –Climate zone –Delivery Mechanism –Housing Type –Rated GPM MFEER Decision Maker Survey Applied to MFEER NTFR only Sources 47 UtilityProgram MFEER Completed Surveys Total57 SDGEMFEER26 SoCalGasMFEER31

48. For each program measure, evaluation team applied UES by installed climate zone & GPM ISR by delivery mechanism & housing type NTG by delivery mechanism & housing type Methodology 48

49. Differences due to selection and application of Disposition –Incorrect base savings assumptions –Incorrect application of climate zone adjustment factors –NTG for DI, rather than self-install Other findings –Commercial vs. residential applications –Recording kitchen vs. bathroom aerators Results 49

50. Program ID Measure Quantity First Year Gross Therm SavingsFirst Year Net Therm Savings Ex AnteEx PostRREx AnteEx PostRR Faucet Aerators SCG 3761 35,513 54,169 51,63295% 35,210 33,56195% SCG 3763 50,673 87,088 83,74996% 56,607 54,43796% SCG 3764 71,812 174,730 167,34796% 103,404 92,04189% SDGE 3207 3,141 1,561 1,769113% 1,009 82882% SDGE 3279 1,865 854 66978% 726 39554% Low Flow Showerheads SCG 3761 17,370 95,537 102,033107% 66,876 71,423107% SCG 3763 25,021 150,643 150,172100% 105,450 105,121100% SCG 3764 47,277 263,605 250,73995% 184,524 137,90675% SDGE 3207 2,121 13,111 9,36971% 7,211 4,38461% SDGE 3223 8,223 50,921 36,38871% 30,553 20,01466% SDGE 3226 105 647 46372% 388 32483% SDGE 3279 811 3,757 3,53194% 3,193 2,47177% Results 50

51. ESPI Reporting Schedule and Next Steps 51 2/29/16:Draft report released 3/10/16:Webinar 3/15/16:Public comments due 4/1/16:Final report released

52. Contact Information 52 Apex Contact Information: Scott Dimetrosky, scottd@apexanalyticsllc.com scottd@apexanalyticsllc.com (303)590-9888 CPUC Contact Information: Tory Francisco, tnf@cpuc.ca.gov (415) 703-2743