Multifamily Programs Focused Impact Evaluation March 10,

Multifamily Programs Impact Evaluation –Program Overviews –Evaluation Overview –Evaluation Methods and Findings –Conclusions and Recommendations 2014 Water Kits ESPI Review Agenda 2

Reporting Schedule 3 12/4/15:Draft report released 12/10/15:Webinar to present findings 12/18/15:Public comments due 1/11/16:Draft Final report released - additional dbase adjustments added and opened for comments 2/12/16:Public comments due - on dbase adjustments, only 2/29/16:Final report released 3/10/16:Webinar

Program Overview Multifamily Energy Efficiency Rebate Program 4

Multifamily Energy Efficiency Rebate Program (MFEER) –Statewide, Core Program – IOU Administered –Self-Install and DI measures –Prescriptive rebate amounts $ $1,400 –Including CoolingPool Equipment AppliancesSpace Heating LightingShell Water Heating Equipment (Large DHW) Faucet Aerators, Low Flow Showerheads (Small DHW) Program Overview - MFEER 5

Rebated >500,000 measures in SCE claimed most rebates (68%) and electricity savings (87%) SoCalGas claimed most natural gas savings (51%) Program Overview - MFEER 6 IOUMeasures Savings (Ex Ante Gross) kWhkWTherms SCE341,61528,510,3732,083— SDG&E64,5242,145, ,112 PG&E77,3432,055,0161,216662,903 SoCalGas21,3288, ,589 Totals504,81032,720,1163,4471,389,604

Similarly, IOUs also spent less than budgeted Program Overview - MFEER 7 IOUBudgetSpent% Spent % Savings achieved (% of goal) kWhkWTherms SCE $ 23,495,962 $ 13,656,15458%34%16%0% SDGE $ 3,402,589 $ 2,302,76768%52%30%10% PG&E $ 5,189,025 $ 3,847,57874%45%145%201% SoCalGas $ 2,767,910 $ 1,720,68862%70%87%56% Totals $ 34,855,486 $ 21,527,18862%35%24%79%

Measure Contributions: Electric measures: lighting dominated the ex ante electricity savings (kWh), –84% for PG&E, 83% for SCE, and 100% for SDG&E Gas Measures: Large DHW contributed most for PG&E (71%) & SoCalGas (94%). –Small DHW contributed most for SDG&E (59%) Program Overview - MFEER 8

Program Overview Multifamily Whole Building Program 9

Multifamily Whole Building Program (MF-WB) IOU and REN implemented First years of program –PG&E in pilot stage Offers technical and financial assistance –Target larger retrofit projects Savings based on building simulations –> flexibility with measures –Must achieve >10% of savings to qualify Program Overview – MF-WB 10

Program Overview – MF-WB 11 IOUProjects Tenant Units Savings (ex ante gross) kWhkWTherms SDGE410826, ,002 PG&E7‡568, ,067 BayREN955,6931,590, ,807 SoCalREN , ,650 Totals1086,1852,570, ,526 Completed 108 projects statewide, representing 6,185 tenant units

Program Overview – MF-WB 12 Most spent less than planned Also fell short of goals, achieved –28% of kWh, –11% of kW, and –35% of Therm goals. –Exception: BayRen exceeded Therm and kWh goals IOUBudgetSpent% Spent % Savings achieved (% of goal) kWhkWTherms SCE $ 2,000,000 $ 300,67615%0% SDGE $ 2,501,496 $ 1,446,35158%13%9% PG&E $ 5,630,116 $ 1,598,39428%NA SoCalGas $ 1,000,000 $ 43,8444%0% BayREN $ 7,293,750 $ 13,202,593181%117%18%111% SoCalREN $ 9,543,801 $ 2,629,97828%6% 5% Totals $ 27,969,163 $ 19,221,83769%28%11%35%

Program Overview 13 MFEER program savings considerably larger than MF-WB

Program Overview 14 MF-WB program also more expensive MF-WB: $651 spent per MMBTU saved MFEER: $86 spent per MMBTU saved PA MF-WBMFEER Spending$/MMBTUSpending$/MMBTU SCE $300,676 NA $13,656,154 $140 SDGE $1,446,351 $2,947 $2,302,767 $258 PG&E $1,598,394 $416 $3,847,578 $52 SoCalGas $43,844 NA $1,720,688 $24 BayREN $13,202,593 $589 NA SoCalREN $2,629,978 $946 NA Totals $19,221,837 $651 $21,527,188 $86

Evaluation Overview 15

Evaluation Overview 16 Evaluation Activities Database Assessment Engineering Desk Review Consumption Analysis Baseline Assessment Freeridership (FR) Estimate Task Name Target Programs Program Administrators MF-WBMFEERIOUsRENs Database Review Engineering Review Consumption Analysis Baseline Assessment FR Estimate

Evaluation Overview – Data Collection 17 IOU & REN provided project level backup data CPUC claim savings database Consumption (billing) data Participant Survey UtilityProgram Complete Surveys Total252 PG&EMFEER43 SCEMFEER152 SDGEMFEER26 SoCalGasMFEER31 SDGEMF-WB0

Evaluation Methods and Findings: Database Assessment 18

Methods and Findings – Database 19 Database Assessment Objectives (MF-WB only) Assess completeness of databases Reasonable ranges Tracking necessary fields RENs and IOUs Lifecycle savings and EULs Specific Fields Assessed Participant contact information Measures installed (quantity, location, efficiency) Preexisting conditions Types of and fuels for property hot water, cooling, and space heating systems Utility account numbers for each property and unit

Methods and Findings – Database 20 The RENs and IOUs databases contain varying levels of completeness: ●: data provided were completely populated; ◑ : some of the data were populated; ◌ : most or all of the data were missing or inaccessible.

Evaluation Activity Details – Database 21

Methods and Findings – Database 22 MF-WB: Difficulty identifying MF projects within whole building programs –CPUC claimed database –PIPs (budgets/goals) MFEER Not all IOUs had clear way of grouping measures into projects –Addresses / account id are not direct link Inaccurate contact info in CPUC claim data –Had to re-request contacts to survey

Evaluation Methods and Findings Engineering Desk Review 23

Methods and Findings– Engineering Review 24 Objective (MF-WB only) Compare ex ante savings claims (simulation models), to savings from engineering algorithms. Method Calculate measure level savings –Inputs from MF-WB backup data –Savings from DEER or TRM algorithms Findings Inconclusive –Only 4 projects with backup data One did not have quantity of windows Two installed non-standard measure (floor insulation)

Evaluation Methods and Findings Consumption Analysis 25

Methods and Findings– Consumption Analysis 26 Objective (MF-WB only) Ensure claimed savings were reasonable (10- 20%+) relative to project annual billing (gas and/or electric). Method Three primary steps: –Linking billing data to project data –Validating comprehensiveness of billing and savings data –Comparing the claimed savings to actual pre- program billing data 12 months of pre-installation billing All 4 SDG&E MF-WB projects analyzed

Methods and Findings– Consumption Analysis 27 Findings Electricity: –Two projects with high savings One >30%, one >90% –Two projects <10%, but gas OK Gas: –One project >20% –Others 10%-20%

Evaluation Methods and Findings Baseline Assessment 28

Methods and Findings– Baseline Assessment 29 Objective (MFEER) Assess consistency and accuracy of IOU baseline values (ER / ROB) –Review and compare tracking database across IOUs –Survey participants to identify baseline for installed equipment Compare against overall tracking baselines Compare against same participants tracking assignments

Methods and Findings– Baseline Assessment 30 Findings – Tracking Review (MFEER) Current designations vary Measure Category PG&ESCESoCalGasSDG&E ShellROBER NA Lighting ROB ER (84%); ROB (16%) N/A ER (34%); ROB (66%) Small DHWROBERN/AROB Large DHWROB ER (89%); ROB (11%) ER All OthersROB ER (86%); ROB (14%) ROBNA

Methods and Findings– Baseline Assessment 31 Survey Method: Participant phone survey responses to: –Equipment working status –Equipment age –Equipment expected remaining life –Retrofit was part of regularly scheduled/government-mandated upgrade

Methods and Findings– Baseline Assessment 32 Findings (MFEER only) Shell and small DHW measures – survey vs overall tracking Large DHW and other measures – survey vs overall tracking Measure Category Measures % Survey ER % Tracking ER Shell (n = 26) Windows, Insulation (Attic, Wall, Floor, Crawlspace) 66%31% Small DHW (n = 30) Faucet Aerator, Low-Flow Showerhead 90%15% Measure CategoryMeasures % Survey ER % Tracking ER Large DHW (n = 27) Storage/Tankless/Boiler Water Heaters, Hot Water Boiler Control 21%48% All Others (n = 29) Appliances, Space Heating, Pool 72%63%

Evaluation Methods and Findings Freeridership Estimate 33

Methods and Findings– Freeridership Estimate 34 Objective (MFEER) Assess influence of program on decision to install measures Capture complex decision-making process associated with MF programs –Explored company policy –Based on the CPUC ED methodology

Methods and Findings– Freeridership Estimate 35 Method: Participant phone survey responses to three equally weighted components (program attribution index, PAI): –PAI-1: influence of the program and non- program related factors –PAI-2: perceived importance of the program relative to non-program factors –PAI–3: likelihood of actions the customer might have taken if program had not been available (counterfactual)

Methods and Findings– Freeridership Estimate 36 Method: Only freeridership (FR), did not include spillover Option to differ by measure Consistency checks FR estimated individually for each respondent, then MMBtu savings weighted for composite value

Methods and Findings– Freeridership Estimate 37 Findings (MFEER): Overall NTFR: 51.6% –PAI-1 (Influence): 47.1% More respondents rated non-program > program influences –PAI-2 (Relative Importance): 56.1% 29% indicated learned of program after deciding to install equipment. –PAI-3 (Install Same Equip): 51.6% High likelihood of installing same efficiency equipment in absence of program

Conclusions and Recommendations 38

Conclusions and Recommendations 39

Conclusions and Recommendations 40 Database Assessment (MFEER) Projects lacked ID to group records –Assign SiteID for all measures installed at site project Inaccurate contact information –Need for accurate transfer of information from application to claim database

Conclusions and Recommendations 41 Engineering and Consumption Analysis (MF-WB) Inconclusive engineering review –Simulation model or billing analysis more effective approach High level of uncertainty linking billing data to tracking –Meter numbers should be tracked to link billing Electric savings ratios from consumption analysis were varied, gas were consistent –Providing aggregate project-level billing to PAs will help calibrate models

Conclusions and Recommendations 42 Baseline and Freeridership (MFEER) Inconsistent baselines & differ from study findings –Intake survey to determine baseline FR estimates represent measure mix –Continue to research and update FR levels Inconsistently assigned NTG in tracking –Correctly assign and document DEER based NTG IDs

Savings Adjustments (MF-WB) Gross Savings: –PG&E savings removed due to insufficient measure level information Net Savings: –SDGE received NTFR of 58% as determined in the REN evaluation Conclusions and Recommendations 43 IOU Savings (ex ante gross)Savings (ex post gross) NTFR % Savings (ex post net) kWhkWThermskWhkWThermskWhkWTherms SDG&E26, ,00226, ,00258%15,40592,321 PG&E568, , %--- Totals594, ,06926, ,00258%15,40592,321

Savings Adjustments (MFEER) Gross Savings: –No gross savings adjustments Net Savings: –Based on decision maker survey findings at measure level Conclusions and Recommendations 44 IOU kWh (net)kW (net)Therms (net) ex anteex postNet RRex anteex postNet RRex anteex postNet RR SCE17,752,79315,224,53986%1,1631,13397%-45,995-42,44992% SDGE1,245,2241,261,840101% %8,2557,54191% PG&E1,171,5961,165,19499% %405,815338,74383% SoCalGas4,8073,32169%3267%380,694270,02471% Totals20,174,42017,654,89488%2,1601,80083%794,764616,30878%

2014 Water Kits ESPI Review 45

Measures: Faucet Aerators and Low Flow Showerheads Programs: Description of ESPI Measure 46 Program IDProgram Name Measure Name Faucet Aerators Low Flow Showerheads SCG 3761Multifamily Home Tune-up SCG 3764Livingwise (School Kits) SCG 3763Multifamily Direct Therm Savings SDGE 3207MFEER SDGE 3279 Comprehensive Manufactured and Mobile Home Program SDGE 3226Commercial Direct Installations SDGE 3223Commercial Deemed Incentives

February 22, 2013 Energy Division Work paper Disposition for Water Fixtures Uses SCG and SDG&E field research (flow rates), DEER values (hot water usage, climate zone multipliers), READI (ISR) Allowed stratification by –Climate zone –Delivery Mechanism –Housing Type –Rated GPM MFEER Decision Maker Survey Applied to MFEER NTFR only Sources 47 UtilityProgram MFEER Completed Surveys Total57 SDGEMFEER26 SoCalGasMFEER31

For each program measure, evaluation team applied UES by installed climate zone & GPM ISR by delivery mechanism & housing type NTG by delivery mechanism & housing type Methodology 48

Differences due to selection and application of Disposition –Incorrect base savings assumptions –Incorrect application of climate zone adjustment factors –NTG for DI, rather than self-install Other findings –Commercial vs. residential applications –Recording kitchen vs. bathroom aerators Results 49

Program ID Measure Quantity First Year Gross Therm SavingsFirst Year Net Therm Savings Ex AnteEx PostRREx AnteEx PostRR Faucet Aerators SCG ,513 54,169 51,63295% 35,210 33,56195% SCG ,673 87,088 83,74996% 56,607 54,43796% SCG , , ,34796% 103,404 92,04189% SDGE ,141 1,561 1,769113% 1, % SDGE , % % Low Flow Showerheads SCG ,370 95, ,033107% 66,876 71,423107% SCG , , ,172100% 105, ,121100% SCG , , ,73995% 184, ,90675% SDGE ,121 13,111 9,36971% 7,211 4,38461% SDGE ,223 50,921 36,38871% 30,553 20,01466% SDGE % % SDGE ,757 3,53194% 3,193 2,47177% Results 50

ESPI Reporting Schedule and Next Steps 51 2/29/16:Draft report released 3/10/16:Webinar 3/15/16:Public comments due 4/1/16:Final report released

Contact Information 52 Apex Contact Information: Scott Dimetrosky, (303) CPUC Contact Information: Tory Francisco, (415)