Regional Energy Networks Impact Evaluation Research Plan July 20,

REN Program Impact Evaluation (Multifamily) –Program Overviews –Evaluation Overview, Activities, Calculations REN Program Impact Evaluation (Single Family) –Program Overviews –Evaluation Overview, Methods and Samples Schedule and Next Steps Agenda 2

How Did We Get Here? 3 7/15/2016: Draft plan released 7/20/2016: Research Plan Webinar 7/29/2016: Comment Period Closes 8/22/2016: Final Research Plan Released

Program Overview: REN Multifamily Whole Building Program 4

Multifamily Whole Building Program (MF-WB) –Administered by BayREN and SoCalREN –Customized project scopes Savings based on building simulations –Technical and financial assistance Project incentives range: $3,750 - $594,000 Program Overview – MF-WB 5

6 Program ElementSoCalRENBayREN Service Territory Joint SCE/SCG territory, minus municipal service territories (e.g. LADWP) 9-County Bay Area Eligibility Requirements 3+ units; SCE & SCG service; 3+ measures; work with Participating Rater 5+ units; 9-county Bay Area; PG&E gas and/or electric; 2+ measures Rater Delivery Model Open RaterTech Assist. direct delivery Audit Requirement ASHRAE Level 2Clipboard audit CAS Testing MF HERCC ProtocolsMF HERCC protocols Energy Modeling Software EnergyProEnergyPro Lite Assessment Incentive # UnitsIncentive Free site visit and technical assistance up to $5000 value 1-49$5, $10, $20/Unit Increment Improvement Incentive Improvement$/UnitImprovement$/Unit 10%$55010%$750 15%$625 20%$800 25%$1,000 _> 30%$1,200 Verification Participating RaterTech. Assist. (AEA)

Incented ~248 properties in –>15,000 dwelling units BayREN claimed most projects (94%) and electricity savings (80%) Program Overview – MF-WB 7

Evaluation Overview: REN Multifamily Whole Building Program 8

Evaluation Overview – MF-WB 9 Evaluation Research Questions 1)What are the gross energy and demand savings (therms, kWh, kW) achieved by the BayREN and SoCalREN programs? 2)What are the net energy and demand savings achieved by the programs? 3)How can the RENs improve their ex ante savings claims so that they align with ex post values? 4)How do the REN savings and costs compare to those claimed from the IOU MF-WB programs?

Evaluation Overview – MF-WB 10 Evaluation Activities Telephone Surveys & Onsite Recruitment Onsite Visits Consumption Analysis Calibrated Simulation Models Savings Calculations

Evaluation Overview – Data Sources 11 REN provided project details CPUC claim savings database Consumption (billing) data REN EnergyPro Models Participant Survey (Property Managers/Owners) – impact assessment completes –2015 completes

Evaluation Overview – MF-WB 12

Evaluation Activities: Telephone Survey and Onsite Recruitment 13

Evaluation Activities– Telephone Survey 14 Goals: Recruit for onsite visits Supplement survey results –NFR Estimation –Baseline Assessment Sample: All participants eligible for onsite recruitment –Survey conducted only with those not contacted in effort

Evaluation Activities– Telephone Survey 15 Survey Topics: Confirmation/verification of installed measures Anticipated actions in absence of program intervention Importance of program education and incentives on the decision to install high efficiency equipment Working status and estimated age of replaced units Timing for building maintenance/upgrades Survey Instrument: Rely on instrument –Comparability, consistency, holistic results, vetted batteries and analysis

Evaluation Activities: Onsite Visits 16

Evaluation Activities– Onsite Visits 17 Goals: 1)Verify key measure & building characteristics –Measures and characteristics that contribute most to project savings –Use in simulation models 2)Collect meter numbers –Link to IOU consumption information for project Sample: All participants eligible Target 22 completed site visits

Evaluation Activities: Consumption Analysis 18

Evaluation Activities– Consumption 19 Goals: 1)Comprehensive pre-program energy use at participant property. –Use in simulation models 2)Determine if meter numbers fully capture / link to IOU consumption information Sample: Completed onsite visits where meter numbers were obtained

Evaluation Activities– Consumption 20 Process: 1)Link meter numbers to IOU consumption information 2)Assess completeness and reasonableness of data

Evaluation Activities: Calibrated Simulation Models 21

Evaluation Activities– Models 22 Goal: Ex post gross savings values –Based on administrator project models –Using EnergyPro –Calibrated to pre-program consumption –Incorporate key measure and building characteristics found onsite –Adjusted for early replacement and replace on burnout scenarios Sample: Completed site visit participants –Meter numbers were obtained –Reasonable consumption information

Evaluation Activities: Savings Calculations 23

Evaluation Review 24

Evaluation Activities– Calculations 25

Program Overview: REN Single Family Home Upgrade Program 26

Overview: Single Family –2 Regional Energy Networks (REN) and 4 Investor Owned Utilities (IOU) –Home Upgrade – deemed savings, deemed net to gross ratio (RENs) –Advanced Home Upgrade – modelled savings, estimated net to gross ratio (IOUs) 27

Overview: Prior Study Recap Prior impact evaluation –Focused Impact Evaluation (PY ) estimated gross savings only for Home Upgrade –Whole House Retrofit (PY ) estimated savings gross and net savings for Advanced Home Upgrade Outcome –Ex post saving estimates were less than ex ante but had wide ranges. –Issues with data quality and quantity limited the application of findings 28 This research will estimate gross and net savings, NTG ratio of free riders, realization rate for Home Upgrade and Advanced Home Upgrade using projects from 2013, 2014 and 2015

Methods: Energy Savings Billing analysis –Billing analysis using meter data and pooled model Comparison group consists of future program participants Variable duration for blackout period (min 30 days) Consideration of interim activities and behavioral changes Hourly data for kW Daily data for kWh Monthly data for therms –Expect population to be around 18,000 homes statewide 29 Approach is similar to last impact evaluation to facilitate comparisons. Refinements will be made from lessons learned and to be as consistent as possible with CalTRACK effort

Methods: Savings Drivers NTG and Exploratory Research –To date, the less than expected savings outcomes from HUP have not been fully explained Combine data and findings from past and current studies Conduct participant surveys –Topic examples: Contractor effect / program messaging? Geographic or demographic patterns? Proportions and types of measures installed? Primary motivations or significant barriers to adoption of one path or the other? Other actions taken by participants? 30 Primary data collection and mine existing data to uncover new insights

Single Family Schedule Data Collection –Start now (July 2016) Project data collection Prior studies Survey informed by billing data and prior studies Interval billing data collection (Sep-Oct 2016) Weather data Analysis –Over the summer (Q3 and Q ) Reporting Q Interim results can be released as available

Schedule and Next Steps 32 Milestone Schedule Draft Research Plan – Comments Due7/29/2016 Final Research Plan made available PDA/Basecamp 8/22/2016 Draft Data Collection Instruments9/15/2016 Telephone and Onsite Data Collection10/1 - 12/31/2016 Consumption Analysis1/1 - 2/28/2017 Simulation Modeling2/1 - 3/31/17 Gross Impact Analysis4/1 - 5/1/17 Net Impact Analysis4/1 - 5/1/17 Draft Report made available to public5/12/2017 Webinar - Draft Report5/17/2017 Draft Report - Comment Period Closes6/2/2017 Final Report made available7/1/2017

Contact Information 33 DNV GL Contact : Jon Vencil, (619) Apex Contact: Katie Parkinson, (303) CPUC Contact: Jeremy Battis, (415)