1. DHP for Houses with Electric FAF Research Plan: Revisions Adam Hadley, Ben Hannas, Bob Davis, My Ton R&E Subcommittee February 25, 2015

2. Agenda Overview of comments Review of revisions made Select detailed comments & responses Other comments & responses Discussions Wrap up 2 - Outline

3. Summary of Comments 1.Reviewers generally agreed with overall study approach. 2.Comments received were in regard to data collection and analysis details. 3.Comment received from reviewers on the following areas: – Proposed Approach – Field Research – Phone Survey – Analysis Plan 3

4. Revision of Research Plan in Response to Comments The Research Plan was restructured, and the following sections of the were revised: Overview of Proposed Approach: added details regarding analysis of data from 2012 Study, data collection and analysis methods Field Research: added details to address sample design and data collection Phone Survey: added details regarding data collection & use Analysis Plan: – added details regarding development of DHP/FAF Curve for SEEM model – added details regarding determination of supplemental fuel adjustment, determination of occupancy adjustment, and determination of control strategy 4

5. Revision of Research Plan in Response to Comments Added details in Overview of Proposed Approach regarding analysis of data from 2012 Study: Construct a DHP/FAF curve for each home Review the curves for relative importance of key variables (UA, Uo, floor area, thermostat behavior, etc.). Re-analyze the 2012 Study data to give an initial estimate of the SEEM calibration factor for DHP/FAF homes based on SEEM runs and the metering results. Add modeled SEEM runs with the DHP zonal setting and duct effects in a post-processing step. This analysis is a proof-of-concept that the methods from the DHP zonal analysis will also work for the DHP/FAF analysis, possibly with some tweaks (e.g., control strategy weights). 5

6. Revision of Research Plan in Response to Comments Added other details in Overview of Proposed Approach regarding data collection and analysis methods: 1.Use the phone survey to collect key characteristics for a larger set of homes analyzed in the billing analysis (thermostat behavior, supplemental fuel use, occupancy, size of home, etc.) 2.Conduct a billing analysis to obtain a direct savings estimate for the measure and provide data for calibration of SEEM output based on supplemental fuels and control types. 3.Develop separate DHP/FAF UES workbooks for SF and MH, after collecting and analyzing the field data, phone survey data, billing data, and after updating SEEM. 6

7. Revision of Research Plan in Response to Comments Added details in Field Research to address: 1.Sample design: Added a table to clearly show how the sites are used in the various analysis types of the proposed research. 2.Data collection: Added specific details about how cooling will be dealt with. Added details in Phone Survey regarding data collection and use: Gather data about thermostat behavior, supplemental fuel use, occupancy patterns (vacations, partial year residence, etc.), and size of home. Integrate with program information (primarily make/model of DHP) to develop a data set comparable to the DHP pilot data set used in the previous evaluations. The data set will be helpful in the billing analysis summaries. 7

8. Revision of Research Plan in Response to Comments Added details in Analysis Plan regarding Development of DHP FAF Curve for SEEM Model:  Use results from Field Research to calculate the ratio of DHP and FAF output by a number of possible explanatory variables: outdoor temperature, control strategy, UA, Uo, floor area.  Choose the best of these explanatory variables to generate a model that will be inserted into SEEM.  Estimates for the ratio of DHP and FAF for each operating strategy come from analyzing total kWh usage, run-time of each system, and DHP capacity curves from the NEEA Studies. 8

9. Revision of Research Plan in Response to Comments Added details in Analysis Plan regarding Determination of Supplemental Fuel Adjustment  In the metered dataset, homes were specifically chosen to have little or no wood heat so that the metered DHP FAF population is different from the total DHP FAF population with regard to supplemental fuel use and an adjustment is needed.  The supplemental fuel adjustment is calculated using both the survey and billing data. Survey data provide whether a home uses supplemental fuels and a rough estimate of how much supplemental fuel. In the dataset, these are then flagged as a Boolean yes/no to whether a home has supplemental fuel.  Billing data are analyzed based on cohorts developed in the survey supplemental fuel analysis to develop a ratio of electric heating use to total heating use (electric + supplemental) on average for the population (possibly by climate zone). 9

10. Revision of Research Plan in Response to Comments Added details in Analysis Plan regarding Determination of Occupancy Adjustment  The survey data will be used to calculate a ratio of occupancy, and the fraction will be multiplied by the DHP/FAF savings, which will reduce the savings. Added details in Analysis Plan regarding Determination of Control Strategy  Responses from the survey will be used to categorize homes into control strategy bins.  The metered homes will also receive the survey, and the responses of the metered group on the survey will be compared to the observations of field staff and analysis of metered heating energy use.  The correlations of survey responses to metered energy use and field staff observations may show a bias in the survey responses. This bias will then be adjusted for in the total survey population. The adjusted population control strategies will be summarized and may be used either inside of SEEM or in the UES analysis. 10

11. Select Detailed Comments & Responses 11

12. Comments & Response Comments regarding Primary Field Research: 1.Need to group metered homes into control strategies based on the same data that is used to group phone survey homes. 2.Use other information available from the metered homes to classify by strategy type. The comparison of these two methods for classifying the metered group into control strategies should be very informative. Would give a sense of how wrong the classification of the billing analysis population is. Response: Added language in analysis plan for Primary Field Research to describe calibration process in more detail 12

13. Comments & Response Comment Regarding: UES Development, SEEM Analysis 1.It is not clear that current Phase I adjustments would be reasonable for DHPs in an FAF home. Response: Incorporated a new SEEM model capable of FAF/DHP simulation and separate calibration of FAF/DHP. 13

14. Comments & Response Comments Regarding: UES Development, SEEM Analysis 1.The ratios of the SEEM runs for the combinations of the two equipment types are are critical. A description of how they are computed is needed. 2.Shouldn’t the savings for each control type be computed from SEEM models that are calibrated to the relevant portions of the metered sample? 3.The results from the metered sample provide the fraction of DHP use, and could later be re-weighted to reflect the distribution of the control types found in the full population. Response: Added details on supplemental fuel and occupancy adjustments, and how they will be used in the SEEM simulation runs 14

15. Comments & Response Comment Regarding: Energy Savings Calculations 1.The field work from 2012 plus this new round give us 60 homes where the heating use is directly metered. This seems like a good opportunity to see if the Phase 1 and 2 calibrations yield heating use that is close to directly metered heating use. Especially as the DHP fractions are coming primarily in this research from the metered sample. Response: Added details on SEEM calibration adjustments, and for comparison of simulation results with billing analysis. 15

16. Comments & Response Comments Regarding: SEEM Calibration 1.The distribution of DHP/FAF use within each control strategy should be focus for calibrating the SEEM and post-SEEM analyses. They are needed to estimate baseline and efficient case energy use. 2.Subsequent calibration work should be done to account for the characteristics of billing population, but only after the heating use is correct for models that match the characteristics of the 60 metered sites. Response: Added details on SEEM calibration adjustments, and for comparison of simulation results with billing analysis. 16

17. Other Comments 17

18. Other Comments 18

19. Other Comments 19

20. Other Comments 20

21. Other Comments 21

22. Discussion of Current Plan Are the revisions/modifications sufficient regarding: Field Research/data collection SEEM calibration factors Energy Savings Calculations Analysis details Other comments 22

23. Discussion of Current Plan Are there other revisions/modifications needed? 23

24. Discussion of Current Plan Is the R&E Subcommittee comfortable with how the contract analysts have addressed the comments received? 24

25. Discussion of Current Plan Can the R&E subcommittee recommend the revised plan for RTF adoption? 25

26. Thank you! 26

27. Research Plan Review: Data Collection 1.Expand on 2012 Study with field metering:  Develop relationship of DHP/FAF heat output fraction to other factors & embed this relationship within SEEM model  Understand interactions of DHP/FAF operations in colder climates  Provide dataset to calibrate DHP/FAF SEEM model 2.Conduct Billing Analysis:  Estimate savings at the meter for the population of installations  Estimate impact of non-electric fuels  Calibrate SEEM by control type, or other factors, to population 3.Conduct Phone Survey  Collect data on wood/supplemental fuel use, control strategy, house size, etc. 27 – Research Plan

28. Research Plan Review: Analysis 1.Use Field Research data to develop and calibrate SEEM simulation (for houses with a solid heat signature) 2.Billing Analysis: Pre/post VBDD for each house  Use Billing Analysis and Phone Survey results to estimate impact of supplemental fuels (similar to method used in zonal study: energy consumption pre and post of those with supplemental fuels vs those without)  Also used as a QC check on resulting UES values after many manipulations through SEEM 3.Use Phone Survey results to establish a population- weighted control strategy mix (and same for other critical variables) 4.Using newly calibrated version of SEEM, estimate UES. 28 – Research Plan