1. 1 NORTHWEST ENERGY EFFICIENCY ALLIANCE DHP UES Key Issues & Decisions Ecotope, Inc. May 21, 2013

2. 2 Agenda Introduction  Presentation Objectives  Measure Description  Research Structure  Summary of Savings  Summary of Key Issues Issues & Strategies Q&A Motions

3. 3 Introduction

4. 4 Presentation Objectives  Review research and findings for DHP pilot evaluation (2008-2013)  Develop a common foundation of understanding for assessing DHP UES  DHP performance  Technical vs. behavioral and/or program design considerations  Key issues for DHP UES  Make several fundamental decisions in advance of full UES workbook presentation

5. 5 Measure Description Measure Classification and Properties Market SectorResidential Market SegmentSingle-family Measure CategorySeeking to convert provisional to proven. Measure DescriptionInstall a ductless heat pump in the main living area of an existing zonal-electrically heated house.

6. 6 DHP Research Structure

7. 7 Saving Distribution, 3,887 kWh/yr

8. 8 Wide Range of DHP Savings  Metered sample  Careful screening to ensure all-electric heat  Measured heat contribution from the output of the DHP  Analyzed pre-installation bills to establish base heating use  Billing Analysis  Screened based on poor VBDD fit  Screened based on intake questionnaire  Determined supplemental fuel savings impact  Determined takeback from improved heating signature  Lower R 2 for pre-installation indicates an electric heat “takeback”,  R 2 measure fit, improves with more electric heat and with consistent thermostat settings  When pre-installation is lower screen and supplemental fuels flag is set to zero, the screen is set assuming that this would indicate electric heat takeback  Substantial agreement with results of metered sample

9. 9 Savings Vary by Screening Approach Savings by Participant GroupSavings (kWh/yr) Screened bills: R2>.45, Space Heat>0MeanEBn All2081813390 No Supplemental Fuels Used2719902295 No Takeback (R 2 )29761011705 Metered Results: Measured Savings from Metered Sub-Sample Total Savings (COP)388737565 Billing Analysis304925193

10. 10 Summary of Issues Issue 1: Which savings estimation approach: Calibrated Engineering (SEEM) or Statistical/Metastatistical (Billing & Metering)? Issue 2: How to determine electric kWh savings and supplemental fuel benefit? Issue 3. How/whether to include “comfort” in the measure savings and/or TRC calculation? Issue 4. Should we combine the savings, or disaggregate by climate zone?

11. 11 Issue 1: Which savings estimation approach: Calibrated Engineering (SEEM) or Statistical/Metastatistical (Billing & Metering)?

12. 12 UES Estimation Procedures  Statistical  DHP pilot population only represents the pilot project participants and is not a representative sample of the region  Pilot evaluation uses a large billing analysis with an intake questionnaire to provide participant information  The billing analysis provides an impact evaluation framework but is not suited to developing the determinants of consumption  More detailed information available on the performance of the DHP from lab and field monitoring  Meta-Statistical  Both the billing analysis and the engineering analysis provide a direct measure of the savings potential  Comparable studies in size and scope have not been attempted elsewhere  Small engineering studies in this region can be used to expand this analysis but are not focused on this measure and the electric zonal heat population targeted  Calibrated Engineering  Integrated research design focused on the performance characteristics of the DHP  Detailed lab testing of system performance  Submetering provide confirmation of lab test and determinants of consumption across a number of climates and participants (95)  Testing and metering results allow the development of a DHP model within the SEEM simulation  Analysis shows the structure of the DHP performance in real homes  Calibration of SEEM model can be quite accurate and allow the results of the research to be generalized for climates and house types

13. 13 Tiered Research Approach Shows Agreement Among Various Performance Measurements Lab Measurements Field Performance Measurements Lab supports field findings DHP Performance Simulation Field Energy Use Measurements Annual Energy Use Simulation: Final Savings Estimates Annual Energy Use Simulation: Final Savings Estimates Billing Data Field measurements & billing data agree Lab & field data combine to develop equipment performance models Primary Measurements:Calibrated Models: Billing & field data used to calibrate simulation

14. 14 Measured Performance Lab data forms the truth set for performance measurements  Provides critical insight into equipment operation including  Maximum & minimum capacity  Low power cut off / cycling limit  Defrost operation  Indoor unit fan power  System standby power Lab data verifies the accuracy of the field measurements Lab & field data combined to develop equipment performance models

15. 15 Performance Curve Development  3 curves constructed from field data  HSPF Levels: 12, 10.6, 8.2  Curves are the modal COP values for a given temperature bin  They incorporate all transient effects including defrosting and part load cycling (previous box plots were steady-state only)

16. 16 Multi-Zones with a Single-Zone Model?  SEEM is a single zone heating simulation  DHPs and ER zonal systems are, by definition, multi- zone  Simulation implementation is to determine the fraction of the entire house heated by the DHP at every hour (DHP ƒ ). Then, fill in the remaining heating requirement with resistance heat.  Field measurements show DHP ƒ depends strongly on outdoor temperature and house heat loss rate

17. 17 DHP ƒ Field Observations Fraction of total metered heating provided by the ductless heat pump DHP ƒ = (DHPheat) / (DHPheat + ERheat)

18. 18 Multi-Zones with a Single-Zone Model

19. 19 Measured and Modeled Heating Energy Pre / Post DHP Source Heating Energy Use (kWh/yr) N MeanSD Pre Billing Data9347389291 SEEM 65.8°F Set point9369411191 SEEM 67.3°F Set point10384440991 Post Metered Data6484389491 SEEM 65.8°F Set point5881301891 SEEM 67.3°F Set point6485323491  SEEM can model both the base case houses (ER heat only) and the post case houses (mix of DHP and ER heating)  Per other calibration efforts, house models constructed from detailed audit data and the SEEM t-stat setting changed to match metered energy

20. 20 Agreement: Metered Sites to Billing Sites  95 metered sites are a subpopulation of the 4,000 billing analysis sites  Metered sites used no supplemental fuel and were continuously occupied year round  When we examine the energy savings the billing population would have were it to use no supplemental fuel and be fully occupied, we see the same savings as from the meters and simulation.  Screened billing analysis savings: ~2,975 kWh/yr for 1706 houses agrees well with calibrated modeled savings from previous slide of 2,865 kWh/yr for 91 sites.  Link between metered and billing populations shows we can expect SEEM to correctly simulate energy use under a variety of conditions.

21. 21 SEEM is Calibrated to Model Energy Use in Houses with DHPs  Sound Engineering & Statistical Analysis: Lab and field measurements show us how the equipment performs apart from the occupant.  Reliable Data & Calibration: We can use the simulation with confidence because it is calibrated and validated against field data.  Flexibility: The simulation of the equipment allows us to model various program designs and occupant behaviors giving the possibility to move beyond this pilot project population.

22. 22 Issue 2: How to determine electric kWh savings and supplemental fuel benefit?

23. 23 Distribution of Total DHP Savings

24. 24 Options for Supplemental Heat Option 1: Electric kWh savings are based on results of the billing/metering studies (first-year); supplemental fuel is quantified as found in the evaluation, and monetized.  Analysis :  Uses billing analysis results from the pilot program  Uses savings that are consistent with supplemental fuel use across the region  Pros:  Reduces uncertainty in savings estimate  Cons:  Potentially underestimates long-term savings  Increases risk that changing fuel preferences will increase electric baseline  Reduces savings estimate for homes without supplemental fuel use  May require differential savings based on supplemental fuel use Option 2: Electric kWh savings are sum of first year savings and supplemental fuel savings (long term = no wood use)  Analysis:  Uses the results of detailed metering as the basis for savings calculation  Pros:  Hedge against changing fuel preferences  Cons:  May over estimate electric savings Option 3: Somewhere in between Option 1 and 2; the percentage of long-term wood use is based on ________.  Pros:  Distributes uncertainty across short and long-term  Cons:  May underestimate savings

25. 25 Pilot Project Eligibility and Intake Screens  Electric resistance (ER) heating must be permanently installed and serve as the primary heating system for the home. (Screen for presence of ER; no screen to establish ER as primary heat, meaning no screen for wood/propane as de facto primary heat)  The consumer must have occupied the home for one year prior. (In addition, the consumer should expect to occupy the home for the next two years.). (No screen for length of past/future occupancy)  The consumer must allow their local utility to make their billing histories available. (Included in terms of intake form)  The consumer must agree to participate in the project, project activities, and project evaluation. (Included in terms of intake form)  Participating homes cannot:  be new construction (No screen for new construction)  have natural gas service to the home. (Screen for gas service)

26. 26 Savings and Supplemental Fuels  Supplemental Fuels  Documented in installation questionnaire  Largely wood heat  Some contractors did not ask about supplemental fuel use (especially in some western climate areas)  Dominated by rural participants  Screened out of the metering sample (not quantified)  Supplemental fuels more significant in some areas  Rural areas of western climates.  Most eastern climates (especially Montana)

27. 27 Supplemental Fuel Use by Pilot Population* *Based on data in Baylon, D., P. Storm, and D. Robison. 2013. Ductless Heat Pump Impact & Process Evaluation: Billing Analysis Report, Northwest Energy Efficiency Alliance. Portland OR. Regional avg. =.35

28. 28 Supplemental Fuel Impact  Supplemental fuel impacts can dominate savings estimates  Electric savings impacted by supplemental fuels  Generally Wood, some propane and other fuels  Differential wood use in eastern clusters  Uncertainty in use of supplemental heating  Reduction in wood use implied by negative bill “savings”  Screening suggests about 2000 kWh/yr savings reduction between homes with and without supplemental fuels  CDA regression used to assess the size of the supplemental fuel and other takeback effects  Supplemental fuel savings impact 400 kWh/yr across the entire pilot program  Savings impact 700 kWh in eastern utilities and 350 kWh in western utilities Supplemental fuel is a occupant choice that does not effect the “primary” electric heating system.  Electric heating system remains in place

29. 29 Space Heating Savings: Supplemental Fuels* *Based on screened billing analysis in Baylon, D., P. Storm, and D. Robison. 2013. Ductless Heat Pump Impact & Process Evaluation: Billing Analysis Report, Northwest Energy Efficiency Alliance. Portland OR.

30. 30 Space Heating Savings: No Supplemental Fuels* *Based on screened billing analysis in Baylon, D., P. Storm, and D. Robison. 2013. Ductless Heat Pump Impact & Process Evaluation: Billing Analysis Report, Northwest Energy Efficiency Alliance. Portland OR.

31. 31 How to monetize supplemental fuels? If supplemental fuel benefits are included in the analysis (Option 1 and 3), how should they be monetized?  Option A: Value of wood taken as the avoided cost of wood fuel: Attempt to value wood directly.  Option B: Value of wood taken as the avoided cost of electricity: Use wholesale electricity prices.  Option C: Value of wood taken as the cost of electricity: Use retail electricity prices.

32. 32 Issue 3: How/whether to include “comfort” in the measure savings and/or TRC calculation?

33. 33 Distribution of Total DHP Savings

34. 34 How/whether to include “comfort” in the measure savings and/or TRC calculation? Option 1: Do not include in analysis at all. Option 2: Include in TRC analysis; convert to $’s (benefit). Option 3: Add to kWh savings.

35. 35 Temperature and Comfort (MPER)  Nearly all (97%) of respondents reported that their home was more comfortable (91%) or equally comfortable (6%) than it was prior to installing the DHP.  Only two respondents indicated that the home was less comfortable.  Respondents gave a variety of ways in which their comfort was improved by the DHP.

36. 36 Thermostat Settings and Setback (MPER)  Pre and post-DHP avg. heating temp. reported at 69-70° F  Pre-DHP, 69% reported setback when leaving the house or at night; 23% said they never setback.  Post-DHP, 42% reported setback when leaving the house or at night; 35% said they never setback.  57% said they were heating the same amount pre and post-DHP, 40% said they were heating it more. The most common reason given for heating the area more was that the new heat was cheaper to operate (61%).

37. 37 Savings with Temperature Setting Offsets The DHP offers an improved thermostat control and a reduction in heating energy.  Reduced savings inferred from the temperature observed in the metering.  Regression suggests a 125 kWh savings reduction per ºF  SEEM calibration suggested a temperature adjustment.  About 2 ºF increase in overall thermostat developed in the calibration  Billing analysis CDA showed other savings offsets after supplemental fuels considered.  Thermostat setting one possible source of takeback

38. 38 Detailed interview: Metered Sample  Temperature changes from interviews for metered sample  20% said they increased temperature by about 3ºF on average.  9% said they decreased temperature by about 3ºF on average.  Relation between reported t-stat setting and measured temperature suggests a higher setting than reported by about 1ºF.

39. 39 Inferred Temperature Impacts  Temperature impacts not directly observed  Metering only included post installation conditions  No questions on heat setting  Indicator variable developed to track homes with better heat signatures  Increased R 2 in post installation period  Assigned indicator to the 40% largest increases in R 2 save  Most screened out with supplemental fuels  Some screened out with pre installation R 2 screening  Screened savings “takeback” summaries based on this variable

40. 40 Space Heating Savings: No Takeback* *Based on screened billing analysis in Baylon, D., P. Storm, and D. Robison. 2013. Ductless Heat Pump Impact & Process Evaluation: Billing Analysis Report, Northwest Energy Efficiency Alliance. Portland OR.

41. 41 Space Heating Savings: Temperature Increase Indicator* *Based on screened billing analysis in Baylon, D., P. Storm, and D. Robison. 2013. Ductless Heat Pump Impact & Process Evaluation: Billing Analysis Report, Northwest Energy Efficiency Alliance. Portland OR.

42. 42 “Takeback” Decisions T-stat increases account for savings takeback  SEEM predicts about 400 kWh/ºF  Temperature take back up to 800 kWh/yr in metered sample  Bill screening suggests 650 kWh/yr per home with this indicator  CDA predicts 500 kWh/yr for homes with this indicator Should these takebacks be included as part of the DHP savings benefits?  Option 1: Do not include in analysis at all.  Option 2: Include in TRC analysis; convert to $’s (benefit).  Option 3: Add to kWh savings.

43. 43 Issue 4: Should we combine the savings, or disaggregate by climate zone?

44. 44 Use Similar DHP Savings Total: All climates  Similar installation standards in all climates  Displacement model anticipates optimum output and cost effectiveness in all climates.  Installation standardized at 1-1.5 tons throughout the region  Occupant use similar across region  Supplemental fuel most significant determinant of savings in all climates. Evaluation of COP by climate shows similar response in metered group  Savings dominated by swing seasons in cold climates  Savings available year round in warm climates  Cooling not a significant offset in any climate Average space heat fraction differs between western and eastern climates but absolute saving similar  Use of wood heating different between west and east so uniform grid savings not expected

45. 45 DHP Total Savings Results* *Based on metered heat output measurements in Baylon, D., L. Larson, P. Storm, and K. Geraghty. 2012. Ductless Heat Pump Impact & Process Evaluation: Field Metering Report, Northwest Energy Efficiency Alliance. Portland OR.

46. 46 Installations Similar Across Climates* Cluster Capacity (tons) One Indoor UnitAll Willamette1.41.7 Puget Sound1.31.6 Coastal1.21.4 Inland Empire1.71.9 Boise/Twin1.31.8 Eastern Idaho1.31.6 Tri-Cities1.31.6 W. Montana1.31.5 Total1.41.6 *See Baylon, D., L. Larson, P. Storm, and K. Geraghty. 2012. Ductless Heat Pump Impact & Process Evaluation: Billing Analysis Report, Northwest Energy Efficiency Alliance. Portland OR.

47. 47 Wood Heat Varies by Occupant not Climate* *Based on conditional demand analysis (CDA) data in Baylon, D., P. Storm, and D. Robison. 2013. Ductless Heat Pump Impact & Process Evaluation: Billing Analysis Report, Northwest Energy Efficiency Alliance. Portland OR. Regional avg. = 393

48. 48 DHP measure should be uniform, based on one regional specification and savings.  Total savings very uniform across climates.  Supplemental fuels should be handled separately in assigning “grid” savings.  With common displacement spec common savings should be anticipated.  Climate distinction an unnecessary complication.

49. 49 Questions & Answers

50. 50 Motions

51. 51 Motion 1: Savings Estimation Approach “I ____ move that the calibrated engineering approach be used to estimate savings for DHP measures and programs.”

52. 52 Motion 2: Approach to Supplemental Fuels “I _______ move to include supplemental heat in the DHP savings estimate analysis using the following option: Option 1: First year savings approach. Electric kWh savings are based on results of the billing/metering studies (first-year); supplemental fuel savings is quantified as found in the evaluation, and monetized. Option 2: Long-term savings approach. Electric kWh savings are sum of first year savings and supplemental fuel (long term = no wood use) Option 3: Hybrid approach. Somewhere in between Option 1 and 2; the percentage of long-term wood use is based on ______.

53. 53 Motion 3: Approach to Takeback “I _______ move to include or exclude takeback offsets in final savings estimates.”

54. 54 Motion 4: Uniform DHP Measure “I _______ move to use a single savings estimate for DHP total savings across all climate zones.”

55. 55 Additional Slides

56. 56 Savings Vary by Screening Approach Savings by Participant GroupSavings (kWh/yr) Screened bills: R2>.45, Space Heat>0MeanEBn All2081813390 Supplemental Fuels Used7471441095 No Supplemental Fuels Used2719902295 Takeback based on R 2 1971181590 No Takeback29761011705 Metered Results: Measured Savings from Metered Sub-Sample Total Savings (COP)388737565 Billing Analysis304925193

57. 57 Performance Curves Compared First 2 are for the units tested in the lab and cover a high HSPF range. Third curve, based solely on field data, developed because we needed a lower performing model. The COP begins to bend down at warmer temperatures because of equipment cycling.

58. 58 CDA Billing Analysis Segmentation  Assess the overall savings and the space heating savings from the DHP installations in the pilot project (n=3620)  Determine the impact of takeback effects on observed savings using CDA  Underlying savings rate (c 1 ) similar to meters  Supplemental fuels offset (c 2 ) consistent except MT  Constant term (C) shows impact of other occupancy takebacks

59. 59 CDA Regression Results Climate Zone Segment Parameter n c1c1 c2c2 C Western0.487-973-7683122 Eastern0.223-1,152-300375 W. Montana0.249-1,683-416123 All0.434-1,110-5613620 SH saved =c 1 SH pre +c 2 SuppFuel+C Climate Zone Segment Parameter n c1c2C Est.EBEst.EBEst.EB Western0.4790.016-1078131-6761403,122 Eastern*0.2190.046-1220456-226519375 W. Montana*0.2410.096-17611263-2751545123 All0.4260.015-1208129-4661393,620

60. 60 CDA Savings Estimates  CDA savings estimates similar to the screened billing analysis results  Supplemental fuel decrements the total savings by about 1200 kWh/yr in those homes  Other “Takeback” captured by constant term  Regression did not include R2 indicator  Screening results suggest about a third of the constant effect is explained by this indicator  Overall savings represents the comparable screening to the metered sample

61. 61 Segmented Regression Results

62. 62 CDA Predicted Space Heating Savings

63. 63 The CDA comparison Overall savings from the CDA equivalent to screened billing analysis  Based only on space heat coefficient (c 1 )  Comparable to Metered Sample billing savings Overall “takeback” from all sources equal to a third of predicted savings  Supplemental fuels account for 40% of that reduction  Temperature “takeback” accounts for about 25% of that reduction  The balance is the result of occupancy or other behavior How should these effects be included in the cost/effectiveness analysis?

64. 64 Savings and C/B Recommendations  Supplemental fuel taken as the value of the offset electric heat  Temperature and occupancy effects should be ignored  Total savings should be used for TRC cost effectiveness  Based on the c 1 coefficient  c 1 =.48 for western climates.24 for eastern climates  Grid Savings would account for observed takeback especially supplemental fuels

65. 65 Modeled and Measured COP