1. EE EM&V Quarterly Stakeholder Meeting Q2 2017
2017 CPUC Impact Evaluations:
Preliminary Results and Budget Update
Robert Hansen, Utilities Engineer
Energy Division – Energy Efficiency Branch
Commercial Programs and Portfolio Evaluation Section

2. Preliminary Results from Recent CPUC Studies
Home Upgrade (SF & MF)*
HVAC 1 & 3
Home Energy Reports*
Commercial Pool Covers
Upstream Lighting
Commercial Pipe Insulation
Downstream Lighting
Agricultural Sprinklers
Custom Lighting
IALC
*Report released in draft 2

3. Recently Completed CPUC Studies: Single and Multi-Family Home Upgrade
Summary of Recommendations:
Improve review process for the Home Upgrade Program for savings expectations and tracking data.
Building envelope measures are most effective among inland climate zones. Research should be done on climate zones 1-4 to develop savings estimates for efficiency projects.
Program managers should target customers in older homes in climate zones with more heating/cooling degree days, and underscore immediate customer benefits in advertising. 3

4. Recently Completed CPUC Studies: Home Energy Reports (HER)
Summary of Conclusions:
Depending on the program administrator, savings was around 1% or not significantly different from zero at the 90% confidence interval.
Net savings was uniformly non-negative. 4

5. Recently Completed CPUC Studies: Upstream and Downstream Lighting
Summary of Recommendations:
Improve data collection and validation.
Shift incentives away from big-box stores where free-ridership is high and UES is low toward smaller retailers, i.e. grocery, drug, and hardware stores.
Study the end uses of high-wattage CFLs, which may be replacing lower-brightness lighting than anticipated.
Establish a clear definition of customer satisfaction and determine how important is lamp quality. 5

6. Recently Completed CPUC Studies: Upstream and Downstream Lighting6

7. Recently Completed CPUC Studies: Upstream and Downstream Lighting
Hybrid waterfall graphic for MSB CFL basic spiral ≤ 30 W, SDG&E
Hybrid waterfall graphic for MSB CFL reflector ≤ 30 W, SCE
Hybrid waterfall graphic for MSB CFL A-lamp ≤ 30 W, SCE
Hybrid waterfall graphic for MSB CFL high-wattage > 30 W, PG&E 7

8. Recently Completed CPUC Studies: Custom Lighting
Summary of Recommendations:
Custom projects should be submitted with complete parameter data in support of savings calculations.
Custom project workbooks should be standardized further so that savings calculations are transparent and consistent across projects.
Projects claiming early replacement should submit more extensive documentation of the justification.
Projects should take care when replacing especially high wattage HID lamps to ensure proper zonal illumination. 8

9. Recently Completed CPUC Studies: Upstream HVAC Programs (HVAC 1)
Summary of Conclusions:
Updates to upstream HVAC measures in DEER were successful in producing more accurate savings estimates.
Program managers should ensure all eligible savings from, e.g., economizers not required by code (<4.5 tons), are claimed.
Consider a way of ensuring economizers are installed and used correctly before distributing incentives.
Program managers should ensure program guidelines and deadlines are adequately publicized to equipment distributors, and that training and marketing resources are available.
Program managers should develop or improve feedback systems to allow distributors to provide insights to the utilities.
Program managers should collect higher quality buyer tracking data to improve future evaluations of causal pathways. 9

10. Recently Completed CPUC Studies: Downstream HVAC Programs (HVAC 3)
Summary of Measure Conclusions:
Thermostat measures should shift toward remotely programmable models to allow greater savings and demand response functionality.
Coil-cleaning remains low-cost and low-savings, but nonetheless worthwhile.
Supply-side fans should be better targeted to buildings where the existing fan is not already set to ‘off’ or ‘auto’.
Refrigerant Recharge measures should be targeted to systems with very low existing charges. 10

11. Recently Completed CPUC Studies: Downstream HVAC Programs (HVAC 3)
Summary of Residential Program Conclusions:
Residential Quality Maintenance programs showed little or no savings, attributed to incorrect maintenance techniques.
Program managers should review implementation issues that may negatively impact savings.
Program managers may consider major program changes and new measures.
Explanatory research may help inform improved residential program design. 11

12. Recently Completed CPUC Studies: Downstream HVAC Programs (HVAC 3)
Summary of Commercial Program Conclusions:
Evaluated savings in commercial programs was relatively low and highly uncertain.
Commercial Programs should consider additional measures.
Commercial Programs should consider deploying the highest quality practices and contractors to the lowest-performing sites in order to achieve the greatest savings. 12

13. Recently Completed CPUC Studies: Commercial Pool Covers
Summary of Conclusions:
Evaluated projects had high instances of in-kind or industry- standard replacements, resulting in zero savings.
Realized savings is heavily dependent upon operational decisions, such as when and how a pool is heated and how the cover is used.
The net lifecycle realization rate for the program was very low at only 0.1%.
Evaluators recommend ending the program or requiring installation of pool covers with higher R-values than pre- existing. 13

14. Recently Completed CPUC Studies: Commercial Pipe Insulation
Summary of Conclusions:
All rebated pipe insulation was installed as tracked.
Affected large commercial boilers were 50% more active than assumed, resulting in greater savings than predicted, while industrial boilers were 21% less active.
Bare pipe temperatures were lower than assumed.
Boilers operate at a higher combustion efficiency than assumed.
Pipe diameters tended to be larger than assumed.
The evaluated NTG of 0.44 was lower than assumed 0.62. 14

15. Recently Completed CPUC Studies: Agricultural Sprinklers
Summary of Conclusions:
A site-specific calculator should be developed to reduce the difficulty in accurately characterizing deemed savings.
The program should perform better data collection and screening of applicants to avoid ineligible projects.
A crop- and irrigation method-specific calculator should be employed by the program to help account for savings when crops are replaced or a farm switches from, e.g., flood to drip irrigation systems.
Operating pump efficiency paperwork is necessary for accurate savings estimations, and should be included in project application paperwork.
The program should continue to claim peak demand savings from micro- nozzle and drip measures. 15

16. Recently Completed CPUC Studies: Industrial, Ag, and Large Comm. (IALC)
Summary of Conclusions:
Program Administrators should improve eligibility requirements, program manuals, training, and quality control procedures to screen out ineligible projects.
Models should be properly calibrated to site conditions, and normalized according to weather, seasonality, and production.
Program administrators should give extra care in selecting baselines according to CPUC policies, and ensure selection criteria are well-known and followed.
Pre-intervention conditions should be recorded more accurately.
Custom IALC programs would benefit from a standardized, statewide document similar to the Project Practices Assessment (PPA).
Program Administrators should make efforts to reduce free ridership. 16

17. Recently Completed CPUC Studies
Evaluation reports are available now on:
CALMAC
and
Energy Division’s Public Document Area 17

18. Current Budget as of June 8, 2017
Analysis Category Study Type
Invoiced To Date
Percent of Total Invoiced to Date
Sum of Budget Amount
Percent of Budget Invoiced to Date
Advisory Consultants
$342,596.00
0.59%
$444,136.00
77.14%
Data Management and Reporting
$124,516.00
0.22%
$219,136.00
56.82%
Technical Advisory Consultants
$218,080.00
0.38%
$225,000.00
96.92%
Program Impact Evaluations and Measurement and Verification Projects
$54,574,498.35
94.35%
$68,706,475.00
79.43%
Administrative
$6,008,449.27
10.39%
$6,596,988.00
91.08%
$4,074,728.58
7.04%
$4,937,492.00
82.53%
Goals and Potential
$3,288,973.41
5.69%
$7,418,453.00
44.34%
Impact Evaluation
$30,608,837.32
52.92%
$36,511,816.00
83.83%
Market Assessment
$5,887,362.48
10.18%
$7,080,785.00
83.15%
Process Evaluation
$4,298,171.02
7.43%
$5,420,786.00
79.29%
Strategic Plan
$407,976.27
0.71%
$740,155.00
55.12%
Technical, Management and Policy Support Consultants
$2,923,230.47
5.05%
$4,000,000.00
73.08%
Ex-ante development and approval
Grand Total
$57,840,324.82
100.00%
$73,150,611.00
79.07% 18