1. Tool Lending Library Program evaluation
Impact Evaluation
December 12th, 2017

2. Agenda Background information Methodology
Project and Tool Borrower Characterization
Savings Potential
Claiming Savings in the Future
Findings
Recommendations
Tool Lending Library Impact Evaluation

3. Background Four PA Supported Tool Lending Libraries (TLLs) Terminology
PG&E’s Pacific Energy Center (PEC)
PG&E’s Energy Training Center in Stockton
SCE’s Agricultural Technology Application Center
SDG&E’s San Diego Energy Innovation Center
Terminology
Application Notes
Borrower
Loan
Project
Project Type
Site
Tool
Tool use
Tool Lending Library Impact Evaluation

4. Research Objectives
Objective 1: Characterize projects initiated through a TLL loan across all four PA’s TLLs in 2016
Objective 2: Quantify the savings potential for two projects supported by the PEC’s TLL and assess the tool’s contribution to those savings
Objective 3: Explore if and how the PEC TLL can claim savings in the future
Tool Lending Library Impact Evaluation

5. Methods Interviews with PEC Staff (n=3) Secondary Data Review
Review of Program Tracking Data
CPUC EM&V Documents
Past evaluation reports
Participant Web Survey (n=107)
Census attempt of 2016 borrowers
Case Studies (n=2)
Engineering analysis of two projects supported though PEC tool loans
Tool Lending Library Impact Evaluation

6. Participant Survey Primary Goals Characterize projects and borrowers
Understand how borrowers used tools
Education and training
Energy Center
Completed Surveys
Response Rate
PG&E Pacific Energy Center (PEC) 67
30%
SDG&E San Diego Energy Innovation Center 23
20%
SCE Agricultural Technology Application Center 15
25%
PG&E Energy Training Center 2
18%
Total
107
27%
Tool Lending Library Impact Evaluation

7. Case Studies—Selection Criteria
Used participant survey to recruit projects for deeper analysis
Employed the following criteria for selection:
Data availability—Ensured borrowers had pre- and post- implementation data
Most recent projects—Borrowers may have used tools to support multiple projects
Projects with direct savings—Asked borrowers through the survey to specific if project’s had measureable savings (or renewable generation)
Tool Lending Library Impact Evaluation

8. Project Characterization
Broad project categories
Baseline Assessment
Pre-Installation Measurement
Health, Safety, and Maintenance
Post-Installation Measurement
Training, Education, and Outreach
TLL projects with savings that received a utility incentive (45%)
Tool Lending Library Impact Evaluation

9. Borrower Characterization
Half of all borrowers said their 2016 loan was not their first TLL experience
How borrowers learned about TLL
Energy center course (21%)
Word of mouth (11%)
Visiting their local energy center (7%)
Energy center website (5%)
Utility incentive program (4%)
Other (4%)
Tool Lending Library Impact Evaluation

10. Borrower Characterization
Majority of borrowers are employed in the energy industry
Nearly one quarter (23%) of borrowers supported projects in borrower’s homes
Tool Use
Percent of Borrowers
For my work
63%
For my home
23%
For a course I taught 2%
For a course I took 5%
Tool testing or evaluation 3%
Other
Tool Lending Library Impact Evaluation

11. Borrower Characterization—Business Type
Nearly half (49%) identified as either ESCO or Contractors
Tool Lending Library Impact Evaluation

12. Borrower Knowledge
Tool Lending Library Impact Evaluation

13. Borrower Education and Training
Application Notes
Received application notes (83%)
Used application notes (82%)
Information in application notes was new (53%)
Application notes were “Very Useful” (85%)
Energy Center Course or Ad-hoc Training
Received additional training (35%)
Information in training was new (81%)
Training was “Very Useful” (89%)
Tool Lending Library Impact Evaluation

14. Borrower Behavior Change
Identifying energy-saving opportunities for the buildings you serve
71% took this action since their 2016 tool loan
78% said the TLL was “Very Influential”
Other energy saving actions
Energy-Saving Action
Took Action
Very Influential
Turned off lights more frequently at home
55%
61%
Purchased energy-saving equipment for your home
50%
62%
Adjusted your HAVC system to save energy at home
49%
Turned off lights more frequently at work
41%
73%
Tool Lending Library Impact Evaluation

15. How Borrowers Use Tools
Tools used largely for benchmarking and measurement
* Percentages will add up to more than 100% as loans may have been used for more than one purpose listed
Tool Lending Library Impact Evaluation

16. Savings Potential—Case Studies
#1 Central Plant EE
#2 Residential Solar PV
Multi-measure C&I controls project
Borrower works for an energy services company
Tools borrowed:
Hobo data logger
Ultrasonic flow meter
Incentive  Yes
Homeowner trying to determine best location to install PV system
Borrower is a retired PG&E employee
Tools borrowed:
Solmetric SunEye
Incentive  No
Tool Lending Library Impact Evaluation

17. Savings Potential—Case Studies
TLL had very high level of influence on both projects
Neutral to higher likelihood projects would have been implemented and had the same savings without the TLL
Survey Question
Case #1
Case #2
TLL Mean Rating
Influence on decision to implement the project 9 8
8.4
Likelihood that the project would have been implemented without the TLL 5 4
3.5
Likelihood that the project would have generated the same capacity without the TLL 7
3.8
Tool Lending Library Impact Evaluation

18. Savings Potential
Majority of loans supported projects that did not produce measureable savings (56%)
Of projects that did lead to savings (36%), majority leveraged tool loan prior to implementation
76% of projects with savings, 27% overall
Revised Project Categorization
Count
Percent
Projects with Savings 38
36%
Tool Used Before Implementation 29
27%
Tool Used After Implementation 9 8%
Projects with No Savings 55
56%
Unsure if Project lead to Savings
Total
107
Tool Lending Library Impact Evaluation

19. Savings Potential—TLL Influence
The majority (88%) of borrowers say the TLL was “Very Influential” on their decision to move their project forward
Fifty-five percent said their project was not likely to move forward without the TLL
Note: Savings are based on survey responses and are self-reported
Tool Lending Library Impact Evaluation

20. Claiming Savings in the Future
Evaluation Protocols
M&V Protocols
Impact Evaluation Protocols
Net Impact Evaluation Methodologies
Indirect Impact Evaluation Protocol
Barriers
Transaction costs
Potential for high levels of Free-ridership
Splitting savings between programs
How to handle projects without savings
Tool Lending Library Impact Evaluation

21. Indirect Impact Evaluation Protocol
Source: California Energy Efficiency Evaluation Protocols, State of California Public Utilities Commission, April 2006
Tool Lending Library Impact Evaluation

22. Claiming Savings in the Future
Based on CA Evaluator’s Protocols
Project-level tracking
Pre- and post-installation energy usage data;
Pre- and post-specifications (i.e. product cut sheets);
Detailed scope of work;
Demographic characteristics; and
Other administrative documentation as needed.
Tool Lending Library Impact Evaluation

23. Key Findings TLLs frequently serve repeat borrowers
Tool loans support a range of project types
Energy measurement activities represent largest share of tool loans
The TLLs play some role in moving projects forward
Loans likely contribute to savings claimed through other programs
Education and training that TLLs provide, specifically those with dedicated staff, is of high value to borrowers
Indirect impact evaluation protocol provides some guidance on how the TLLs may claim savings in the future
Tool Lending Library Impact Evaluation

24. Recommendations
The PEC should consider tracking additional categorical details at the project-level
If claiming savings, decision makers should consider:
How to track data projects with indirect savings potential (44%)
Added transaction costs
Process for multiple PA programs to claim savings
TLL decision makers and staff should build on education and support offered to tool borrowers
Tool Lending Library Impact Evaluation

25. Thank You! Ellen Steiner, Ph.D. Vice President