1. Energy Efficiency Evaluation Update Energy Division
Quarterly Stakeholder Meeting
June 10, 2014
California Public Utilities Commission

2. EM&V Status Overview Status of 2010-2012 EM&V Work Plan:
Budget adjustments
Spend to date
Evaluation report timelines
Status of the EM&V Work Plan
Current status & timeline
6 month update on EM&V plans
Expected revisions to V4 of research roadmaps [9/14]

3. Status of 2010-2012 EM&V Work Plan
Expenditures to date are about 88 percent of total allocated for Energy Division evaluation.
$74.8 million has been invoiced to date on evaluation, advisory, and support contracts
Small shifts may still occur for budgets on
All contracts funded via authorization will be closed out by December 31, 2014.
“Evaluation Project Status Reporting”
website for current budgets, projects and status:
Expenditures to date can be found on the same page at the following link [Select “expenditures” from the Home menu:
- Denominator = total ED authorization (75% of 86 million = 65 M + 10M Advisory + 10 M Reserve= 85 M) Numerator = consultant + advisory + DEER + CIEE invoiced to date on CMS SOURCE = = $74.8 M.

4. 2010-2012 Evaluation report timelines
Several impact evaluations provided preliminary findings in 2013
Program managers and other stakeholders are using in planning activities for 2015.
Ex Ante and Evaluation impact teams are coordinating to ensure information is available for the next DEER update.
Final results from remaining impact studies and for the full portfolio will be provided in 2014;
Final results will cover 70% of the portfolio claimed kWh savings
Results from individual studies will be vetted with stakeholders in the upcoming months.
Eval based summary of portfolio will be provided summer 2014

5. 2010-2012 Impact Evaluation UpdatesWO
% of Claim Evaluated
Title
Expected Date of Public webinar(s)
red = change
Parameter
WO28
33% kWh
Upstream Lighting
July 18, 2013 (Gross)
Q March (Net)
Irate, UES, NTG
WO29
16% kWh
Down Stream Lighting
Q March
UES, NTG, EUL/RUL, Irate
WO32
2% kWh
1% therms
HVAC
Q (prelim done)
Q (final) - March
UES, NTG
WO33
13%kWh
28% therms
Ag/Industrial/Com Custom
Q (PY10-11) - done;
Q (PY10-12) March
NTG, RR
WO34
Business and Consumer Electronics
May 29, 2013 (Done)
NTG
WO35
Appliance Recycling
Q1 2014
UES, NTG, Irate
WO46
<1% kWh
Whole Building - Res
Total:
70% kWh
30% therms
Check with the assigned Energy Division Staff lead for details; and stay tuned to the Service List announcements for webinar dates and logistics.

6. 2010-2012 Other Pending Studies
Commercial Market Share Tracking and Commercial Saturation Study
Webinar set for Wednesday, June 18, :30 PM – Lisa Paulo
Codes and Standards Impact Evaluation
Draft report being reviewed internally
Webinar will be scheduled in the next month Paula Gruendling
Market Studies
Residential Multi Family webinar held 5/29
Whole House Market Baseline Study (today)
LED Market Baseline Study (today)          

7. Energy Division Annual Progress and Evaluation Report Timeline
Full report on evaluation-based results will be released after all results are finalized. Current goal is Q
Commission staff is revising future portfolio-level reporting strategy to account for:
Division needs
Legislative requirements and needs
Energy Savings Performance Incentive (ESPI)
California Energy Commission load forecasting and California Independent System Operator needs

8. 2013-2014 EM&V planning: process, status & timeline,
Energy Division and IOU staff released v.3 of Joint EMV Plan
Next update expected Q3 2014
Current version :
Updates to Non-Residential research roadmaps will be in V4 due to 3 month delay in contracting.
Evaluation Plans will be shared via Public Webinars and announced to service list prior to the plan update.
Residential – DNV-GL
Non-Residential- Itron
Finance/Non-Resource – Opinion Dynamics Corporation
Residential
Ind/Ag/Lrg Commercial
Finance
Codes and Standards (no update - delay in eval)
Small and Medium Commercial
Workforce Education & Training
HVAC
Lighting
Integrated DSM
Market Studies
Government Partnership
Marketing Education & Outreach (delay due to new Decision)
Reporting and Data Management
Emerging Technologies Program

9. Market Effects Studies
CPUC-led
Market Effects Studies
June 10 EE EM&V Quarterly Stakeholder meeting
CPUC Courtyard Room

10. WO 54 Study Selection Process
Criteria for selection of markets and technologies
for studies
Sufficient recent or planned IOU program activity to stimulate lasting changes in customer and/or supplier attitudes, knowledge or behavior.
Market is likely to experience significant changes over the next 3-5 years in terms of technology performance, price, market share.
The market or technology represents significant forecasted savings.
Relatively little attention from prior evaluations and studies.
Market or technology identified as strategically important in CPUC and other state planning documents.
Steps in Process
Solicited input on candidate projects
DNV KEMA / NMR prepared initial assessment
Developed full study plan

11. Market Effects Studies Selected
Topics considered but not selected
Commercial refrigeration
Zero Net Energy programs
Various workforce development programs
Building automation and controls
Topics selected
LED lighting (today)
Whole House Retrofits (today)
HVAC Maintenance- likely late July or August
Multifamily New Construction (posted May 29)
Prime contractor: DNV KEMA
Key sub- contractors: NMR and TRC.

12. WO54 Market Effects Studies~ Basic Two Phase Approach
Baseline Study. Collect data in California and non-program areas on key market development indicators, including:
Customer awareness, knowledge and reported adoption of targeted products and services
Sales and market share of targeted products and services
Promotional, technical and sales support for targeted products and services from firms in the supply chain
Manufacturer development of new products and targeting of geographic markets
Follow up Study (2-4 years later)
Collect data to develop the same indicators
Use the difference between the California and non-program areas in the change in market indicators over time to characterize and quantify market effects

13. And, a few reminders…. Tools for Stakeholder Participation From 2013-14 ED-IOU EM&V Plan
All EM&V project scopes and work orders developed by Energy Division and the IOUs will be posted on the Energy Division public document website at
In some cases, draft work orders will be posted for public comment. Notice of posting of work orders will be sent to work order topic subscribers on the public document website and, when appropriate, to the applicable CPUC service list.  
Comments on draft work orders will be submitted to and will be publicly available on the public document website.
Energy Division and the IOUs may hold public meetings (in person, via webinar, or via webcast), separately or jointly as the case may be, to solicit input on certain draft work orders posted on the public document website.
The finalized work orders or project scopes will be posted on the public document website.

14. Public Review of EM&V Technical Reports
Drafts of any interim or final EM&V technical reports produced by ED and the IOUs will be posted on the public document website.
A notice will be distributed to the appropriate service list(s) and to subscribers to the pertinent topic in the public document website.
EM&V technical reports are defined as reports produced by the ED and the IOUs, working with their respective contractors that document completed analysis and are identified as a milestone in the Evaluation Plan or in a work order.
ED and the IOUs will hold public meetings (in person, via webinar, or via webcast), separately or jointly as the case may be, to solicit input on the EM&V Technical Reports.
Parties may submit written comments on these reports before and/or after the public meetings.

15. Public Review of EM&V Technical Reports (cont)
Comments on draft documents must be submitted to the public document website and will be publicly available.
Energy Division will also review and reserves the right to provide written comments to the IOUs on their EM&V contractors’ draft reports.
Energy Division and the IOUs, working with their respective EM&V contractors, will finalize the draft reports, taking into consideration the parties’ written comments and/or input during the public meetings.
The final EM&V technical reports will be posted on the public document website.
Responses to written comments on the draft reports will be prepared and posted at the same time or soon after the revised reports are posted. Notice of posting of final technical reports will be sent to subscribers to the relevant topics on the public document website and to the appropriate service list.

16. Public Review of EM&V Technical Reports (cont)
Within 60 days of public release of a final report, the program administrators will respond in writing to the final report findings and recommendations indicating what action, if any, will be taken as a result of study findings. The IOU responses will be posted on the public document website.