1. Michael Li, Senior Policy Advisor Office of Energy Efficiency and Renewable Energy U.S. Department of Energy

2. What is Green Button?
Common-sense idea that electricity customers should be able to download their own energy usage information in a consumer- and computer-friendly format.
Source: 2

3. Who is implementing green button?
Utilities and electricity suppliers in 24 states across various regulatory regimes have committed to provide 30 million US homes and businesses Green Button data.
Commitment to Green Button:
American Electric Power
Austin Energy
Baltimore Gas & Electric
CenterPoint Energy
Chattanooga EPB
Commonwealth Edison
Glendale Water and Power
National Grid
Oncor
PECO
Pepco Holdings
PPL Electric Utilities
Pacific Power
Rocky Mountain Power
Southern California Edison
Virginia Dominion Power
Utilities & Electricity Suppliers with Green Button today:
(almost 10 million homes)
NSTAR
PG&E
Reliant
SDG&E
TXU Energy
NSTAR
PG&E
Reliant
SDG&E
TXU Energy

4. Green Button Facts What it is:
Energy usage information in a common XML format (NAESB ESPI data std)
PG&E example: how much electricity (kWh) did one metered customer consume every hour for the last year
Markets: Residential, commercial and industrial
Type: initially electricity, but the data standard is extensible to gas and water data
Timeliness: Not real time data, but 24 hour-old back office data.
Time interval:
Again the data standard is extensible and could include any interval of data.
Most will provide 15 minute interval or hourly data.
However, some will provide monthly data, too.
Metering system:
You don’t need to have AMI to participate. AMR works, too.
Transfer of data:
Green Button, Download My Data – goes directly from the utility to the customer; most will implement this version.
Green Button Connect - automated data transfer from the utility to a third party with customer authorization is the 2nd part of the data std; may be implemented as early as this year in 1-2 states.

5. Helping to spur new innovation and Green Button Apps

6. Apps for Energy Winners
Best Overall App Grand Prize: Leafully
Location: Seattle, Washington
Leafully, helps utility customers visualize their Green Button data, as a variety of units, such as the amount of trees needed to offset an individual’s energy usage. 
Best Overall App Second Prize: Melon
Location: Washington, DC
The app uses Green Button to evaluate the energy performance of commercial buildings. 
Best Overall App Third Prize: VELObill
Location: New York, NY
 VELObill app helps makes it easier for utility customers to view their energy usage, measure whether it is high or low, and compare it to that of their peers. 
Best Student App Grand Prize: wotz 
Location: Irvine, CA
Best Student App Second Prize: Budget It Yourself
Location: Cleveland, OH

7. SEE Action Working Groups
Working Group Leadership Susan Ackerman, OR PUC Vaughn Clark, OK SEO Todd Currier, WA SEO Jennifer Easler, IA Consumer Advocate Joshua Epel, CO PUC Jim Gallagher, NY ISO Bryan Garcia, CT Clean Energy Fund Frank Murray, NY SEO Pat Oshie, WA PUC Phyllis Reha, MN PUC Cheryl Roberto, OH PUC Janet Streff, MN SEO Keith Welks, PA Treasury Malcolm Woolf, MD SEO
Each of the working groups developed a blueprint (or roadmap) for achieving near- and long-term aggressive goals critical to capturing all cost-effective energy efficiency within the respective sectors.

8. Evaluation, Measurement, and Verification (EM&V) of Residential Behavior-Based Energy Efficiency Programs: Issues and Recommendations
June 25, 2012
Michael Li US Department of Energy
Annika Todd Lawrence Berkeley National Lab

10. Outline: Evaluation of Behavior-Based Programs
What is a behavior-based energy program?
Why is evaluation of these programs hard?
Why is designing a program as a “randomized controlled trial” (RCT) so important?

11. What is a behavior-based energy program?
Programs that affect the way that consumers use energy without using traditional methods, such as prices and rebates
Instead, use simple psychological levers or information to change behavior
Example 1: Comparing your energy use with your neighbors
Example 2: Providing real-time information about energy use
Other examples:
Competitions, rewards: Turning energy use into a game
Education / Outreach: Information about energy behavior
Display of feedback: Simplify / Framing

12. What are the potential benefits and concerns?
In theory, potentially cheap to implement and result in significant energy savings  cost effective
As a result, increasingly being adopted nationwide
Potential Concerns
In reality, these programs are relatively new and evidence of energy saving effects is unclear
Potential for unsubstantiated claims

13. Why is evaluation crucially important?
 It is very important to measure effect of these programs
Need to gain information about how well different types of programs work
Are the estimates energy savings valid for utilities to claim savings?

14. Why is evaluation of these programs hard?
Strong problems of “selection bias”: households who join (choice, screening) are fundamentally different
Observed differences might be due to program; might difference between groups
Selection bias can skew the results of the evaluation
Join
Didn’t
Join
Population

15. Why is evaluation of these programs hard?
Energy programs “selection bias”: households who opt-in may be more energy conscious
Observed difference in energy use might be due to the program; but might be difference between groups
Opt-in
Don’t opt-in

16. Why is evaluation of these programs hard?
It may be more difficult to measure the impact of behavior-based programs correctly (in contrast to other programs such as appliance rebates)
Impacts vary significantly between households
Within a household, hard to disentangle changes in overall energy usage between program, other factors
Savings are relatively small: often 1-5%, so if an evaluation is biased, large implications

17. Why is evaluation of these programs hard?
 Bad evaluation could lead to bad policy decisions

18. SEE Action Report
“Evaluation, Measurement, and Verification (EM&V) of Residential Behavior-Based Energy Efficiency Programs”
Provides guidelines and best practices for
Program design
Program analysis and evaluation given design
Provides rankings for different methods
Target audiences:
Senior managers responsible for overseeing and reviewing efficiency program designs and evaluations
Practitioners, evaluation professionals, and staff responsible for reviewing efficiency program designs and evaluations

19. Why is designing a program as a randomized controlled trial (RCT) so important?
Primary recommendation – a well designed, RCT program results in:
Transparent, straightforward analysis
Robust, accurate, valid program impact estimates
High degree of confidence in program effectiveness
Why?
RCT means that households are assigned to the program randomly (as opposed to household choice or screening criteria)
Solves selection bias

20. Why is designing a program as a randomized controlled trial (RCT) so important?
If RCTs are not feasible, recommendations for acceptable “quasi-experimental” methods
More opaque, difficult, complex analysis
Quasi-experimental methods try to correct for selection bias
Lower degree of confidence in validity of savings estimates

21. Other Key Recommendations
Problem: Potential conflicts of interest
Recommendation: Third-party evaluator transparently defines and implements program evaluation, assignment to control and treatment groups, data selection
Problem: The same savings may be claimed by two programs (e.g., a behavioral program & appliance rebate program both claim savings from appliances)
Recommendation: Estimate and account for this “double counted savings” overlap to the extent possible by comparing control to treatment groups

22. Recommendations for the Future
The hope is that in the future, we will have conclusive evidence about the effectiveness of different types of behavior-based programs
Move away from RCTs
We are not yet at this point…

23. Questions?
Main point: evaluation of behavior-based programs is difficult, but if the program is designed in the right way (using a RCT) then we can be confident that the evaluation of the program’s energy savings is valid
Many guidelines and technical recommendations in the report:
SEE Action website,
Lawrence Berkeley National Lab website: behavioranalytics.lbl.gov
Mike Li:
Annika Todd: