1. Why Data Matters! Building and Sustaining a Business Case Kansas City NEUAC June 18, 2014

2. Presentation Outline Performance Metrics Example –Findings from Evaluation –Targets for Performance Measurement –Follow-Up Assessment through Evaluation Using Evaluation to Quantify Benefits –Elderly Health Benefits from Air Conditioning –Client Affordability Benefits from WX –Societal Environmental Benefits from WX Using Evaluation to Dispel Myths –Speak with Confidence about “Take-Back” 2

3. Performance Metrics Example 3

4. Performance Measurement is Cool (60s), Radical (80s), Awesome (90s), Sick! (current) …and a lot of different types of people/organizations are doing it 4

5. NBA EXAMPLE Rose is the type of player who is defined by more than his statistics. His impact on the offense goes beyond what he contributes to box scores. Before his injury issues started escalating in 2012, the Bulls were the third most efficient offense in the NBA, per NBA.com/STATS. Modern analytics have helped to shape what offenses do. By tracking shot locations, it’s become apparent that some areas of the court are more critical than others…. By Kelly Scaletta, Featured Columnist Aug 15, 2013 http://bleacherreport.com/articles/1739774-what-is-the-derrick-rose-effect 5

6. Why would an NBA Player hire a personal Data Analyst? To get better! To sit beside him/her during contract negotiations! 6

7. Why would the LIHEAP Program or the Weatherization Program get involved with Performance Measurement? To get better! To use during contract negotiations …. budget discussions! 7

8. Performance Metrics Framework Program Evaluation vs. Performance Measurement 8

9. Comparison 9 Evaluation PeriodicIn-DepthExternal Performance Measurement OngoingDevelopmentalInternal

10. Weatherization Evaluation 10 Climate ZoneMy StateDifference PreWX Usage 1,150 therms1,412 therms+23% Mean Savings 225 therms195 therms-15% Saving Percent 20%14%-30%

11. Explanatory Factors 11 Factor Climate ZoneMy StateDifference Savings 225 therms195 therms-15% PreWX Usage 1,150 therms1,412 therms+23% % Attic Insulation 71%60%-15% % Wall Insulation 35%10%-71% % Furnace Replacement 23%39%+70% Average Spending $3,752$4,340+16%

12. Field Inspections 12 High Priority Measures: Were all wall insulation and attic insulation opportunities addressed? What are the opportunities and barriers? Furnace Replacements: What share of furnace replacements were required for health and safety? Measure Prioritization: How were measures prioritized for each home? Cost per Home: What are the cost implications for changing measure priorities?

13. Performance Goals 13 Wall Insulation – From 10% to 20% Attic Insulation – From 60% to 70% Furnace Replacement – From 39% to 30% –Maintain ALL Health and Safety Replacements Spending – Limit annual increase to 3%

14. Follow-Up 14 Staff Reports – Successes in and/or barriers to installing attic and/or wall insulation. Quarterly Report – Installation rates vs. installation targets, including required H&S measures. Annual Report – Summary statistics AND outcomes of field inspections. Follow-Up Evaluation – Do changes in performance lead to improvements in savings?

15. Documenting Program Benefits 15

16. Colorado Evaporative Cooler Program 16 Program Target – Elderly and disabled clients in hot areas. Intervention – Install modern evaporative coolers that use no more energy than fans. Question – How does this impact clients who receive the intervention? Method – Baseline interview in Summer 2008 / Follow-up interview Summer 2009 (Interviews matched for time and temperature)

17. Elderly/Disabled Client Health Outcomes 17 Symptom / Activity No ACAC Installed% Difference Loss of Appetite 42%10%-76% Dizziness 35%10%-71% Low Energy 66%22%-67% Nausea 26%7%-73% Household Chores (frequently) 44%66%+50% Cooking (frequently) 44%62%+41% Symptom = Experienced in the last week / caused or made worse by the heat

18. Conclusions Having air conditioning and using it, reduces the incidence of heat related illness symptoms. Having air conditioning and using it increases the independence of elderly and disabled individuals. 18

19. Weatherization Program Affordability Impacts 19 Program Target – Low-Income Households. Intervention – Comprehensive Weatherization Services / WAP and LIHEAP. Question – How does weatherization affect energy bill affordability for clients? Method – Client Survey PreWX compared to Client Survey Post-WX.

20. Low-Income Household Affordability Outcomes 20 Payment Outcomes PreWXPostWX% Difference Payment Problem / 1 Month+ 46%36%-22% Shutoff Notice / 1 Month+ 39%33%-16% Utility Shutoff 13%9%-31% Use High Interest Loan 19%12%-37%

21. Conclusions Getting comprehensive weatherization helps clients to pay their utility bills. Getting comprehensive weatherization helps clients to avoid short-term high interest loans that lead to more trouble in the long run. 21

22. Weatherization Program Emissions Impacts for a Representative State 22 Emissions Impacts First YearLifetime CO2 Equivalent $54$1,010 NOX $24$432 PM2.5 $2.50$44 SO2 $44$684 VOC <$1$8 TOTAL $125$2,178

23. Weatherization “Take Back” 23

24. “Take Back” Issue “Weatherizing the homes of low-income households doesn’t save energy. After weatherization clients just turn up the thermostat and don’t have any reduction in energy use” 24

25. Client Comfort 25 PreWXPostWXDifference Change in Thermostat Home is Cold/Very Cold 39%20%-49%No change Home is Comfortable 58%78%+34%No change

26. Client Comfort 26 Home is… PreWXPostWXDifference Drafty all of the time 13%4%-70% Drafty most of the time 16%5%-68%

27. Determinants of Comfort 27 Group Average Temperature Percent of Homes Drafty PreWX Cold/Very Cold 69 degrees50% PreWX Comfortable 70 degrees16%

28. Client Comfort 28 Home is… PreWXPostWXDifference Drafty all of the time 13%4%-70% Drafty most of the time 16%5%-68%

29. WAP Occupant Survey Shows… Client perceptions of cold vs. comfortable mainly related to draftiness, not to thermostat settings. PostWX comfort levels high without higher thermostat settings. But … these are just survey data …. 29

31. WAP Field Study Shows… Temperatures go up in some homes. Temperatures go down in other homes. Measured indoor temperatures preWX vs. PostWX do not change. 31

32. Case 1: -28% savings

33. 2348-0038 (ID) 442

34. Ceiling insulation Floor insulation Air leakage reduction from 2,000 to 1,500 CFM50 Heating system replacement 7012--0118 (MO) 636

35. Ceiling insulation (4 bags) Floor insulation (2 bags) Air leakage reduction from 2,000 to 1,500 CFM50 (measured 2,100 at time of visit) Heating system replacement (gas heater broke year before Wx) …and client reported keeping thermostat set higher after Wx (for health reasons)

36. Conclusions Some clients “take back”. Once their home is weatherized they can heat their homes to a safe and comfortable level. Some clients “give back.” Once their homes is weatherized they can lower the thermostat setting and still be comfortable. 36

37. Summary 37

38. Data is used for… Benchmarks… –Understanding who clients are, what issues they faced, and what benefits they received. Evaluation and Measurement… –Evaluation to identify high impact services/strategies AND on- going measurement to track progress toward goals. Partnership –Field staff report on client needs and services, management tracks progress against goals, policymakers pay attention and furnish more $$$, AND available $$$ can go farther and have more impact!! 38

39. Contact 39 David Carroll, 609-252-8010 david-carroll@appriseinc.org APPRISE 32 Nassau Street, Suite 200 Princeton, NJ 08540