1. Overview: Energy Efficiency Cost Effectiveness

2. Total Resource Cost (TRC) Test
Cost effectiveness tests are defined in the California Standard Practice Manual and updates have been made in CPUC decisions.
Each test measures cost effectiveness through a different perspective.
TRC is a ratio of benefits / costs
Ratio of 1.0 means that for every dollar spent, an equal amount of benefit is captured.
D , Decision Addressing Energy Efficiency Business Plans outlines how EE portfolios are gauged by the California Public Utilities Commission.
Forecasted Portfolio TRC of 1.25 (1.00 during ramp years of )
Evaluated Portfolio TRC of 1.00

3. Why TRC Matters
Energy Efficiency funding comes directly from the ratepayers.
A cost effective portfolio is required by regulatory statute to ensure that Energy Efficiency programs are spending ratepayer dollars effectively and efficiently.
Cost effectiveness calculations are performed on both a prospective and retrospective basis.
TRC calculations used as an assessment of both budgets and actual expenditures.
Annual Budget Advice Letter (ABAL) -> Annual Reporting (ex-ante) -> Ex-Post Evaluation

4. TRC Benefits = Avoided Costs
Avoided costs of generation, transmission and distribution, energy, ancillary services, renewable portfolio standard (RPS), greenhouse gases (GHG)
Electric benefit values are calculated in the Cost Effectiveness Tool (CET) based on an electric end use (load) shape, climate zone, and sector permutation based on amount of kWh.
Gas benefit values are calculated in the CET based on gas sector and gas savings profile permutation based on amount of therms.
Both electric and gas savings are calculated using the lifecycle net savings.

5. TRC Benefit Trend
The proliferation of solar and the decrease in price of natural gas has caused a shift in avoided costs.
Decrease in magnitude.
Later in the day.
Later in the year.
Mid-day savings decreasing in value and higher evening avoided costs.
Avoided costs were updated in 2018 and again in 2019.
CZ4 shown; ftp://ftp.cpuc.ca.gov/gopher-data/energy_division/EnergyEfficiency/CostEffectiveness/ACC_2018_v1f.xlsb

6. Incentive to Customers Incentive to Customers
TRC Costs Formula
TRC Benefits
TRC Costs
Measure Costs
NonIncentive Costs
Incentive to Customers
Participant Cost
Incentive to Customers
Participant Cost
IOU/3rd Party Costs
Non-Free Riders (NTG)
Free Riders (1-NTG)

7. TRC Sensitivity
Portfolio TRC is most sensitive to measure costs and kWh/therm savings.
kW does not impact the calculation of TRC.

8. CEDARS Website: https://cedars.sound-data.com/
CPUC’s reporting system that all program administrators (PAs) currently use for reporting of monthly, quarterly, and annual data submissions.
Calculations are now done on the cloud and will no longer require a database to be downloaded/installed.
Starting with 2019 avoided costs, the CET on CEDARS is the only approved calculator.

9. Appendix: Algebraic Formula for TRC

10. Cost Effectiveness Tool (CET) Overview and Demo

11. How to Access the CET
Step 1: Register for an account on CEDARS: data.com/
Under Affiliation, select “Community”
CEDARS will send you an to verify your information; click on the link
Step 2: Once registered, login here: data.com/accounts/login/
Step 3: Click on the Cost Effectiveness Tool (CET) tab
Please register under the Community affiliation. Must register in order to run CET calculations. Do not register under a specific PA as that is used by the reporting teams for uploading of budget and data.

12. CET Guide and Specifications
User Guide: contains basic details on the data rules, how to use the CET, and how to prepare your inputs for submission.
Specifications: contains all data rules and requirements for CET input data (i.e., acceptable values).
CEI Value Lists provide allowable values for various fields
CEI Source of Truth files provides the data specification, validation, and warning rules
We will reference the CEI value lists in CEDARS for the demo portion, so keep this window open.
Talking Points: Once logged into CEDARS, you will have access to the CET. Click on User Guide. The user Guide explains the data parameters and how to prepare your files for submission. The Specification contains the cost effective inputs (CEI) value lists, i.e., valid values for specific fields in the measure input file. For example under building type, you find the description that meets your program, say commercial, and use the corresponding “code” as the value for that field (i.e. building type) in the measure input file. It must match exactly. You can also download this data and save to your desktop but for now we are going to use the data directly in CEDARS.

13. Using .csv Files in Windows
Workaround for .csv files in Windows
In Windows, go to your control panel
Open “Region and Language”
Could also be under Date and Time
Open “Additional settings”
Set list separator to “|”
Click “OK”
.csv files (opened with excel) that are | delimited should now break into individual columns
Do not use commas anywhere on the .csv file as the comma will create a delimiter in Access.

14. CET Input Files There are two CET Input CSV Files:
ProgramCost.csv: includes program level non-incentive costs (i.e. admin, marketing, and direct implementation non-incentive)
Measure.csv: includes measure level savings, cost data and specifications
ProgramCost and Measure files have to be filled out completely
Files must be in pipe delimited
i.e. separated using |
Both files must be zipped together to run in CET
CET numerical values cannot exceed 8 decimal places
DO NOT:
Add or rename columns to the inputs files
Change the name(s) of the input files
Change the file type; must remain as *.csv
Two input files, ProgramCost and Measure. These files must be filled out completely. The measure file has a couple of fields that are optional, and do not impact the CE calculation. When uploaded the CET will reference the two files, so any overlapping fields need to align and match (like PA, PrgID, ClaimYearQuarter) Also the files need to be in | delimited.

15. ProgramCost File Overview
Populate costs ($) only if applicable; otherwise enter 0
AdminCostsOverheadAndGA (column D): non-incentive admin costs
MarketingOutreach (column F): all marketing costs
DIActivity (column G): direct implementation non-incentive program costs
UserInputIncentive (column M): is for populating lump sum incentives at the program level
Measure level incentives go in the Measure file
PA must be populated with valid PA
Blanks will cause for errors, use 0 for all blanks

16. Measure Input File Overview
User Provided Values
Values found in CET CEI Specification (CEDARS)
Values found in Workpapers (Deemed/DI); Engineering Assumptions/Modeling (Custom)
CEInputID
BldgType
EUL_Yrs
ClaimYearQuarter
DeliveryType
MeasCode
InstallationRatekW
E3ClimateZone
MeasDescription
InstallationRatekWh
E3GasSavProfile
MeasureID
InstallationRateTherm
E3GasSector
NTGRCost
NumUnits
E3MeaElecEndUseShape
NTGRkW
PrgID
E3TargetSector
NTGRkWh
RealizationRatekW
EUL_ID
NTGRTherm
RealizationRatekWh
GSIA_ID
PreDesc
RealizationRateTherm
MeasAppType
RUL_ID
Residential_Flag
MeasImpactType
RUL_Yrs
SourceDesc
NormUnit
StdDesc
UnitEndUserRebate
NTG_ID
UnitDirectInstallLab
UnitIncentiveToOthers PA
UnitDirectInstallMat
Upstream_Flag
Sector
UnitkW1stBaseline
TechGroup
UnitkW2ndBaseline
TechType
UnitkWh1stBaseline
UseCategory
UnitkWh2ndBaseline
UseSubCategory
UnitMeaCost1stBaseline
Version
UnitMeaCost2ndBaseline
UnitTherm1stBaseline
UnitTherm2ndBaseline
Unlike the ProgramCost table, Measure can have blanks in certain fields

17. Description/definition
Measure Input File Overview
CE Input Fields
Description/definition
CEInputID
Unique and persistent filing identifier
ClaimYearQuarter
The quarter when the claim is being made (eg. '2019Q1')
InstallationRatekW
kW weighted installation rate for all measures included in the associated claim (Custom=1)
InstallationRatekWh
kWh weighted installation rate for all measures included in the associated claim (Custom=1)
InstallationRateTherm
Therm weighted installation rate for all measures included in the associated claim (Custom=1)
NumUnits
Number of units or quantity associated with this claim/measure/project (eg. 206)
PrgID
Links Claim to Program table (eg. PGE2001)
RealizationRatekW
kW weighted Realization rate for all measures included in this claim record (Deemed/DI=1; Custom=0.9)
RealizationRatekWh
kWh weighted Realization rate for all measures included in this claim record (Deemed/DI=1; Custom=0.9)
RealizationRateTherm
Therm weighted Realization rate for all measures included in this claim record (Deemed/DI=1; Custom=0.9)
Residential_Flag
Flag to identify Residential sites (True= 1; False= 0)
SourceDesc
Description of the specific source for the data; The workpaper or DEER ID including revision (eg. PGE3PHVC149r0)
UnitEndUserRebate
Actual cost in dollars of rebates to the end user
UnitIncentiveToOthers
Actual cost in dollars of incentives to others besides the end user
Upstream_Flag
Flag to identify upstream claims (True= 1; False= 0)

18. Description/definition
Measure Input File Overview
CE Input Fields
Description/definition
BldgType
Standard ExAnte Building Type (AKA Sub-Sector)
DeliveryType
Standard ExAnte method used to deliver the measure to the customer (eg. Direct Install)
E3ClimateZone
Climate zone where EE measure is installed
E3GasSavProfile
E3 Gas savings profile
E3GasSector
E3 Gas sector
E3MeaElecEndUseShape
E3 measure electric end use shape (i.e. load shape)
E3TargetSector
E3 target sector
EUL_ID
Specifies a row in EUL table that specifies the estimated useful life of the measure technology
GSIA_ID
Reference to the Ex Ante Gross Savings and Installation Adjustment table
MeasAppType
Standard ExAnte Measure application type (eg. ROB)
MeasImpactType
Measure Impact Type
NormUnit
Type of normalizing units, or basis of NumUnits or quantity used by the CE calculation (eg. 'Lamp')
NTG_ID
Claimed ExAnte NTG_ID, if different than the default NTG_ID associated with the implementation PA
The Program Administrator code (eg. PGE)
Sector
Standard ExAnte sector (eg. Res)
TechGroup
Standard ExAnte Technology Group
TechType
ExAnte Standard Technology Type used to categorize measure (eg. Lighting)
UseCategory
Standard ExAnte end use category
UseSubCategory
Standard ExAnte subcategory describing the end use of the measure (eg. Vending)
Version
The version of the Workpaper or DEER (eg Ex-Ante)

19. Measure Input File Overview
CE Input Fields
Description/definition
EUL_Yrs
Effective useful life of EE measure in years
MeasCode
PA-specific Measure code
MeasDescription
Description of measure; include enough information to allow understanding of what EE measure(s) being installed
MeasureID
Unique and persistent measure identifier
NTGRCost
The cost net to gross ratio
NTGRkW
The kW net to gross ratio
NTGRkWh
The kWh net to gross ratio
NTGRTherm
The therm net to gross ratio
PreDesc
Pre-exsiting measure description
RUL_ID
Specifies a row in ExAnte EUL table that identifies the remaining useful life of the existing technology
RUL_Yrs
Remaining useful life of pre-existing measure in years
StdDesc
Used to describe the technologies associated with the measure with enough detail to identify the various technologies and differentiate the measure from other similar measures
UnitDirectInstallLab
Actual cost in dollars of installation labor; overrides ExAnte values for deemed claims
UnitDirectInstallMat
Actual cost in dollars of installation materials; overrides ExAnte values for deemed claims
UnitkW1stBaseline
The first baseline kW Impact per Unit
UnitkW2ndBaseline
The second baseline kW Impact per Unit
UnitkWh1stBaseline
The first baseline kWh Impact per Unit
UnitkWh2ndBaseline
The second baseline kWh Impact per Unit
UnitMeaCost1stBaseline
Total measure cost for dual baseline, incremental measure cost over Code/Std for single baseline
UnitMeaCost2ndBaseline
Incremental measure cost over Code/Std for dual baseline
UnitTherm1stBaseline
The first baseline therm Impact per Unit
UnitTherm2ndBaseline
The second baseline therm Impact per Unit

20. Example Use Cases
In this portion of the training we will develop input files for the following three scenarios
Example 1: Residential Deemed Program
Example 2: Commercial Custom Program
Example 3: NMEC Program (still in draft)

21. CET Input Files
Create a new folder on your desktop “CET_Training_Jan2019”
Download the zipped file“CET_input_files_Jan2019_training” that has been ed to you.
Please raise your hand if you have not received the files via .
Extract all the files and save to the CET_Training_Jan2019 folder on your desktop.
Three sub folders for each example (Example 1, Example 2 and Example 3).
Also included is a “Completed” folder in each example that has a finished input.
Avoided Cost Combo Look-up table can be downloaded off CEDARS
Cost Effectiveness Tool > Specifications > Avoided Cost Combo > Download This Data
Included in the zip file ed
Invalid avoided cost combinations will cause for an error that will prevent running of cost effectiveness

22. Example 1: Residential Deemed
Develop input for a residential lighting program being funded by SCE
Assumptions:
Measure: LT-19048
Measure Name: LED PAR20: ≤ 11 Watts
Target is 200,000 bulbs
Single Family Measure
Available in SCE territory, climate zone 9

23. Example 1: Residential Deemed ProgramCost.csv
Open the ProgramCost file and enter in the following assumptions:
PrgID = Example 1
PrgYear = 2019
ClaimYearQuarter= 2019Q4
Note the PrgYear and ClaimYear should match
AdminCostsOverheadAndGA = 1000 (example)
MarketingOutreach = 500 (example)
DIActivity = (example)
PA = SCE
For the remaining columns = 0
Click Save and exit
Note: Do not add any formatting or spaces to data
We use Q4 because it is the most conservative approach for forecasting. Using earlier quarters does have a slight impact on TRC.
In the RFP, the amount of program costs should be justified.

24. Example 1: Residential Deemed Measure.csv
Open the Measure file and the Workpaper folder > Calculation template > SFm
Will be using the Impact and Cost tabs
On both tabs, filter for solution code: LT and climate zone 9 (BldgLoc)
Refer to Specification on CEDARS for allowable choices.
Enter:
CEInputID: 1 (or anything you want, but must be unique)
PrgID: Example 1 (must match ProgramCost)
ClaimYearQuarter: 2019Q4 (must match ProgramCost)
Sector: Res
Delivery Type: DnDeemed
Building Type: SFm
E3ClimateZone: 9 (format should be without CZ)
Sometimes climate zone will be labeled CZ09, but should be entered as 9 for CET. Climate zones are specific to IOUs. Workpapers are IOU specific and RFP submittals should use the corresponding workpaper.

25. Example 1: Residential Deemed Measure.csv
Enter (cont’d):
E3GasSavProfile: Annual (default)
E3GasSector: Residential
E3MeaElecEndUseShape (LoadShape): DEER:Indoor_CFL_Ltg
E3TargetSector (E3_BT): RES
MeasAppType: ROB
MeasCode (SolutionCode): LT-19048
MeasDescription (MeasIndex): LED PAR20: =11 Watts
≤ changed to = because CET is unable to accept non-basic symbols
MeasImpactType: IOU-Deemed
MeasureID: LT-19048
TechGroup: Ltg_Lamp
Cannot use “,” in measure description.

26. Example 1: Residential Deemed Measure.csv
Enter (cont’d):
TechType: LED_lamp
UseCategory: (blank)
UseSubCategory: InGen
PreDesc: (blank)
StdDesc: (blank)
SourceDesc (WP#): SCE17LG127.2
Version: (blank)
NormUnit: lamp
NumUnits:
UnitkW1stBaseline:
UnitkWh1stBaseline:
Blanks do not mean that data does not exist, but rather that the CET does not enforce the blank.

27. Example 1: Residential Deemed Measure.csv
Enter (cont’d):
UnitTherm1stBaseline:
UnitkW2ndBaseline: 0
UnitkWh2ndBaseline: 0
UnitTherm2ndBaseline: 0
UnitMeaCost1stBaseline: 3.46
UnitMeaCost2ndBaseline: 0
UnitDirectInstallLab (only use if direct install measure): 0
UnitDirectInstallMat (only use if direct install measure): 0
UnitEndUserRebate: 2.25
UnitIncentiveToOthers: 0
NTG_ID: All-Ltg-ScrwInLED
Highlighted inputs are incentives. Should generally be lower than measure cost.

28. Example 1: Residential Deemed Measure.csv
Enter (cont’d):
NTGRkW: 0.91
NTGRkWh: 0.91
NTGRTherm: 0.91
NTGRCost: 0.91
EUL_ID: ILtg-Res-LED-20000hr
EUL: 16
RUL_ID: (blank)
RUL_Yrs: 0
GSIA_ID: (blank)
RealizationRatekW/kWh/Therm: 1
InstallationRatekW/kWh/Therm: 1
Deemed measures use a 1.0 for realization rate and installation rate from WP. Custom measures use a 0.9 realization rate and a 1.0 installation rate.

29. Example 1: Residential Deemed Measure.csv
Enter (cont’d):
Residential_Flag: 1
Upstream_Flag: 0
PA: SCE
MarketEffectsBenefits: 0.05
MarketEffectsCosts: 0.05
RateScheduleElec/RateScheduleGas/CombustionType: (blank)
MeasInflation: 0
Comments: (blank)
Now ready to run the CET file.
Zip the ProgramCost.csv and Measure.csv files into a single zipped folder.
Can be named anything you’d like.
Flags: 1 for yes, 0 for no

30. Running the CET
In CEDARS, click the Cost Effectiveness Tool (CET) tab at the top.
Click the orange 'Run CET' button at the top of the CET page, and a window will pop up.
Select the appropriate First Year and Avoided Costs versions
First Year = 2019
Avoided Costs = 2019
Select the CET input .zip file you prepared on your local machine
Click the 'Run CET' button; your job will be queued for processing
The window will close and your submission will appear as the most recent job in your CET upload history.
CEDARS only generates cost effectiveness results once all QC is passed.
You will also receive an confirming the submission and the status.

31. Example 2: Commercial Custom
Develop input for a commercial custom program being funded by PG&E
Assumptions:
Measure: CHC11
Measure Name: HVAC Retrofit/New-Chillers-Air Cooled-Efficient Unit
Target is 114,750 First Yr Net kWh and 15 First Yr Net kW
Available in PG&E territory, climate zone 4

32. Example 2: Commercial Custom ProgramCost.csv
Open the ProgramCost file and enter in the following assumptions:
PrgID = Example 2
PrgYear = 2019
ClaimYearQuarter= 2019Q4
Note the PrgYear and ClaimYear should match
AdminCostsOverheadAndGA = 500 (example)
MarketingOutreach = 1000 (example)
DIActivity = (example)
PA = PGE
For the remaining columns = 0
Click Save and exit
Note: Do not add any formatting or spaces to data
We use Q4 because it is the most conservative approach for forecasting. Using earlier quarters does have a slight impact on TRC.
In the RFP, the amount of program costs should be justified.

33. Example 2: Commercial Custom Measure.csv
Fields that Will Differ from Deemed Example:
NormUnit: kWh
NumUnits:
UnitkW1stBaseline: = 20/150000
UnitkWh1stBaseline: 1
UnitMeaCost1stBaseline: 0.16
UnitEndUserRebate: 0.08
Equals to $0.08/kWh
If customer incentive also for kW, then must back into that amount
If kWh savings change, must recalculate unit kW
Realization Rates: 0.9
Installation Rates: 1.0

34. Example 2: Commercial Custom Measure.csv
Alternative Way of Modeling:
NormUnit: Each
NumUnits: 1
UnitkW1stBaseline: 20
UnitkWh1stBaseline:
UnitMeaCost1stBaseline: 24000
UnitEndUserRebate: 12000
Equals to $0.08/kWh but calculated at the total
Extra caution in future revision, if savings change, incentive must be recalculated
Realization Rates: 0.9
Installation Rates: 1.0
Avoid splitting the kWh and kW into separate rows unless you are prepared to combine for cost effectiveness analysis.

35. Example 3: NMEC (still in draft pending additional CPUC Guidance)
Under Technical Guidance > “Weighted Average Expected Useful Life/Net to Gross Method”
Spreadsheet to calculate weighted average EUL
Also calculated average NTG on the sheet, but is overridden by E-4952
Resolution E-4952 (p. A-45) set default NTG for NMEC measures at:
Residential: 0.85
Multifamily: 0.55
Commercial: 0.95
Missing guidance to populate avoided cost combo for NMEC measures.
Will be forthcoming in RFP

36. CET Results The submitted file will appear in your Upload History
Most recent upload summary appears in the lower half of the page
The job Summary includes:
Table record counts (number of measures/claims) uploaded and returned by the CET (if any).
Portfolio Filing Summary with sector level summaries of CE results, savings, and budget.
Program(s) summary CE results, savings, and budget.
QC feedback (Errors/Warnings).

37. CET Results Under Upload History, you can download the CET output.
If QC fails, no CET Output will be generated.
Download QC Feedback to see what row(s) caused the error.
QC is at the measure level, using the CEInputID as the key.
Filename will match whatever the .zip file name.
Clicking on Summary in the rightmost column changes the Summary section in CEDARS (below the Upload History)
Placeholder for running through the results. It is hard for me to put content here without an actual example. I will develop an example next week.

38. CET Output Files
When the CET Output is downloaded, there will be 4 files within the zip folder.
Zip folder will be numbered based on number of CET files run.
Input, Output, ProgramCost, and Validation are the files within.
Input and ProgramCost files are the measure and program cost data.
The outputs contain the cost effectiveness results.
Results are provided for each claim/CEInputID, at the individual measure level.
The summary page in the CET has the Program and Portfolio level results.
The Validation file contains the error and warning messages.
Program TRC is calculated by (ElecBen + GasBen) / TRCCost = (Column Y + Column Z) / (Column AC)
When calculating program TRC, take care to not use ElecBenGross or TRCCostNoAdmin as those are different calculations.

39. CET Common Errors
Errors will prevent CET from calculating cost effectiveness.
Invalid Format for files – check that files in .zip folder are named ProgramCost.csv and Measure.csv
CEInputID values aren’t unique for Measure.csv file – check that CEInputID is not duplicated
Mismatched PrgIDs between Measure table and ProgramCost table – check that both files use the exact same PrgID
Values (‘e3_climate_zone’, ‘e3_mea_elec_end_use_shape’, ‘e3_target_sector’, ‘upload_PA’) are not allowed for AvoidedCostCombo – check that climate zone, end use shape, and target sector exist exactly on the avoided cost combo list.
Requirement_Missing_X – parameter necessary for CET calculation is missing, update specific row and parameter
Invalid Format – check that column names match exactly
Value Not Valid – Check that the format for the specific value does not include $ or , or –
Problem with RulYrs: RUL_Yrs should be less than EUL_Yrs – Check that RUL < EUL.
When calculating program TRC, take care to not use ElecBenGross or TRCCostNoAdmin as those are different calculations.

40. CET Common Warnings
Warnings will still allow CET to run, but should be checked.
Warning about RULID: When RUL_Yrs is greater than 0, RUL_ID should not be null, but is empty – Input RUL_ID into data or check that RUL_Yrs should be populated
Warning about RUL_Yrs: RUL_Yrs should be less than or equal to EUL_Yrs /2.75 – Check that RUL is 1/3 of EUL
Warning about UnitMeaCost1stBaseline: UnitMeaCost1stBaseline should be within the interval [0, ) – Check that Measure Cost is not negative.
When calculating program TRC, take care to not use ElecBenGross or TRCCostNoAdmin as those are different calculations.

41. Cost Effectiveness Resources
Energy Efficiency Policy Manual: Overview of EE policy requirements and term definitions
Energy Efficiency Standard Practice Manual: Overview of Cost Effectiveness Calculations
CPUC Energy Efficiency Website: CPUC resources for Energy Efficiency
Placeholder for list of common errors and solutions.