1. Eastern Research Group Cindy Palacios Tim DeFries Sandeep Kishan Jim Lindner IM Solutions Training Forum Sacramento May 20-23, 2012

2. Objective & Approach Attempt to quantify OBD emission reductions How? By comparing average emission levels of initial OBD pass vehicles to average emissions of initial OBD fail vehicles With what? Paired Colorado OBD/IM240 data OBD Advisory only (after April 2003) Pre-April 2003 OBD was Pass/Fail

3. Three Step Method Step 1- Use pre-April 2003 to validate the use of initial OBD pass inspection as reasonable surrogate for final passed OBD inspection of OBD Fail/Repair/Pass vehicles Step 2- Quantify IM240 emissions difference between OBD pass vehicles and OBD fail vehicles Step 3- Use differences to project emissions reductions in OBD-Only fleets

4. Step 1: Validate Use of OBD IP Emissions as Surrogate for OBD FRP Emissions Use pre-April 2003 data where MIL-on vehicles had to be repaired and return for inspection. OBD Initial Pass (OBD IP) OBD Fail/Repair/Pass (OBD FRP) Ideally same vehicle Before & After OBD/IM240 tests Too many subsets after April 2003 Using only OBD initial tests as flag removes tests influence Use OBD IP emission levels as OBD FRP level for post-April 2003 data This will underestimate the repair delta for OBD Fail/Pass vehicles

5. IM240 HC Emissions: OBD IP and OBD FRP Vehicles

6. IM240 CO Emissions: OBD IP and OBD FRP Vehicles

7. IM240 NOx Emissions: OBD IP and OBD FRP Vehicles

8. Observations OBD FRP emissions lower than OBD IP emissions This difference can be thought of as emissions “creep” OBD IP vehicles have a range of deterioration where emissions are slowly increasing or creeping toward triggering an OBD or IM240 failure. Use of initial OBD pass emissions as surrogate for FRP will therefore underestimate the impact of OBD-induced repairs on IM240 emissions

9. Step 1: Observations (cont) HC and CO trends different from NOx HC & CO display a substantial difference between OBD FRP and OBD IP emissions and a small positive slope NOx has very little difference between OBD FRP and OBD IP and a larger positive slope Possible causes: repair effectiveness, cutpoints, component deterioration specific to NOx emissions

10. Step 2: Quantify IM240 Emissions Differences for OBD Pass and OBD Fail Use all data Compare IM240 emissions of Initial OBD Pass inspections to Initial OBD Fail inspections Only OBD result used as Pass/Fail criteria Stratify by model year and age at time of inspection Look at overall OBD result, as well as individual DTCs and groups of related DTCs.

11. Mean Initial IM240 HC Emissions by OBD Result

12. Observations Mean HC “pass” emissions for 4-year-old MY03 are lower than the mean HC “pass” emissions for a 4-year-old MY98. Mean emissions for vehicles either passing or failing OBD are generally both very low and far below IM240 cutpoints. Mean OBD failing emissions are consistently higher than OBD passing vehicles. At 95% confidence interval there were significant emissions reductions in the range of 30-40% for each MY and vehicle age group Where there was not a significant emission reduction at the 95% confidence interval it was where the sample sizes were less than 100 failing vehicles Similar analyses were done for CO and NOx.

13. Step 3: Estimating OBD-only Fleet Emissions Can this data be used to estimate OBD fleet emissions? Spreadsheet model (SSM) Input number of OBD passing and failing vehicles stratified by calendar year, MY and vehicle type Look up Colorado IM240 emissions for a given OBD result as f(calendar year, MY, vehicle type) Very rough as it does not include DTC categories Multiplies the counts by the emissions Sum the total emissions for each calendar year and then normalize by the number of vehicles Average IM240 emissions by MY for each calendar year. Final output is an estimate of the initial emissions of the OBD fleet, an estimate of the final emissions of the OBD fleet after repairs, and a percentage change or reduction in fleet emissions This is for a given test cycle No VMT input The spreadsheet model output was compared to MOVES runs for the Arizona program in 2007

14. Spreadsheet Model Layout: Inputs Vehicle Years/Ages Vehicle Counts by OBD Result (Initial/Final) Passenger CarsLight Trucks Model Year Calendar Year Vehicle AgePassFail/ PassFail/FailPassFail/ PassFail/Fail 19961999318,8192181115,5111608 19971999217,117106615,5841035 19981999113,67119110,072231 1999 07,3141016,251101 19962000430,4034242227,01644123 19972000323,1172321218,68318110 19982000215,14479416,286824 19992000114,43732210,878201 2000 07,7771217,062171 19962001521,1974562417,45442322 19972001434,3714122230,10453728 19982001321,716157820,3821568 19992001214,27065315,488593 20002001115,35528110,375221 2001 08,4372016,701151 19962002623,3695572920,50958931 19972002522,2774422318,39945624 19982002429,7033041631,58730316 19992002319,522127719,4021056 20002002213,44957313,261312 20012002112,4664728,395281

15. Spreadsheet Model Layout: Graphical Results

16. Spreadsheet Model Layout: Program Summary Results Annual Projected Results on a Per- Vehicle Basis: Calendar Year No. Vehicles HC Initial (g/mi) 95% Conf. Final (g/mi)95% Conf. Change (%) 95% Conf. 1999105,0200.1590.00120.1580.001-0.50.01 2000172,4020.1460.00110.1450.001-0.80.01 2001218,3220.1700.00100.1690.001-0.70.01 2002247,4600.1920.00120.1910.001-0.90.01 2003274,7670.2000.00100.1970.001-1.30.01 2004318,2900.1870.00080.1830.001-2.00.01 2005280,5910.1940.00100.1890.001-2.70.01 2006336,7750.2090.00100.2030.001-3.00.01 2007323,9830.1860.00110.1790.001-3.80.01

17. Spreadsheet Model Results SSM allows user to compare projected emissions reductions resulting from OBD program, over different calendar years Could also use SSM to compare one area to another Current SSM reductions ~8-10% lower than MOVES MOVES compares IM to no-IM scenario SSM compares one year of IM to next year of IM This is common problem with IM Program Evaluation

18. Spreadsheet Model Results (cont) Direct comparison of MOVES output to SSM output would have to be done across an IM cycle or cycles to avoid the No IM baseline issue Differentiate between annual and biennial program effects MOVES incorporates VMT while the SSM does not Incorporate DTC categories Issue with assigning emission reductions with multiple DTCs ERG has used the Repair Slate concept in previous studies and that approach could be applied here Step-change analysis using existing data Austin Before IM program & After IM program RSD data OBD-only & TSI