1. The FMCSA Program Effectiveness Study How Effective Are Compliance Reviews?

2. Why Measure Effectiveness? To address the requirements of the Government Performance and Results Act (GPRA) of 1993, which obligates federal agencies to measure the effectiveness of their programs as part of the budget cycle process To give FMCSA a means to measure how much of an impact the Compliance Review program activities have on avoiding crashes involving motor carriers and reducing the numbers of resulting injuries and fatalities

3. What Is a Compliance Review? An on-site examination of a motor carrier’s records of operation (including driver log books, maintenance and inspection records, driver qualification records, controlled substance and alcohol testing data,, financial responsibility information, crash data, hazardous materials documentation, etc.) to determine whether a motor carrier meets a safety fitness standard. FMCSA expects that through the Compliance Review (CR) process, carriers will improve the safety of their commercial vehicle operations, and, ultimately, reduce their crash rates.

4. Question: How does one measure the effectiveness of Compliance Reviews? Answer: Examine the crash history of carriers both prior to, and after the CR process.

5. What Is the Compliance Review Impact Assessment Model? An analytic model designed to measure the effectiveness of the Compliance Review in terms of: –Crash Reduction = Crashes Avoided –Injury Reduction = Injuries Avoided –Fatality Reduction = Lives Saves

6. CR Impact Assessment Model Overview The CR Impact Assessment Model –Shows the direct impact of compliance reviews on carrier safety –It is based on “before and after” changes in the safety performance of carriers that received CRs –It compares a motor carrier’s crash rate in a time period after an on-site compliance review to its crash rate prior to that review

7. CR Impact Assessment Model Data Collection The model uses crash and mileage data collected during compliance reviews, and later stored in the FMCSA’s Motor Carrier Management Information System (MCMIS) During the CR, investigators obtain the number of recordable crashes in which the carrier was involved over the past 12 months, and the number of vehicle miles traveled (VMT) From these data, a “pre-Compliance Review” crash rate is calculated for all carriers receiving reviews.

8. CR Impact Assessment Model CR Follow-up Survey Data To perform a “before and after” comparison, post-Compliance Review data are obtained from a Follow-up Survey. The Follow-up Survey collects motor carrier mileage and crash information for the 1-year period after the review. The percentage reduction in the recordable crash rate is then calculated for all carriers.

9. CR Impact Assessment Model Effectiveness Calculations The number of crashes avoided the following calculation is performed: Pre-CR average crash rate x crash rate reduction x VMT Historical data are then used to breakdown the total crashes avoided into the following categories: fatal crashes avoided, injury crashes avoided, and towaway avoided crashes. These numbers are then translated into injuries avoided and lives saved, also based on historical data.

10. How Effective Were the Compliance Reviews Conducted in 2000?

11. How Effective Were the Compliance Reviews Conducted in 2001?

12. Historical Effectiveness of Compliance Reviews Based on CRs conducted in:1998199920002001 Number of Carriers Receiving CRs 6,055 8,87711,340 8,924 Total Vehicle Miles Traveled (VMT) (million miles) 13,84417,40922,61018,455 Pre-CR Average Crash Rate (crashes/million VMT).823.804.757.715 Model Results Estimated for:1999200020012002 Total Number of Crashes Avoided1,2001,5002,2001,600 Number of Towaway Crashes Avoided 677 8461,241 902 Number of Injury Crashes Avoided 480 600 880 690 Number of Fatal Crashed Avoided 43 54 79 58 Number of Injuries Avoided 8221,0281,3951,105 Number of Lives Saved 51 64 91 67

13. Where Can I Find More Information on the FMCSA Program Effectiveness Study? The FMCSA website, A&I Online (www.ai.volpe.dot.gov), presents information on the CR Impact Assessment Model. Access or download: Detailed Reports One Page Summaries of the Model Model Results Overview Presentations (such as this presentation) Or contact Mr. Richard Gruberg, FMCSA Analysis Division, at 202-366-2959 or richard.gruberg@fmcsa.dot.gov for copies of the reports and related information about the Program Effectiveness Study

14. Where Can I Find Program Effectiveness on A&I Online? The A&I Program Measures Module includes Program Effectiveness as a sub-module