Analysis Division
October 2007

How Effective are Compliance Reviews?

The on-site compliance review (CR) is perhaps the single greatest resource-consuming activity of the Federal Motor Carrier Safety Administration (FMCSA). Thousands of CRs are conducted each year. In fiscal year 2005, Federal and State enforcement personnel conducted over 11,000 CRs on individual motor carriers. FMCSA expects that through education, heightened safety regulation awareness, and the enforcement effects of the CR, motor carriers will improve the safety of their commercial vehicle operations, and, ultimately, reduce the number and severity of crashes in which they are involved.

FMCSA, in cooperation with the Volpe National Transportation Systems Center, has developed an analytic model to measure the effectiveness of the CR in terms of crashes avoided, injuries avoided, and lives saved. This tool provides FMCSA management with the information it needs to address the requirements of the Government Performance and Results Act (GPRA) of 1993, which obligates Federal agencies to measure the effectiveness of their programs as part of the budget cycle process. It also provides FMCSA and State safety program managers with a quantitative basis for optimizing the allocation of field safety resources. This analytic tool is known as the CR Effectiveness Model. The model has been implemented for compliance reviews conducted in FY 2005.

The CR Effectiveness Model shows the direct impact of compliance reviews on motor carrier safety, but not the "deterrent" effects (i.e., the effect on a carrier's behavior due to the potential of having a CR). The model is based entirely on "before and after" changes in the safety performance of motor carriers that received CRs. The model compares a motor carrier's crash rate in the 12-month period after a CR to its crash rate in the 12-month period prior to that review. To make this comparison, the model uses (1) crash data reported by the states and (2) power unit data reported by carriers or obtained during CRs.

Benefits of CRs Conducted in FY 2005

The table below shows the number of compliance reviews conducted in FY 2005, along with the number that were included in the implementation of the model. Some compliance reviews are removed from the model because the motor carrier receiving the compliance review was not active 12 months after the CR, had zero power units, or had crash and power unit data that did not pass edit checks designed to screen out erroneous data.

The model estimated that the average crash rate of motor carriers receving CRs decreased by 16.3 percent for compliance reviews conducted in FY 2005. As a result of this crash rate reduction, the model produced the following estimates for the immediate 12 months following the FY 2005 CRs:

What else can the Model tell us?

Certain motor carriers respond more to CRs (i.e., lower their crash rates more) than other motor carriers do. With Analysis Division assistance, FMCSA managers can use the model to determine which motor carriers do or do not improve after receiving CRs, and the extent of the improvement of those that do improve. For example, the results of the implementation of the model can be further analyzed by size of motor carrier, as measured by number of power units. These results show that small motor carriers reduce their crash rates more than large motor carriers.

The results of this analysis will reveal the types of motor carriers that will most likely respond positively to CRs. By focusing on motor carriers that are likely to respond positively to CRs, the effectiveness of the compliance review program may be improved. Alternative (to the CR) treatment approaches may be suggested for motor carriers that are at risk, but will most likely not respond positively to CRs.

Please contact Mr. Richard Gruberg, of FMCSA's Analysis Division, at 202-366-2959 or richard.gruberg@dot.gov if you have questions or comments, or would like a copy of the final report.