The following four changes to the SafeStat methodology are proposed to improve the performance of the current SafeStat algorithm (Version 8.6). 

1. New Traffic Violations Indicator (TVI)

Proposed Improvement

Replace the Moving Violations Indicator (MVI) with the Traffic Violations Indicator (TVI) in the Driver Safety Evaluation Area (SEA) Value calculation.

The TVI differs from the MVI in four major respects:

• Applies a different measure of exposure: The TVI will be normalized by the average number of power units a carrier operates at the end of the three time periods used for time weighting in the TVI calculation (0-6 months, 7-18 months, and 19-24 months) rather than the current number of drivers a carrier employs.
• Expands use of traffic violation data: The TVI will expand the use of traffic violations in the Driver SEA by considering 392.2 (general/local laws) violations in addition to the twelve specified moving violations currently used in calculating the MVI. The 392.2 general violations will be given less weight than specified violations.
• Emphasizes driver on-road performance in the calculating Driver SEA Value and SafeStat Score: The TVI will be assigned the same weight as the DRI and DII. The highest of the TVI, DRI, and DII will be assigned to the Driver SEA Value. This change will allow the TVI to impact a carrier’s Driver SEA Value more than the MVI. This approach makes the treatment of the TVI in the Driver SEA consistent with the use of the other two indicators and with the treatment of indicators in the other SEAs of assigning the highest indicator to the SEA value.
• Compares performance of similarly sized carriers: Each carrier receiving a Traffic Violations Measure (TVM) will be assigned to one of four peer groups by average number of power units (i.e., Group 1: <=5, Group 2: >5 and <=50, Group 3: >50 and <=500, Group 4: >500). Each carrier’s TVM will be ranked against the TVMs of all of the other carriers within the same peer group and assigned a percentile from 0 (representing the lowest TVM) to 100 (representing the highest TVM). As with the other indicators, the percentile value determined by the rank of the measure (the TVM) will be the indicator value (TVI).

The methodology on how the TVI will be calculated can be found by click here.

Reason for Change

Issues such as SafeStat’s ability to identify high crash risk carriers, fairness in assessing a carrier’s safety status, data quality, algorithm consistency, and carrier coverage were examined in developing the TVI. Results of these analyses show that incorporating the TVI accomplishes the following:

• Identifies more carriers with higher crash risk: The SafeStat effectiveness study has been used to measure SafeStat’s ability to identify high crash risk carriers by observing the subsequent crash rates of SafeStat-identified (Category A & B) carriers.Click here to see the SafeStat Effectiveness study.  By applying the methodology of the SafeStat effectiveness study to the algorithm with the proposed TVI changes, the number of Category A & B carriers increased by 30% and their subsequent group crash rate rose 12% over the results of the current algorithm. These increases indicate that the incorporation of the TVI will improve SafeStat’s ability to find unsafe carriers.
• Improves geographical coverage: The inclusion of 392.2 (general/local laws) violations in the TVI increases the amount of traffic violations considered by SafeStat, particularly in states with relatively low reporting of the specified moving violations used in the MVI, thereby reducing the chances of geographical bias occurring.
• Simplifies and improves consistency of the SafeStat algorithm: The TVI uses the same measure of exposure, average power units, that is already used in calculating the Accident Involvement Indicator (AII). Also, assigning the highest driver indicator (i.e., DII, DRI, or TVI) to the Driver SEA Value makes the Driver SEA Value calculation consistent with the Vehicle and Safety Management SEA Value calculations.
• Uses Better Data: SafeStat requires an accurate measure of exposure to properly calculate the TVI. The data used to calculate the number of power units are more complete than the data used to calculate the number of drivers. Also, the use of average power units more accurately reflects changes in exposure over the period of time that carriers are being evaluated by the TVI.
• Covers more carriers: Twice as many carriers have sufficient data to receive a TVI than have sufficient data to receive an MVI.

2. Shortened Data Exposure Time Period

Proposed Improvement

Reduce the amount of time considered from 30 months of data to 24 months for:

• State-reported crash data when calculating the Accident Involvement Indicator (AII),
• Inspection data when calculating the Vehicle Inspection Indicator (VII) and Driver Inspection Indicator (DII),
• Traffic violation data when calculating the Traffic Violations Indicator (TVI), and
• Recently closed enforcement case data when calculating the Enforcement History Indicator (EHI).

Reason for Change

In 1996, a 30-month time frame of data was set for the various SafeStat indicators to obtain enough safety information to assess the safety status of a large portion of the carrier population. Since then, the amount of safety data available to SafeStat has increased and the timeliness of these data has substantially improved, thereby providing more safety data to assess the safety status of individual motor carriers. Adjusting the 30-month time frame to 24 months:

• Increases emphasis on more recent events: This change will reduce the instances in which a carrier has made recent substantive changes to it safety and compliance performance, yet the results from the older events prevent SafeStat from recognizing these changes quickly.
• Has little impact on the effectiveness of identifying carriers with high crash risk: The SafeStat effectiveness study has been used to measure SafeStat’s ability to identify high crash risk carriers by observing the subsequent crash rates of SafeStat-Scored (Categories A, B, and C) carriers, carriers with high SEA Values, and carriers with Indicators of 75+. By applying the methodology of the SafeStat effectiveness study to the algorithm with the proposed 24-month change, the effectiveness of identifying high-crash risk carriers does not change substantially at the Category, SEA, or Indicator levels.
• Reduces the number of carriers assessed by SafeStat:SafeStat requires that a carrier have a sufficient amount of data before its safety status can be evaluated. Reducing the length of time from 30 to 24 months will reduce the number of carriers with sufficient data to be evaluated by SafeStat by 10%. According to the SafeStat effectiveness study results, the group of carriers that no longer had sufficient data due to the reduction in time frame had a crash rate that was half the crash rate of the group of carriers that had sufficient data. Thus, while this change will cause many carriers to fall under the “SafeStat radar,” the effectiveness study shows that these carriers, as a whole, have a low crash risk.
• Aligns the SafeStat time frame with other FMCSA systems and reports that use two years of safety event data (e.g., SAFER, Motor Carrier Profile).

3. Simplifying the Calculation of the Accident SEA Value

Proposed Improvement

Simplify the calculation of the Accident SEA value to be equal to the Accident Involvement Indicator (AII).

Reason for Change

By assigning the Accident SEA value to the AII and no longer using the RAI will:

• Establish one standard of measuring crash rate, thereby avoiding potentially conflicting results of multiple standards.
• Provide a stable value to better monitor safety performance over time.

4. Applying More Vehicle Out-of-Service (OOS) Violation Data for the Vehicle Inspection Indicator (VII)

Proposed Improvement

Include the results of Level 3 (Driver only) inspections that resulted in vehicle OOS violations when calculating the VII.

Reason for Change

While conducting Level 3 inspections of the driver, enforcement officer may discover vehicle OOS violations. Although these vehicle OOS violations are currently not being used in the VII, which considers only the result of vehicle inspections (Levels 1, 2, and 5), they are indicative of unsafe vehicle conditions. Including the results of Level 3 inspections with vehicle OOS violations will improve SafeStat’s ability to identify carriers with vehicle safety problems.