SafeStat

Safety Status

Measurement System

(Version 8.6)

What is SafeStat?

• SafeStat was developed at the Volpe Center for the Federal Motor Carrier Safety Administration (FMCSA).

• Data used are maintained and managed at the Federal level by the FMCSA.

Uses of SafeStat

·        FMCSA Compliance Reviews (CRs):

o       Semiannually identifies and prioritizes carriers for on-site FMCSA compliance reviews.

·         PRISM Program: 

o       Identifies and monitors poorly performing carriers for the PRISM Federal/State safety improvement process (MCSIP).

·         Inspection Selection System (ISS):

o       Supports recommendation of evaluated carriers’ drivers and vehicles for roadside inspections.

·         A&I Online (http://www.ai.volpe.dot.gov/)

o       Makes SafeStat results available via the Internet to industry and the public to promote safety awareness and self-improvement.

SafeStat Methodology

• Involves analytically assessing a motor carrier in four Safety Evaluation Areas (SEAs):
  • Accident SEA
  • Driver SEA
  • Vehicle SEA
  • Safety Management SEA

• Each SEA is based on two or more indicators supported by different data sources.

SafeStat Design

• Event and exposure data are used to calculate normalized measures for each carrier
• Measures are ranked and assigned percentile values (Indicators) from 0-100 - with 100 being highest or worst
• Indicators are combined into SEA values
• SEA values are weighted and summed to derive the SafeStat score.

SafeStat Hierarchy and Features

• The SafeStat algorithm design is computationally hierarchical beginning with safety event data building to a SafeStat Score.
• Algorithm features data sufficiency tests, normalization and weighting.

Description of Hierarchy and Features Pyramid:

Base Level:     Carrier Descriptive Data

Level Two:     Safety Event Data

Level Three:     Safety Measures

Level Four:     Indicators

Level Five:     SEA Values

Top Level:     SafeStat Score

SafeStat Overview Pyramid:

Base Level:               Data Sources by Carrier

·         State Reported & FMCSA Collected Truck Crashes

·         Moving Violations

·         Out-of-Service Violations from Roadside Inspections

·         Violations from onsite Compliance Reviews

·         Closed Enforcement Cases            

      Level Two:             Measures       

      Level Three:          Indicators

      Level Four:            Accident SEA, Driver SEA, Vehicle SEA,

                                    Safety Management SEA

      Top Level:             Motor Carrier SafeStat Score

SafeStat Data

At the foundation of SafeStat are safety event data and exposure data.

• Carrier specific safety event data reflect the carrier’s safety compliance and performance and include:
  • State Reported Crashes (last 30 mos.)
  • Recordable Crashes from CRs (last 12 mos. CRs)
  • Roadside Inspection Violations (last 30 mos.)
  • Moving Volations (last 30 mos.)
  • Compliance Review Violations (last 18 mos. CRs)
  • Closed Enforcement Cases (last 6 years)
• Exposure data normalize a carrier’s safety event data:
  • Number of drivers/vehicles and VMT
  • Number of inspections

SafeStat Measures

• Safety measures are the result of normalizing safety event data.
• Example:  accident event data are converted to accident rates, which take into account differences in exposure.

SafeStat Indicators

• Indicators rank carriers by their safety measures converted to a percentile (0-100) scale.
• When determining indicators, SafeStat may employ peer groupings that reflect differences in operations to assure appropriate comparisons among carrier types and size classes.

SafeStat SEA Values

• Safety Evaluation Areas (SEAs) represent the four major criteria used to evaluate carriers’ safety status.
• A SEA Value, also on a 0-100, percentile scale, is derived from the Indicators related to that SEA.

• For example a SEA Value of 85 means:

o        85% of the carriers (that have sufficient data) have a better safety status in that SEA

o       15% have a worse safety status

SafeStat SEAs

·         Accident

·         Driver

·         Vehicle

·         Safety Management

Accident SEA Summary

Base Level:                            

• State-Reported Crash Data and Census Data
• Compliance Review Data

Level Two:                              

• Accident Involvement Measure (AIM)
• Recordable Accident Rate (RAR)

Level Three:

• Accident Involvement Indicator (AII)
• Recordable Accident Indicator (RAI)

Top Level:  Accident SEA

Accident SEA

Data & Measures:

• State reported crashes (using NGA standard) over past 30 months normalized by # of power units from FMCSA carrier registration data yield the Accident Involvement Measure (AIM).

• Recordable crashes from Compliance Reviews (CRs) conducted during the previous 12 months normalized by vehicle miles traveled (VMT) yield the Recordable Accident Rate (RAR) Measure.

Accident SEA Accident Involvement Indicator (AII)

Base Level:                             State-Reported Crash Data and Census Data

Level Two:                               Accident Involvement Measure (AIM)

Level Three:                            Accident Involvement Indicator (AII)

Top Level:                               Accident SEA

• Accident Involvement Measure (AIM) applies time weighting (most recent crashes have greatest weight) and severity weighting (crashes involving injury/fatality and/or HM release have more weight).
• AIM is calculated by dividing weighted-crashes by average # of power units.

• AII is percentile ranking of weighted AIM.
• Carriers are peer-grouped by similar # of crashes and ranked on a percentile basis to obtain an AII.

Accident SEA - Recordable Accident Indicator (RAI)

Base Level:                             Compliance Review Data

Level Two:                               Recordable Accident Rate (RAR)

Level Three:                            Recordable Accident Indicator (RAI)

Top Level:                               Accident SEA

• # of Recordable Crashes and last 12 months VMT data are gathered during CR.
• Recordable crashes are divided by VMT and multiplied by 1 million to obtain a Recordable Accident Rate (RAR) per million miles traveled.
• Carriers are peer-grouped by similar # of crashes and then ranked on a percentile basis to obtain the RAI.

Accident SEA Value Calculation

• For Carriers with no CR within past 12 months:
  • Accident SEA = AII
• Carriers with CR within past 12 months and no state-reported crashes since the CR
  • Accident SEA = RAI
• Carriers with CR within past 12 months and at least one state-reported crash since the Compliance Review:
  • Accident SEA = highest of (AII, RAI)

Driver SEA Summary

 Base Level:

• Driver Roadside Inspections
• Compliance Review Data
• Moving Violation Data from Roadside Inspections

Level Two:

• Driver Inspections Measure (DIM)
• Driver Review Measure (DRM)
• Moving Violations Measure (MVM)

Level Three:

• Driver Inspections Indicator (DII)
• Driver Review Indicator (DRI)
• Moving Violations Indicator (MVI)

Top Level:  Driver SEA

Driver SEA

Data and Measures:

• Driver OOS Violations normalized by number of roadside inspections over past 30 months yield the Driver Inspection Measure (DIM).
• Driver-Related Critical and Acute Violations from CRs completed within past 18 months yield the Driver Review Measure (DRM).
• Moving Violations over past 30 months normalized by number of drivers yield the Moving Violation Measure (MVM).

Driver SEA Driver Inspection Indicator (DII)

• DIM is based on OOS inspections that are severity-weighted (based on # of DOOS viol.) and time-weighted:
  • 0-6 months old # of inspections (3x)
  • 7-18 months old # or inspections (2x)
  • 19-30 months old # of inspections (1x)
• DIM is adjusted up for violations of OOS orders (aka jumping OOS orders)
• Carriers are peer-grouped by similar # of driver inspections
• DII is percentile ranking of DIM

Driver SEA - Driver Review Indicator (DRI)

• Driver Review Measure (DRM) is based on violations of driver-related Critical and Acute Regulations from CRs.

• DRM accounts for the number and severity of violations.

• DRI is percentile ranking of DRM.
• Carriers with CR and no violations are given a DRI of 0.

Driver SEA - Moving Violations Indicator (MVI)

Base Level:       Moving Violation Data from Roadside Inspections
Level Two:         Moving Violations Measure (MVM)
Level Three:      Moving Violations Indicator (MVI)
Top Level:         Driver SEA

• MVM is based on Moving Violations (MVs) issued in conjunction with roadside inspections normalized by the number of drivers.
  • Time weighting is applied to violations:
    • 0-6 months old # of MVs (3x).
    • 7-18 months old # or MVs (2x).
    • 19-30 months old # of MVs (1x).
• Carriers are peer-grouped by similar # of MVs.
• MVI is percentile ranking of MVM.

Driver SEA Value Calculation

• Driver SEA Value is the highest of the DII and DRI and uses the MVI when its value is greater than the DII and DRI.
• When the MVI is greater than the DII and DRI, Driver SEA is equal to the weighted average of MVI and the highest of the DII and DRI.

Vehicle SEA Summary

Base Level:

• Vehicle Roadside Inspections
• Compliance Review Data

Level Two:

• Vehicle Inspections Measure (VIM)
• Vehicle Review Measure (VRM)

Level Three:

• Vehicle Inspections Indicator (VII)
• Vehicle Review Indicator (VRI)

Top Level:  Vehicle SEA

Vehicle SEA

Data and Measures:

• Vehicle OOS Violations normalized by number of roadside inspections over past 30 months yield the Vehicle Inspection Measure (VIM)

• Vehicle-related Critical and Acute Violations from CRs completed within past 18 months yield the Vehicle Review Measure (VRM)

Vehicle SEA - Indicators VII & VRI

• VII is similar to DII but uses Vehicle OOS violations instead of Driver OOS violations.
  
  Hierarchy
  Base Level: Vehicle Roadside Inspections
  Level Two: Vehicle Inspections Measure (VIM)
  Level Three: Vehicle Inspections Indicator (VII)
  Top Level: Vehicle SEA
  
• VRI is similar to DRI but uses Vehicle-related violations of Acute/Critical regulations.
  
  Hierarchy
  Base Level: Compliance Review Data
  Level Two: Vehicle Review Measure (VRM)
  Level Three: Vehicle Review Indicator (VRI)
  Top Level: Vehicle SEA
  

Vehicle SEA Value Calculation

Level Three:

• Vehicle Inspections Indicator (VII)
• Vehicle Review Indicator (VRI)

Top Level:  Vehicle SEA Value

• Vehicle SEA Value is the highest of the VII and VRI.

Safety Management SEA Summary

 Base Level:

• Closed Enforcement Cases
• Compliance Review

Level Two:

• Enforcement Severity Measure (ESM)
• Safety Management Review Measure (SMRM)
• HM Review Measure (HMRM)

Level Three:

• Enforcement History Indicator (EHI)
• Safety Management Review Indicator (SMRI)
• HM Review Indicator (HMRI)

Top Level:  Safety Management SEA

Safety Management SEA

Data and Measures:

• Safety Management/HM Critical and Acute Violations from CRs completed within past 18 months yield the Safety Mgmt Review Measure (SMRM) and the HM Review Measure (HMRM).
• SMRM and HMRM account for the number and severity of violations.
• Closed Enforcement Cases are used to determine the Enforcement Severity Measure (ESM).

Safety Mgmt SEA - Indicators SMRI & HMRI