Slide 1: FMCSA State Safety Data Quality Enhancing the Evaluation August 2006 Slide 2: Presentation Overview State Safety Data Quality Enhancements - Where are we now? - How can we improve? - Where are we headed? Slide 3: Where are we now? Date Driven - Through our partnership in the MCSAP the FMCSA shares a common safety goal with the States. - FMCSA and State partners share in recognition that quality data is essential - Uniform, complete, accurate, and timely information needs to be collected and reported to FMCSA on crashes and roadside inspections. Slide 4: Where are we now? Data Quality Efforts Parts of the FMCSA Data Quality Improvement Program 1) DataQs 2) Ongoing Evaluation of State Safety Data Quality 3) Monthly Monitoring of State Data Quality 4) UMTRI Off-site Analysis of State Crash Data 5) On-site State Data Quality Reviews 6) Training - Analysis and Improvement of State PARs 7) Safety Data Improvement Funding Slide 5: Where are we now? Data Quality Efforts Parts of the FMCSA Data Quality Improvement Program 1) DataQs 2) Ongoing Evaluation of State Safety Data Quality (highlighted box) 3) Monthly Monitoring of State Data Quality 4) UMTRI Off-site Analysis of State Crash Data 5) On-site State Data Quality Reviews 6) Training - Analysis and Improvement of State PARs 7) Safety Data Improvement Funding Slide 6: Where are we now? State Safety Data Quality (SSDQ) - Objective - To evaluate the completeness, timeliness, accuracy and consistency of State-reported crash and inspection data in the Motor Carrier Management Information System. (MCMIS) - Methodology - Consists of 5 performance measures and one overriding indicator - Results in a scorecard for each State Slide 7: SSDQ Measures and Indicator Data Quality Efforts Parts of the Overall Rating (highlighted box) 1) Fatal Crash Completeness 2) Crash Timeliness 3) Crash Accuracy 4) Crash Reporting Consistency Indicator (flag graphic) 5) Inspection Timeliness 6) Inspection Accuracy Slide 8: Where are we now? SSDQ Measures Overall State Ratings Slide 9: Where are we now? SSDQ Enhancements to date March 2006: Inclusion of "Change" records into Timeliness analysis March 2004: SSDQ released to public December 2004: Inclusion of HM Intrastate records into Accuracy analyses June 2006: Exclusion of "carrier non identifiable" crash records from Accuracy analyses Slide 10: Where are we now? June 2006 SSDQ results - June 23, 2006 (graphic of U.S. map with color coded state ratings) Slide 11: Where are we now? SSDQ results: 27 Month comparison Overall State Ratings March 2004 24 States had a "Good" rating 13 States had a "Fair" rating 14 States had a "Poor" rating June 2006 32 States had a "Good" rating 12 States had a "Fair" rating 7 States had a "Poor" rating Slide 12: How can we improve? 2005 GAO Data Quality Report -GAO Recommendations related to SSDQ - Develop an Action Plan to implement enhancements to the SSDQ map and methodology - Provide a crash-specific data quality rating separate from the Overall State rating - Provide information on the website on the limiations of the map and the underlying ratings Slide 13: How can we improve? Potential Enhancements to SSDQ - GAO's evaluation highlighted areas for improvement in the SSDQ methodolgy - FMCSA is addressing these items as well as developing potential new measures Slide 14: Where are we headed? Potential New Crash Measures - Crash Reporting Completeness - Crash Record Completeness - Crash Record Accuracy Slide 15: Where are we headed? Crash Reporting Completeness Non-Fatal Crash Completeness Measures - Evaluates the completeness of reporting non-fatal crash records - Fatal to non-fatal crash records in MCMIS compared to the expected number of fatal to non- fatal crash records Slide 16: Where are we headed? Crash Record Completeness Driver Identification Record Completeness - Evaluates whether a crash record has driver identification fields filled out completely - Driver Identification fields 1) Driver license number 2) Driver date-of-birth 3) Driver First Name 4) Driver Last Name 5) License class Slide 17: Where are we headed? Crash Record Completeness Vehicle Identification Record Completeness - Evaluates whether a crash record has truck and or bus identification fields filled out completely -Vehicle Identification fields - vehicle identification number - license plate number - vehicle configuration - cargo body type - GVWR Slide 18: Where are we headed? Crash Record Accuracy Hazmat Information Accuracy - Evaluates the integrity/agreement of the hazmat information reported within a crash record - Hazardous Material fields - HM Placard - HM Class Number - HM Release - HM 4 digit number Slide 19: Where are we headed? Crash Record Accuracy Vehicle Information Accuracy - Evaluates the integrity/agreement of the vehicle information reported within a crash record - Vehicle Information fields - Vehicle configuration - Cargo body type Slide 20: Where are we headed? Next steps in Measure Development - Perform analyses to determine the existing quality of data reported by the States in the areas of interest. - Develop potential criteria for 'good', 'fair', and 'poor' performance - Provide opportunity for States to give feedback on new measures - Integrate new measures into SSDQ methodology Slide 21: Where are we headed? Development of SSDQ Crash Rating Parts of Crash Overall Rating -Crash Driver Identification Completeness -Crash Vehicle Identification Completeness -Crash Hazmat Accuracy -Crash Vehicle Information Accuracy -Non Fatal Crash Completeness -Fatal Crash Completeness (faded into background) -Crash Timeliness (faded into background) -Crash Accuracy (faded into background) -Crash Reporting Consistency Indicator(faded into background) Slide 22: In conclusion We've made a lot of progress... - SSDQ methodology shows improvement in the quality of data reported to MCMIS There are opportunities for improvement - Develop additional measures for a more complete evaluation of the quality of data reported by the states. Slide 23: For more information or if you are interested in providing input..... http://ai.fmcsa/dot/gov/DataQuality/dataquality.asp Contact info: Betsy Benkowski, FMCSA Analysis Division 202-366-5387