Title Slide: The Evolution of FMCSA’s Data Quality Program - July 2007 Slide 2:Evolution of the Data Quality Program Yesterday - Collection, Data Driven Decisions, Dissemination Today - Evaluation, Improvement, Data Correction Tomorrow - Increased Data Use, Improved Evaluation, Improvement Slide 3: YESTERDAY (1980s and 1990s) 1. Safety Data Collection 2. Data Driven Decisions 3. Dissemination Slide 4:TODAY (2000–today) Data Quality Program Evaluation and Monitoring, Data Improvement, Data Correction Slide 5:TODAY Data Quality Program DQ Evaluation and Monitoring - State Safety Data Quality (SSDQ) - Monthly Progress Report (MPR) - State Crash Data Analysis DQ Improvement - Analysis and Improvement of State Police Accident Reporting (PAR) / Training - State Reviews - Technical Assistance - Funding Data Correction - DataQs Online Tool - MCMIS/FARS Matching Tool Slide 6: (Photo of Jack Nicholson with a "serious face on") Jack was hired as a State Safety Program Manager. One of his performance objectives is to improve data quality. Slide 7: (Photo of Jack Nicholson with a "serious face on") How can Jack evaluate how his state is doing in terms of Data Quality? Slide 8: State Safety Data Quality (SSDQ map), Monthly Progress report (sample graph), State Crash data analysis (report cover) Slide 9: What is available to help Jack improve his state’s data quality? Slide 10: Training, State Reviews Slide 11: Technical Assistance, Grants Slide 12: What can Jack do to correct his state’s safety data quality problems? Slide 13: DataQs website, FARS/MCMIS Matching Tool Slide 14: Jack wants to know if the Data Quality Program has been helpful to states? Slide 15: 38 States achieved a Good Overall State Rating (June 2007) 24,000+ data challenges filed (67% resulting in updates) 7 State Reviews were conducted since 2005 Slide 16: Jack wants to know how tomorrow’s FMCSA data quality program will be enhanced? Slide 17: Tomorrow Improvements DQ Evaluation and Monitoring - Enhance State Safety Data Quality Evaluation - Continue State Crash Data Analysis DQ Improvement - Conduct more State Reviews - Expand Technical Assistance - Continue PAR Analysis and Training Slide 18: Current SSDQ Map, June 2007 Overall Rating: 38 States Good, 10 States Fair, 3 States Poor. Slide 19: State Safety Data Quality Evaluation New SSDQ Crash Measures - Crash Record Completeness - Driver Identification - Vehicle Identification - Non-fatal Crash Completeness Slide 20: New SSDQ Crash Measures Crash Record Completeness Evaluates the completeness of driver and vehicle identification data on reported crash records Driver Identification - Driver License # - Driver Date-of-Birth - Driver First Name - Driver Last Name - License Class Vehicle Identification - VIN - License plate # - Vehicle configuration - Cargo body type - Gross Vehicle Weight Rating Slide 21: New SSDQ Crash Measures Crash Record Completeness Measure Driver ID Measure --> Vehicle ID Measure --> Crash Completeness Measure Rating Criteria Crash Record Completeness Good: % of complete driver and vehicle data reported is >= 85% Fair: % of complete driver and vehicle data reported is 70-84% Poor: % of complete driver and vehicle data reported is < 70% Slide 22: New SSDQ Crash Measures Non-Fatal Crash Completeness Measure % Non-Fatal Crash Completeness = # Reported Non-Fatals # Predicted Non-Fatals Rating Criteria Non-Fatal Crash Completeness Rating Good: % of Non-Fatal crash records reported is >= 75%. Fair: % of Non-Fatal crash records reported is 50-74%. Poor: % of Non-Fatal crash records reported is < 50%. Insufficient Data: Average # of FARS crash records < 15 AND % on Non-Fatal crash records reported is <50%. Slide 23: Enhanced SSDQ Methodology Existing Measures and Overriding Indicator Crash Timeliness --> Overall State Rating Crash Accuracy --> Overall State Rating Fatal Crash Completeness --> Overall State Rating Inspection Accuracy --> Overall State Rating Inspection Timeliness --> Overall State Rating Crash Consistency (Overriding Indicator) --> Overall State Rating New Measures Non-Fatal Crash Completeness --> Overall State Rating Driver ID/Vehicle ID-->Crash Record Completeness --> Overall State Rating Slide 24: Enhanced SSDQ Map, June 2007 17 States, Good --> Minimum of 1 Good and 0 Poor 23 States, Fair --> Maximum of 1 Poor 11 States, Poor --> 2+ Poor or Red Flagged Slide 25: Impact of New Measures Example: State of Mississippi June 2006 1 Good measure 4 Poor measures Red Flagged June 2007 4 Good measures 1 Fair measure June 2007, New Methodology 4 Good measures 2 Fair measures 1 Poor measure Slide 26: Implementation Incremental Approach Post New Measure Analyses on A&I Online website Integrate new measures into SSDQ methodology first quarter 2008 Re-evaluate rating criteria of the new measures and new measure weights relative to the overall map rating during 2008 Slide 27: FARS/MCMIS Fatal Crash Record Matching Tool A tool designed to help reconcile differences between the number of fatal crash records in the FARS and MCMIS databases. >>"match" fatal large truck and bus crash records >>records must contain the same information in several key fields (e.g., county, date, time, VIN, DOT #, etc.). Slide 28: FARS/MCMIS Fatal Crash Record Matching Tool Individual Results - Step by Step approach to review results - Determine if crash records were categorized incorrectly in MCMIS - Identify missing or incomplete data fields - Take action to upload new information or records to MCMIS Slide 29: (Photo of Jack Nicholson, smiling) No need to worry, Jack! The FMCSA Data Quality Program will help. Slide 30: Analysis and Information (A&I) Online website: http://ai.fmcsa.dot.gov Data Quality Module Betsy Benkowski 202-366-5387 Dana Larkin 617-494-2821 Candy Brown 617-494-3856