fmcsa analysis divisiondata quality program august 2005 federal motor carrier safety administration...
DESCRIPTION
FMCSA ANALYSIS DIVISION Data Quality Program A web-based system for filing data quality concerns about Federal and State data What is DataQs?TRANSCRIPT
![Page 1: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/1.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
August 2005
Federal Motor Carrier Safety Administration
What Programs are Available to the States to Improve Data Quality?
![Page 2: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/2.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
Safety DataImprovement
Funding
MonthlyMonitoring
of StateData Quality
Analysis andImprovementof State PARs
On-siteState Data
Quality Reviews
DataQs
Training(FMCSA, State
& Local Officials)
Ongoing Evaluationof State-reported
Motor Carrier Crashand Inspection Data
UMTRIOff-site Analysisof State Crash
Data
FMCSAData Quality
Improvement Program
FMCSA Data Quality Programs
![Page 3: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/3.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
A web-based system for filing data quality concerns about Federal and State data
What is DataQs?
![Page 4: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/4.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
► It is:• A single location for data
challenge entry and response
• A web-based interface to view the status of data challenges and supporting documentation throughout the challenge process
• A support tool that assists FMCSA and States in generating responses
► It is not:• A change in either the
State or FMCSA data correction guidelines
• A system that allows or requires FMCSA to intervene in the State data correction process
What is DataQs?
![Page 5: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/5.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
►Allows users to submit data quality challenges related to motor carrier safety data
►Routes the challenges to the appropriate organization for resolution based on the type of challenge
What Does DataQs Do?
![Page 6: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/6.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
►Allows all involved in a data quality challenge to view the challenge and its status at any time
►Stores all information related to the challenge in a centralized place, e.g., correspondence between the user that submitted the challenge and the organization that is assigned the challenge, documentation pertaining to the challenge, status of the challenge, etc.
What Does DataQs Do?
![Page 7: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/7.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
►Motor Carriers and the General Public can use DataQs to submit data quality challenges related to motor carrier safety data
►FMCSA HQ, Field, and State personnel use DataQs to enter, respond to, track and manage the challenges
Who Can Use DataQs?
![Page 8: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/8.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
DataQs Activity
![Page 9: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/9.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
Total Challenges 6,913
Status as of 7/19/05
DataQs Challenge Type Breakdown
![Page 10: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/10.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
Status as of 7/19/05
Total Challenges 3,721
DataQs Inspection Challenge Type Breakdown
![Page 11: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/11.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
Status as of 7/19/05
Total Challenges 2,828
DataQs Crash Challenge Type Breakdown
![Page 12: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/12.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
What Makes For A Successful Challenge?►Challenge Type
• Crash Data – Assigned to Wrong Carrier, Duplicate, Not Reportable
• Inspection Data – Assigned to Wrong Carrier►Providing Supporting Documents
• Crash challenges improves from 65% to 86%• Inspection challenges: 68% to 80%• Carrier information: 35% to 87%
►State Differences• Some States have very high rate of corrective actions
taken compared to others (TN, WY, NM, FL, OH, VA)
![Page 13: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/13.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
What Makes For An Unsuccessful Challenge?►Challenge Type
• Crash – Not Preventable• Inspection – Violation Data (Citation vs. Conviction)• Inspection – Duplicate• Enforcement Data (Compliance Review, Safety Audits,
Enforcement Actions)►State Differences
• Some States have a lower rate of corrective actions taken compared to others (KS, WA, KY, CT, SC, TX, WI, OH)
![Page 14: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/14.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
Safety DataImprovement
Funding
MonthlyMonitoring
of StateData Quality
Analysis andImprovementof State PARs
On-siteState Data
Quality Reviews
DataQs
Training(FMCSA, State
& Local Officials)
Ongoing Evaluationof State-reported
Motor Carrier Crashand Inspection Data
UMTRIOff-site Analysisof State Crash
Data
FMCSAData Quality
Improvement Program
FMCSA Data Quality Programs
![Page 15: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/15.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
►A program that assists FMCSA and States to:• Monitor State-reported CMV crash and inspection
data quality on a monthly basis• Measure progress towards the quarterly State data
quality rating• Analyze current and historical MCMIS and DataQs
management reports• Provide current status and historical trends of CMV
data quality
What is Monthly Monitoring of State Data Quality?
![Page 16: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/16.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
► State-reported data• Fatal and Non-fatal Commercial Motor Vehicle Crash
data• Roadside Inspection data
► Resources• Motor Carrier Management Information System
(MCMIS) Crash and Inspection files • National Highway Traffic Safety (NHTSA) Fatality
Analysis Reporting System (FARS) Crash files• MCMIS online reports• DataQs management reports
What Data & Resources are used for Monthly Monitoring?
![Page 17: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/17.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
►State Safety Data Quality (SSDQ) Analysis• Compares the quality of State-reported crash and
inspection data to FMCSA standards• Uses the same evaluation methodology used by
FMCSA to rate States’ data quality on a quarterly basis• Consists of 5 measures and one overriding indicator• Results in an overall rating of “Good”, “Fair”, or “Poor”
What Analysis is Conducted in the Monthly Monitoring Process?
![Page 18: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/18.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
►Review of MCMIS Online Reports• Review 8 reports
Crash & Inspection Timeliness, Accuracy, and Duplicate analysis
• Identify Discrepancies►Review DataQs Management Reports
• Indication of persistent problems in data collection or correction
• Review data challenges entered and open over certain time limits
What Analysis is Conducted in the Monthly Monitoring Process?
![Page 19: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/19.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
►Two types of reports are prepared• Headquarters Progress Report• State Progress Report
►Reports will be posted online• Results available on A&I Online’s Data Quality Module (
http://ai.fmcsa.dot.gov)• Site will be protected with UAS password (UAS
password available through FMCSA Technical Support)
How are Monthly Monitoring Results Presented?
![Page 20: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/20.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
Results provide historic view of data quality
State Crash Timeliness Historical Report
How are Monthly Monitoring Results Presented?
![Page 21: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/21.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
Safety DataImprovement
Funding
MonthlyMonitoring
of StateData Quality
Analysis andImprovementof State PARs
On-siteState Data
Quality Reviews
DataQs
Training(FMCSA, State
& Local Officials)
Ongoing Evaluationof State-reported
Motor Carrier Crashand Inspection Data
UMTRIOff-site Analysisof State Crash
Data
FMCSAData Quality
Improvement Program
FMCSA Data Quality Programs
![Page 22: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/22.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
What is the Program?►Provides an analysis of the State’s data reporting to the FMCSA Motor Carrier Management Information System Crash file ► Similar evaluation method for each State► State police accident report (PAR) file is evaluated for all reportable crashes► Comparison of the PAR case file to the MCMIS Crash file
UMTRI Analysis of State Crash Data
![Page 23: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/23.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
Outcomes► Determine the nature and extent of underreporting► Determine patterns of MCMIS Crash File under- or over-reporting ► Determine sources of under- or over-reporting List of Reports► California, Florida, Illinois, Michigan, Missouri, New Mexico, North Carolina, Ohio
UMTRI Analysis of State Crash Data
![Page 24: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/24.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
Safety DataImprovement
Funding
MonthlyMonitoring
of StateData Quality
Analysis andImprovementof State PARs
On-siteState Data
Quality Reviews
DataQs
Training(FMCSA, State
& Local Officials)
Ongoing Evaluationof State-reported
Motor Carrier Crashand Inspection Data
UMTRIOff-site Analysisof State Crash
Data
FMCSAData Quality
Improvement Program
FMCSA Data Quality Programs
![Page 25: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/25.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
►FMCSA team goes on-site to a State to:• Review a State’s crash and roadside inspection
collection and reporting processes • Works with the States to identify areas for improving
the data quality, and establishing methods for monitoring the effectiveness of the improvements.
What is the FMCSA State Data Quality Review Program?
![Page 26: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/26.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
1. State selection/coordination for the on-site review
2. Pre-analysis of State crash and inspection data3. An on-site visit consisting of
interviews/observations of State data collection process
4. Prepare recommendations/strategies for improving State’s data quality
5. Follow-up and monitoring effectiveness of improvements
What does the FMCSA State Data Quality Review Program Involve?
![Page 27: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/27.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
Data Quality Review Team – FMCSA HQ, Volpe Center staff, NISR, and Data Nexus
• Crash data expertise• Roadside inspection data expertise• System/Technical expertise• Collection and Process expertise
Who does the FMCSA State Data Quality Review Team Consist of?
![Page 28: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/28.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
►Conducted Pilot in State of NH – (June ‘05)► In FY ’06 - plan 1 State Data Quality Review
every quarter
What is the status of the FMCSA State Data Quality Review Program?
![Page 29: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/29.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
Safety DataImprovement
Funding
MonthlyMonitoring
of StateData Quality
Analysis andImprovementof State PARs
On-siteState Data
Quality Reviews
DataQs
Training(FMCSA, State
& Local Officials)
Ongoing Evaluationof State-reported
Motor Carrier Crashand Inspection Data
UMTRIOff-site Analysisof State Crash
Data
FMCSAData Quality
Improvement Program
FMCSA Data Quality Programs
![Page 30: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/30.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
What is the Program?►CVARS – Commercial Vehicle Analysis Reporting System (grants and cooperative agreements)
►Provides funding via grants and cooperative agreements to State agencies to help them improve the collection and reporting of eligible large truck and bus safety data
Safety Data Improvement Funding
![Page 31: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/31.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
Assistance
► Federal funding available up to 100%► No State match is required► Existing projects are also eligible for assistance► Expert staff available to help States identify data
quality challenges and design workable solutions
Safety Data Improvement Funding
![Page 32: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/32.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
Projects Initiated by States
► Hire staff to manage data quality improvement programs► Revise outdated crash report forms► Hire staff to code and enter crash data► Train law enforcement officers in collecting crash data► Develop software to transfer data from the State
repository to SAFETYNET► Purchase equipment to support data quality activities► Purchase software for field data collection and data
transfer
Safety Data Improvement Funding
![Page 33: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/33.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
Grants and cooperative agreements awarded in 2002 - 2005 totaled nearly $ 18,400,000
Safety Data Improvement Funding
![Page 34: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/34.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
Contact
►Betsy Benkowski• FMCSA• (202) 366-5387• [email protected]
Safety Data Improvement Funding
![Page 35: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/35.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
Program ►Presents data quality analysis and reports►Located in A&I Online website (http://ai.fmcsa.dot.gov)►Provides quarterly evaluations of State-reported data
in MCMIS►Provides an electronic means for filing concerns about
Federal and State data►Access tools for developing State strategic plans
(e.g., CVSPs, State safety plan, etc.)
A&I Data Quality Module
![Page 36: FMCSA ANALYSIS DIVISIONData Quality Program August 2005 Federal Motor Carrier Safety Administration What Programs are Available to the States to Improve](https://reader035.vdocuments.us/reader035/viewer/2022062905/5a4d1ad07f8b9ab059971024/html5/thumbnails/36.jpg)
FMCSA ANALYSIS DIVISION Data Quality Program
A&I Data Quality Module