Detailed Information on the
Federal Motor Carrier Safety Administration Grant Program Assessment

Create an annual performance evaluation process that includes setting goals and tracking progress for states, Divisions and Service Centers.

Develop and implement an action plan to enhance grant management oversight, to include new delegations of authority, conducting regular financial reviews and improving overall processes across all aspects of grant development.

Develop an efficiency measure that tracks improvements in program efficiency.

Measure: Highway fatality rate per 100 million vehicle miles traveled (VMT) (CY).

Explanation:

This performance measure is shared by multiple operating administrations within DOT (FMCSA, NHTSA, and FHWA). Preliminary data shows that the 2007 fatality rate is 1.37 which equates to 41, 059 lives lost from motor vehicle crashes. USDOT's original fatality goal began as a 1998 NHTSA and FHWA goal to reduce the number of transportation deaths by 20 percent, which equated to achieving an absolute number of 33,500 annual motor vehicle fatalities by 2008. The number was changed to an equivalent rate in 2002. The original goal was based on overly optimistic behavioral assumptions??a 90% seat belt usage rate and alcohol-related fatalities falling to 11,000 annually. Current figures indicate that the national seat belt usage rate is at 82% and that there are over 17,000 alcohol-related highway fatalities annually. Additionally, an unpredictable, and sustained spike in motorcycle rider fatalities began when the original goals were set??from a historic low of 2,116 in 1997 to 4,810 in 2006 (a 127% increase).

The Department remains committed to reducing highway fatalities and fully supports the goal of reducing fatalities to a rate of 1.0 per 100 million VMT. The Department has established four fatality sub-measures??passenger vehicles, nonoccupants, motorcycle riders, and large-truck- and bus-related fatalities??which represent the breadth of all highway users. This enables DOT to more closely examine the fatality rates of these different segments of highway users, in order to develop new strategies to target specific challenges to achieving the overall 1.0 goal. These four fatality sub-measures have been raised from agency specific goals to Departmental metrics to highlight the overall commitment by the Department.

Measure: Large Truck and Bus Fatalities per 100 million total Vehicles Miles Traveled (VMT)

Explanation:This is the primary FMCSA safety goal and one of four Department of Transportation level safety goals that contribute to lowering the over all fatality rate on the Nation's highways. Targets for this measure accounts for the fact that seventy five percent of large truck and bus crashes involve passenger vehicles as both types share a common operating environment. The targets now correlate statistically to the DOT 1.0 goal of less than 1.0 highway-related vehicle fatalities per 100 million VMT by the end of 2011.

Measure: Safety belt use by CMV operators

Explanation:Safety belt usage is an important part of ensuring the safety of large truck drivers and occupants. About half of large truck-occupant fatalities involve the vehicle rolling over. Safety belt use reduces the likelihood of death by 30 times in these crashes. In 2004, FMCSA began providing extra emphasis on large truck driver safety belt use after 48% of large truck drivers were found to be belted compared to about 80% of passenger vehicle occupants. Progress in this performance measure will contribute to the agency's overall safety goals. Starting in 2007 this data comes from NHTSA's National Occupant Protection use Survey (NOPUS). Due to the success of the program in 2007, FMCSA revised targets for 2008 and subsequent years.

Measure: Injury-Crashes involving Large Trucks & Buses per 100 million VMT

Explanation:One of FMCSA's priority performance measures is the reduction of injuries by reducing CMV crashes.

Measure: Fatal-Crashes involving Large Trucks & Buses per 100 million VMT.

Explanation:One of FMCSA's priority performance measures is the reduction of fatalities by reducing CMV crashes.

Measure: Total-Crashes involving Large Trucks & Buses per 100 million VMT.

Explanation:One of FMCSA's priority performance measures is the reduction of congestion on the nation's highways by reducing CMV crashes.

Measure: States meeting "green" on the State Safety Data Quality (SSDQ) Map

Explanation:FMCSA depends on the safety data collected by States from the activities conducted through MCSAP. This information is fed into a safety risk algorithm, called SafeStat, which is used to prioritize large truck and bus companies for roadside inspections and on-site safety compliance reviews. The data are used in effectiveness studies of FMCSA programs and in the regulatory development for future changes to the Federal Motor Carrier Safety Regulations. The DOT Office of Inspector General and the Government Accountability have identified issues with data quality. To address quality concerns and to improve transparency, FMCSA has implemented the State Safety Data Quality (SSDQ) Map. States must report safety data in a timely, accurate, and complete manner to achieve a "green" rating. Better data quality will enable FMCSA to more effectively target its resources and achieve greater gains in overall large truck and bus safety. SSDQ Map - http://ai.fmcsa.dot.gov/DataQuality/dataquality.asp * 2008 Figure from the February 2008 SSDQ Map Significant changes were made in September 2007 that included new requirements of the States in order to obtain a "green" rating. Initially the number of green States dropped from 38 in August 2007 to 21 in September, although analysis from earlier in the year projected that far more States would lose their Green rating.

Measure: Percentage of State-conducted new entrant safety audits performed "on-time" (within 18 months).

Explanation:On average between 45,000 and 50,000 new motor carriers come into interstate commerce each year. New entrant safety audits are to be done within the first 18 months that a motor carrier comes into business

Measure: Number of serious reportable HAZMAT Incidents involving commercial motor vehicles

Explanation:In 2000, FMCSA set a goal of a 20 percent improvement by 2010. New more aggressive targets were established beginning in 2009.

Measure: Lives saved efficiency index (in constant 2000 dollars ($K))

Explanation:Measures the efficiency by which we deliver our programs, along with the number of lives we save through our program. The efficiency measure demonstrates the resources we use to save lives on our nations highways and roads as a life-saved index.

Is the program purpose clear?

Explanation: The purpose of the Motor Carrier Safety Assistance Program (MCSAP) is to reduce commercial motor vehicle (CMV)-involved crashes, fatalities, and injuries through consistent, uniform and effective State-level CMV safety programs. MCSAP grants provide States with financial assistance to conduct roadside inspections, safety audits of new motor carriers, safety compliance reviews of high-risk motor carriers. In addition, the grants provide incentives for States to enact motor carrier safety laws that conform to Federal regulations. Program activities contribute significantly to the overall safety missions of the Federal Motor Carrier Safety Administration (FMCSA) and the Department of Transportation (DOT).

Evidence: Legislation: Surface Transportation Assistance Act of 1982 (STAA, P.L. 97-424), Section 4003 of the Transportation Equity Act for the 21st Century (TEA-21, P.L. 105-178 112 Stat. 395-398), Section 103 of the Motor Carrier Safety Improvement Act of 1999 (MCSIA, P.L. 106-159) and the Safe, Accountable, Flexible, Efficient Transportation Equity Act - a Legacy for Users (SAFETEA-LU, P.L. 109-59, Sections 4106, 4107, and 4138). Authority: 49 U.S.C. 31100-31104, 31108, 31136, 31140-31141, 31161, 31310-31311, 31502; and 49 CFR 1.48. Regulation: 49 CFR Part 350. Policy: DOT highway safety performance and safety strategic goals reflected in the DOT Strategic Plan and the FMCSA Strategic Plan.

Does the program address a specific and existing problem, interest, or need?

Explanation: Large trucks are over-represented in fatal highway crashes. Twelve percent of all fatalities in motor vehicle crashes involve in an incident with a large truck, despite the fact that large trucks represent only 3 percent of registered vehicles and approximately 7 percent of total vehicle miles traveled.

Evidence: Audits: Aspects of large truck safety have been identified as top management challenges by the Government Accountability Office (GAO) and DOT Office of Inspector General (OIG). DOT OIG recommendations (TR-1999-01) are specifically addressed in sections 206, 208, 217, and 222 of the Motor Carrier Safety Improvement Act (MCSIA, 1999). OIG has listed large truck safety as a top management concern in recent years (PT-2005-008, PT-2007-004, PT-2008-008). Data: Fatal crash data comes from the Fatality Analysis Reporting System (FARS) which is the DOT standard for fatal motor vehicle crash statistics, maintained by the National Highway Traffic Safety Administration's (NHTSA). Vehicle miles traveled (VMT) and registered vehicle data comes from the Federal Highway Administration's (FHWA) annual Highway Statistics publication (table VM-1).

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: Federal financial assistance to State agencies extends the reach of State safety enforcement. FMCSA regulates interstate and cross-border commercial motor vehicle transportation. State involvement extends these interstate enforcement activities to the roadside and expands enforcement to intrastate commerce. The MCSAP maintenance of effort requirements (see 49 CFR Part 350.301) ensures these Federal grant funds do not supplant State funds for commercial vehicle safety efforts.

Evidence: Audits: GAO (GAO-02-495) confirms the complementary nature of Federal and MCSAP-supported State programs. FMCSA is the only Federal agency that addresses and remediates the causes of commercial motor vehicle crashes and resulting injuries and fatalities. Prior to 1982, few States focused on commercial vehicle safety issues, motor carrier compliance or commercial driver certification. Currently, every State, the District of Columbia, and some territories have Commercial Vehicle Safety Plans. Unlike NHTSA's State grant program that focuses on educational traffic safety efforts, MCSAP grants provide for direct State motor carrier enforcement activities. Legislation: STAA, TEA-21, MCSIA, and SAFETEA-LU (See Evidence for 1.1). Authority: 49 U.S.C. 31100-31104, 31108, 31136, 31140-31141, 31161, 31310-31311, 31502; and 49 CFR 1.48. Regulation: 49 CFR Part 350. Policy: DOT Highway safety performance and safety strategic goals reflected in the DOT Strategic Plan and the FMCSA Strategic Plan.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The design of MCSAP allows FMCSA to extend its effects of its programs to the States while limiting the direct Federal role in enforcement. MCSAP regulations in 49 CFR Part 350 specify the conditions that States must meet to participate in the MCSAP and receive grant funds. These conditions include the requirement that States adopt and enforce compatible CMV safety and hazardous materials regulations for both interstate and intrastate commerce. To continue receiving Federal funds, States must comply with requirements for maintenance of effort and provide matching shares of costs. Additionally, FMCSA issues a MCSAP planning memo each year to inform States of agency and national priorities as they develop their annual Commercial Vehicle Safety Plans (CVSP). These comprehensive state plans provide a framework for monitoring performance and expenditures throughout the year. FMCSA also performs year-end closeout of financial and performance benchmarks. Evaluations are conducted on grantees over a four-year cycle by FMCSA with support from an independent contractor.

Evidence: Audits: "Under the Motor Carrier Safety Assistance Program (MCSAP), states that meet grant requirements can take responsibility for enforcing these regulations for both intrastate and interstate vehicles, and nearly all states have done so." (GAO-02-495) "MCSAP is designed to improve safety by employing a performance-based approach" (GAO-06-156). States that fail to perform according to their approved CVSP or otherwise fail to meet the conditions in Part 350 may receive a written determination of non-conformity from the Administrator. If the State does not resolve the non-conformity issue then all MCSAP Basic and Incentive grant funds are withheld. Letters: 1) Non-conformity letter from FMCSA Administrator to Governor of Alabama; 2) Response letter from Alabama Governor to FMCSA Administrator.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: FMCSA provides MCSAP grants directly to the State organizations that promote large truck and bus safety. Through the CVSPs, States explain the benefit that the grant will provide in terms of output and outcome safety performance measures. FMCSA staff provide oversight to make sure funds are being used for the intended purposes. MCSAP and coodinating strategies provides the widest range of exposure for safety laws and regulations on the motor carrier industry.

Evidence: MCSAP supports State- and locally-conducted motor carrier safety activities to ensure compliance with Federal Motor Carrier Safety Regulations (FMCSRs), including compliance reviews, roadside inspections, and traffic enforcement. MCSAP grants to States contribute, with other FMCSA safety mitigation strategies/programs (including partnership, outreach, information/research, education, rulemaking, compliance, and enforcement), as an integral part of a coordinated strategy to increase compliance with FMCSRs and reduce crashes, fatalities, and injuries. STAA, TEA-21, MCSIA and SAFETEA-LU (See Evidence of 1.1). Authority: 49 U.S.C. 31100-31104, 31108, 31136, 31140-31141, 31161, 31310-31311, 31502; and 49 CFR 1.48. Regulation: 49 CFR Part 350. Policy: DOT Highway Safety performance and Safety strategic goals reflected in the DOT Strategic Plan and the FMCSA Strategic Plan. Data: Safety statistics from FARS and Highway Statistics.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: MCSAP contributes towards the Agency's long-term goal - to reduce the rate of fatalities in crashes involving large trucks and buses to no more than 0.160 fatalities per 100 million VMT by 2011, a reduction of 22% from the 2000 baseline. This long-term goal has been translated into specific annual performance targets and is directly tied to the DOT's safety strategic goal, FMCSA strategic plan and the DOT Safety Action Plan for Large Trucks and Buses. MCSAP also has a number of other output and outcome goals that support the overall goal that are listed in the Agency's annual Performance Budget submission to Congress as well as this PART Review.

Evidence: Policy: DOT Strategic Plan 2006-2011, FMCSA Strategic Plan 2006-2011, DOT Large Truck and Bus Safety Action Plan, FMCSA FY 2009 Congressional Budget Submission.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: FMCSA has developed annual targets for reduction of fatality rate, and monitors fatalities and injuries as leading indicators. Annual performance targets are established for combined Federal/State roadside inspections and Federal compliance reviews.

Evidence: The FMCSA's long-term safety objective is to reduce the rate of large truck and bus related fatalities to no more than 0.160 per 100 million total Vehicle Miles Traveled (VMT), by the year 2011. This goal equates to a cumulative reduction of 25 fatalities per year in the face of an ever-increasing motor carrier activity and vehicle traffic on our Nation's highways. FMCSA sets annual performance targets for achieving this reduction and is on track towards achieving the 0.160 long-term goal, having reduced the rate of truck-related fatalities over the past two years. Targets for essential operational program outputs, including those supported by the MCSAP program, are addressed in the agency's annual performance plan and performance budget submission. Supporting performance measures, such as large truck safety belt use and improved data quality, contribute towards completion of the overall safety goal. See additional information in the performance measures section. Policy: FMCSA Strategic Plan 2006-2011, DOT Large Truck and Bus Safety Action Plan, FMCSA FY 2009 Congressional Budget Submission.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: Performance results are calculated annually for all major measures, with some measures evaluated more frequently. States address five MCSAP safety performance elements in their Commercial Vehicle Safety Plans (CVSPs) and grant applications. In addition, qualification for special incentive funding requires State adoption of specific performance objectives for: reduction of fatal accidents, reduction of CMV-involved fatality rate, CDL verification, and inspection and accident data timeliness, completion and accuracy. FMCSA tracks the performance of each State with regard to the outcome goal of reducing truck-related fatalities and accidents in each State, and tracks their progress in achieving these goals.

Evidence: Audit: The GAO (GAO-02-495) found that annual State plans include quantifiable performance objectives and measures and strategies and specific activities for achieving the objectives. Policy: The MCSAP Program Office sends out an annual planning memorandum to its State partners which outlines national performance goals for the upcoming fiscal year. Since 2000, all State MCSAP CVSPs were required to be prepared in a performance-based format. See CVSPs and MCSAP grant applications, and the performance measures in this PART review.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: FMCSA's annual integrated performance budget and performance plans and reports include baselines and targets that the MCSAP influences. The FMCSA's long-term safety objective is to reduce the rate of large truck and bus related fatalities to no more than 0.160 fatalities per 100 million total Vehicle Miles Traveled (VMT), by the year 2011. This goal equates to a cumulative reduction of 25 fatalities per year in the face of an ever growing population of motor carriers and increasing vehicle traffic on our Nation's highways. This goal equates to a 22% reduction in rate of fatalities from the baseline fatality rate of 0.205 fatalities per 100 million VMT in 2000.

Evidence: Policy: FMCSA FY 2009 Congressional Budget submission, FY 2007 Performance and Accountability Report (PAR), MCSAP annual planning memos

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: MCSAP partners commit to and work towards the annual and long term goals of the program. FMCSA headquarters and field staff communicate constantly with MCSAP partners throughout each fiscal year to ensure that these goals are realized. This one-on-one relationship, continued throughout the lifecycle of the grant, ensures that program partners and FMCSA are committed and dedicated to the goals of the MCSAP program. States address five MCSAP safety performance elements in their Commercial Vehicle Safety Plans (CVSPs)/grant applications. FMCSA tracks the performance of each State with regard to the goal of reducing truck-related fatalities and crashes in each State. States provide data on activities conducted, and these data provide the basis for Agency decisions regarding identifying high-risk motor carriers, determining effectiveness of Agency programs and contribute highly to cost-benefit analysis used in rulemakings. A MCSAP grantee that fails to perform according to an approved CVSP may not be eligible for MCSAP funds per 49 CFR 350.215.

Evidence: Audit: The GAO (GAO-02-495) found that annual State plans include quantifiable performance objectives and measures and strategies and specific activities for achieving the objectives. Policy: The MCSAP Program Office sends out an annual planning memorandum to its State partners that outline the performance goals for the upcoming fiscal year. Since 2000, all State MCSAP CVSPs were required to be prepared in a performance-based format. Risk-based training is provided to States in prioritizing goals and assigning resources. Regulation: Requirements for MCSAP are found in 49 CFR Part 350 (specifically 350.201, 350.209, 350.213, and 350.215)

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Every year, FMCSA engages an outside, independent contractor to review the effectiveness of the activities funded by the MCSAP program and assess the contribution of these activities towards the agency's outcome goals. The Volpe National Transportation Systems Center annually evaluates and issues reports on the effectiveness of safety mitigation strategies: Compliance Review Impact Assessment Model, June 2007; Intervention Model: Roadside Inspection and Traffic Enforcement Effectiveness Assessment, September 2006. Evaluations are also performed on the State grantees by Science Applications International Corporation (SAIC) and the North American Driver Safety Foundation (NADSF). These reviews began in FY 2007 with 6 State reviews conducted. In FY 2008, 10 additional State reviews are planned with another 12 to be conducted in FY 2009 so that every State will be reviewed in a 4-5 year cycle. The reviews examine the compatibility of State regulations, financial procedures and program performance. For each finding, the State must develop a corrective action plan which is approved by FMCSA and monitored by the FMCSA Division offices. External evaluations are also performed by DOT OIG and GAO.

Evidence: Data: The compliance review effectiveness model and roadside inspection and traffic enforcement effectiveness assessment can be found online at: http://ai.fmcsa.dot.gov/ProgramMeasures/PM/PM.asp. Audits: MCSAP has been reviewed by GAO (GAO-02-495, Regulatory Programs: Balancing Federal and State Responsibilities, GAO-06-156, Federal Enforcement Efforts Have Been Stronger Since 2000, but Oversight of State Grants Needs Improvement) and DOT OIG emphasizes the importance of strong enforcement to ensuring motor carrier safety (AS-FH-4-012, 6/94 and TR-1999-091, 4/99). Policy: State corrective action plans.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The FMCSA has developed a set of logic models, included in its annual integrated performance budget, that details the alignment between funding for all programs, including the MCSAP program, and performance on Agency strategic goals. The impact of funding, policy and legislative changes are reflected in the budget submission. In these logic models, MCSAP funding aligns with the agency safety program objective "Save Lives and Reduce Injuries by Preventing and Minimizing the Severity of Truck and Bus Crashes" which links to the performance goal/objective "Save lives and reduce injuries by preventing truck and bus crashes" which contributes to the DOT Highway Safety performance goal.

Evidence: Policy: FMCSA FY 2009 Congressional Budget Submission (Section 4 - Performance Budget Estimates), FMCSA Strategic Plan 2006-2011, DOT Strategic Plan 2006-2011.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The Agency has integrated its performance planning and reporting with its budget process and has a strong strategic planning process. FMCSA actively works to mitigate the primary weakness of the program—the lack of direct authority over State agencies—by including State grantees in the strategic planning process. MCSAP is a major contributor to FMCSA's overall strategy aimed at reducing large truck- and bus-related fatalities and injuries.

Evidence: Policy: MCSAP is identified in the FMCSA's annual integrated performance budget (pages 4A-24 to 4A-27), performance plans, and DOT's FY 2007 Performance and Accountability Report (PAR). MCSAP corrective action plans are submitted by States following findings made from MCSAP State reviews conducted by an independent contractor.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: FMCSA requires both performance data and policy information from State grantees to ensure that states are working toward agency goals. States furnish detailed reports on enforcement activity and information on crashes involving large trucks and buses. FMCSA uses this information to track State progress toward performance goals. To assure data quality, FMCSA provides regular reports on timeliness, accuracy and completeness of grantees' roadside inspection and crash data. The program also collects and analyzes State rulemaking information to ensure that State regulations are compatible with Federal regulations.

Evidence: Data: State Safety Data Quality Map, updated monthly, (http://ai.fmcsa.dot.gov/DataQuality/dataquality.asp). Timeliness of Uploads Report and the Data Non-Match Reports from the Motor Carrier Management Information System (MCMIS) are used to quickly identify upload deficiencies, allowing for remedial action. Policy: Data quality improvement is a required part of each State's CVSP, and State reporting must meet the standards set forth in 49 CFR Part 390.5 and other guidance. FMCSA also examines the compatibility of State regulations to those contained in the Federal Motor Carrier Safety Regulations (FMCSRs) and Hazardous Materials Regulations (HMRs), and this compatibility is a key program goal of MCSAP. MCSAP funding is conditioned upon State adoption and enforcement of State laws that are compatible with the FMCSRs and HMRs, as such, it contributes to elevating regulatory and enforcement standards across the country.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Federal managers for MCSAP are held accountable for cost, schedule and performance results in their yearly performance plana. In addition, MCSAP distributes incentive grants for grantees that demonstrate improvement in specific safety and program performance areas. MCSAP's policy is to reduce or withhold funding in instances in which program partners do not have compatible CMV safety laws and regulations pertaining to interstate and intrastate commerce. However, independent reviews have found that MCSAP does not always adequately implement existing mechanisms for overseeing grant activity.

Evidence: Policy: FMCSA State Division Administrators prepare monitoring plans for each grant program. Reimbursement vouchers are scrutinized to ensure that costs are reasonable and fall within the Commercial Vehicle Safety Plan (CVSP) budget. Audit: "While MCSAP is designed to ensure that its grants to states contribute to the agency's mission of saving lives and reducing injuries by preventing truck crashes, FMCSA's oversight of state grantees is lacking" (GAO-06-516).

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: In fiscal year 2009, FMCSA will initiate a quarterly Financial Status Report (FSR) for the grantees. This new process within the Agency will be in addition to the vouchering requirements and monitoring plans already set in place. Filing the FSR will allow FMCSA to ensure that payments are made in a timely manner and that the grantee avoids penalties that would otherwise result from delays. The new processes will ensure that no unobligated funds remain at the end of the year. The program office reimburses States after approved expenditures are incurred. FMCSA division administrators are responsible for reviewing vouches to ensure costs claimed are in accordance with the approved CVSP and eligible for reimbursement.

Evidence: Policy: FMCSA's MCSAP program managers monitor the obligation activity for timeliness on a regular basis. FMCSA State Division Administrators review all reimbursement vouchers to ensure that claimed expenses are in conformance with the budget submitted with the approved CVSP. Any questionable expenditures are immediately resolved with State partners. Audit: (GAO-07-119 pages 44 and 45)

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: MCSAP does not currently have a cost-based efficiency measure, nor does the program track State-level efficiency measures. MCSAP is, however, increasing the proportion of driver-only roadside inspections to total inspections. Several studies have shown that increases in both roadside inspections and compliance reviews significantly reduce crashes and prevent fatalities and injuries. In addition to the basic MCSAP formula grant, FMCSA awards incentive grants to States that demonstrate performance improvement with respect to specific performance goals. One objective of the SAIC / NADSF program evaluation contract is to promote operational efficiency and effectiveness through the exchange of ideas. While current findings have concentrated mostly on state improvements, Federal processes are within the scope of the contract.

Evidence: Audit: GAO (GAO-02-495) recognizes MCSAP financial incentives for States to achieve reductions in CMV fatal crash rates. State incentive grant funding requires State performance improvement with regard to the following specific safety goals: (1) reduce the number of fatal crashes involving large trucks, (2) reduce the large-truck-involved fatal crash rate or maintain a rate that is among the lowest 10% of all MCSAP recipients, (3) verify commercial driver's licenses during all roadside inspections, (4) Upload CMV inspection data in accordance with current FMCSA policy guidelines, and (5) upload CMV accident reports in accordance with current FMCSA policy guidelines. Report: Volpe National Transportation Systems Center reports on the Compliance Review Effectiveness Model and the Roadside Intervention Effectiveness Model (http://www.ai.fmcsa.dot.gov/CarrierResearchResults/Outreach.asp?pgs=16).

Does the program collaborate and coordinate effectively with related programs?

Explanation: MCSAP grants are directly integrated into the FMCSA's coordinated motor carrier safety strategy, along with motor carrier partnerships, education/outreach, information/research, rulemaking, compliance, and enforcement program activities, which all contribute to the achievement of motor carrier safety outcomes. DOT requires each State to submit a strategic highway safety plan to receive DOT grants. This strategic plan incorporates multi-modal planning at the state level and includes FMCSA CMV initiatives with the activities of the Federal Highway Administration and the National Highway Traffic Safety Administration, among others. States also coordinate development of their CVSP with the Strategic Highway Safety Plan that the State completes for highway safety grants from FHWA and NHTSA. FMCSA participates in the Department's multimodal, multidisciplinary, Speed Management Team, which consists of members from other modes within DOT (i.e., NHTSA, FHWA). Lastly, members of FMCSA grant programs regularly sit on the grants review teams of other DOT modes (i.e., NHTSA 408 Grants review team) and exchange information regarding which States are applying for grant opportunities in more than one USDOT grant program and which States were approved for particular grants in past years.

Evidence: Policy: See pages 4A-9 through 4A-17 in the FY 2009 FMCSA Congressional Budget Submission for examples of FMCSA collaborating and coordinating with similar highway safety organizations within DOT. Planning for/integration of MCSAP goals in broader agency fatality and injury outcomes is manifest in the agency performance budget and performance reports, with accountability for performance cascaded through the organization (HQ and Field) via a network of performance accountability contracts. Department of Transportation-level collaboration is reflected in the consolidated Highway Safety planning in the DOT Strategic Plan, activities planned in the DOT Large Truck and Bus Safety Action Plan, performance metrics and monitoring in the DOT Performance Budget. Regulation: MCSAP implementing regulations require States to coordinate their CVSPs with State highway safety programs under 49 CFR 350.211.

Does the program use strong financial management practices?

Explanation: A recent financial audit by an independent CPA firm, hired by OIG, gave a clean opinion to DOT internal financial controls. The audit concluded that management's description of controls presents fairly, in all material respects, the controls that have been placed in operation as of May 31, 2007. A second related audit concluded that DOT's consolidated financial statements from FY 2006 and 2007 are fairly presented, in all material respects, in conformity with generally accepted accounting principles.

Evidence: Audits: OIG reports QC-2007-072 "DOT Delphi Financial Systems Control" and FI-2008-011 "Report on DOT's Consolidated Financial Statements for FY07 and FY06".

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The effectiveness of essential program enforcement interventions is evaluated on an annual basis to inform strategy development and resource allocation decisions. Program evaluation findings are reviewed in the development of annual Agency and Departmental strategic and performance plans and reports.

Evidence: Policy: State Correctives Action Plans from completed State program effectiveness reviews conducted by SAIC / NADSF (See 3.3). The FMCSA strives for continuous improvement in its management and has an active evaluation program aimed at improving program effectiveness, efficiency and agency performance. These evaluation activities are outlined in FMCSA's annual budget submission to Congress. When the MCSAP program staff identifies management deficiencies in areas such as strategic staffing management and financial resource allocation, they initiate corrections and/or modification to management plans and practices. Periodic leadership meetings, including an annual forum with State partners, are held to vet identified program issues and to develop strategies for incorporating corrections into ongoing management operations.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: FMCSA Division Administrators establish monitoring plans to review State performance activities set out in the CVSP. These plans include site visits to ensure program conformance with the CVSP. States submit quarterly performance reports that document completion of projected activities and goals. Progress and final vouchers contain expenditure details which ensure resources are used for the purposes identified in the State's CVSPs. In fiscal year 2009, FMCSA will initiate a quarterly Financial Status Report (FSR) for the grantees. The FSR will allow FMCSA to ensure that the grantee avoids penalties, that payments are made in a timely manner, and avoid delays in award funding.

Evidence: Policy: Narrative quarterly performance reports, provided by State, documentation of periodic site visits by division staff, and review of expense vouchers for funds used for their designated purpose.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: Overall program performance data (inspections, crashes, and compliance reviews) are electronically uploaded continually throughout the year to the Motor Carrier Management Information System (MCMIS). The data are available to the public from the FMCSA's Analysis and Information Online website (http://ai.fmcsa.dot.gov/). However, in a review of the seven largest MCSAP grant recipient States in 2005, GAO found that FMCSA does not always follow its procedures for communicating program goals to States, assessing States' performance on annual goals, and ensuring that all States had quantifiable targets.

Evidence: Data: Performance data are available to the public through the FMCSA's Analysis and Information Online (A&I) website including information on the number of inspections, compliance reviews and crashes submitted by the States, as well as breakdowns of the data quality, program effectiveness and other statistical analyses. Policy: Progress in reducing motor carrier crashes, fatalities, and injuries is reported in the FMCSA and DOT annual performance reports, DOT Large Truck and Bus Safety Action Plan, FMCSA FY 2009 Budget Submission to Congress, DOT FY 2007 Performance and Accountability Report. Audit: GAO-06-156

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: In 2006, FMCSA, NHTSA and FHWA harmonized their safety goals with the new consolidated Department of Transportation highway safety goal. The agency's long-term safety goal is to reduce the large truck and bus fatality rate 22% from 2000 to 2011, to a rate of 0.160. In the face of increased exposure owing to annual increases in commercial and total motor vehicle miles traveled, fatalities involving large trucks and buses have been reduced 5.5% from 2000 to 2006. Injuries and injury rates that the agency tracks as leading indicators have also been reduced. Robust State safety programs and enforcement interventions supported by MCSAP are important factors in saving these lives and avoiding these injuries.

Evidence: Policy: FMCSA FY 2009 Congressional Budget Submission, DOT Large Truck and Bus Safety Action Plan, performance measures in PART.

Does the program (including program partners) achieve its annual performance goals?

Explanation: MCSAP annual goals link and contribute to the accomplishment of the motor carrier safety long-term outcome goal of reducing CMV-related fatality rate. MCSAP is achieving specific program-level goals such as safety belt use and data quality of State-reported crash and roadside inspection data. However, MCSAP could improve oversight of State Grantees to ensure that they are achieving their annual performance goals.

Evidence: Audit: GAO-02-495 indicates that annual State plans include quantifiable performance objectives and measures and strategies and specific activities for achieving the objectives, but GAO-06-156 indicates that oversight of actual performance results is lax. Policy: FMCSA FY 2009 Congressional Budget Submission (pages 4A-24 to 4A-27), DOT Large Truck and Bus Safety Action Plan, performance goals in this PART.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: MCSAP does not currently have a cost-oriented efficiency measure. FMCSA has, however, taken made efforts to focus more of its resources towards large truck driver inspections and traffic enforcement on passenger vehicle drivers driving around large trucks. The Large Truck Crash Causation study has identified that driver actions, of both large trucks and passenger vehicles, play a significant role in large truck crashes.

Evidence: Policy: FMCSA FY 2009 Congressional Budget Submission, DOT Large Truck and Bus Safety Action Plan, DOT FY 2007 Performance and Accountability Report (PAR), efficiency measures in PART.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation:

Several agencies within the Federal government strive to reduce highway fatalities. Within the Department of Transportation, NHTSA, FHWA and FMCSA share the highway fatality goal since each of the three agencies have a responsibility to improve safety on our nation's highways. Because vehicles of all kinds share the highways there is a common responsibility among these agencies to coordinate efforts in tracking performance. The shared DOT performance measure relative to this common goal is to reduce fatalities on the nation's highways to no more than 1 fatality per 100 million vehicle miles traveled by the end of 2011.

In the February 2008 report to Congress on the DOT Fatality Rate Goal, FMCSA explained that the target date for achieving the 1.0 goal had been revised from 2008 to 2011, to account for the dramatically changing nature of the challenges currently facing highway safety. To most effectively align program and policy actions needed to meet key challenges, the Department has established four fatality submeasures??passenger vehicles, nonoccupants, motorcycle riders, and large-truck and bus-related fatalities??which represent the breadth of all highway users. The purpose of this approach is to more closely examine the fatality rates of the different segments of highway users, devote greater energy and resources, and develop new strategies to combat sub-measure trends that are impeding progress to the overall 1.0 goal.

The alignment of all of these goals provides a unique comparison of the effectiveness of FMCSA's activities to the other DOT activities. In striving for a lower fatality rate, NHTSA and FMCSA employ a similar combination of approaches, including a focus on drivers, vehicle standards, outreach, education and research. All three agencies have comparable partnership relationships to the transportation industry, safety advocacy partners, State government partners and other stakeholder groups.

The FMCSA goal that contributes to the overall DOT goal aims to reduce the rate of large truck and bus fatalities per 100 million vehicle miles travelled. Fatalities in crashes involving large trucks and buses declined seven percent between 1997 and 2006, according to the FY 2007 PAR. With the agency's achievement its 2006 target, progress toward reduction of truck-related fatalities and injuries is tracking with Departmental objectives.

Evidence: The agency has achieved its target for large truck fatality rate reduction in 2006. Despite annual increases in VMT and CMV VMT, fatalities and injuries in crashes involving large trucks have been reduced. Large-truck-related fatality and injury rates have also been reduced and safety belt use by large truck drivers has increased . Results are reflected in the 2006 Large Truck Crash Facts, DOT FY 2007 PAR, and FMCSA's FY 2009 Congressional Budget Submission. Data supporting DOT Highway Safety measures for all highway modal Administrations are housed in FARS.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: In 2005 GAO reported that "in reviewing the safety goals of the seven states that received the largest MCSAP grants, we could not determine whether states substantially met 37 of their 61 goals (61 percent) to improve truck safety." However, the Volpe Center's evaluations of the major MCSAP-supported activities have found some positive effect of compliance reviews and inspections, in terms of lives saved.

Evidence: Audits: MCSAP has been reviewed by GAO (GAO-02-495 and GAO-06-156) and DOT OIG (AS-FH-4-012, TR-1999-091). The effectiveness of compliance reviews, roadside inspections, and traffic enforcement in reducing crashes, fatalities, and injuries is evaluated on an annual basis using the: (1) Intervention Model and (2) Compliance Review Impact Assessment Model. These evaluation models yield annual estimates of crashes avoided, injuries avoided, and lives saved attributable to these interventions. Additional information about the Intervention Model and the Compliance Review Impact Assessment Model accompany this assessment as attachments (more information can be found online at: http://ai.fmcsa.dot.gov/ProgramMeasures/PM/PM.asp).