Detailed Information on the
Railroad Safety Program Assessment

Demonstrate use of efficiency measure in FY 2008

Accomplish at least 90 percent of the 2007 action items identified in FRA's National Rail Safety Action Plan -- May 2007 Update.

Measure: Rail Related Accidents and Incidents per million train-miles (New measure, added February 2008)

Explanation:This measure tracks Fray's performance in reducing the rate of all accidents and incidents

Measure: Grade crossing incidents per million train-miles (New measure, added February 2008)

Explanation:This measure tracks FRA's performance in reducing the number of grade crossing incidents per million train-miles.

Measure: Human-factors-caused train accidents per million train-miles (New measure, added February 2008)

Explanation:This measure tracks FRA's performance in reducing the number of human-factors-caused train accidents per million train-miles.

Measure: Track-caused accidents per million train-miles (New measure, added February 2008)

Explanation:This measure tracks FRA's performance in reducing the number of track-caused accidents per million train-miles.

Measure: Equipment-caused accidents per million train-miles (New measure, added February 2008)

Explanation:This measure tracks FRA's performance in reducing the number of equipment-caused accidents per million train-miles.

Measure: Other train accidents per million train-miles (New measure, added February 2008)

Explanation:This measure tracks FRA's performance in reducing the number of other train accidents per million train-miles.

Measure: Non-accident hazardous materials releases per million train-miles (New measure, added February 2008)

Explanation:This measure tracks FRA's performance in reducing the number of non-accident hazardous materials releases per million train-miles

Measure: Ratio of Indirect Spending on safety activities

Explanation:

Is the program purpose clear?

Explanation: The Railroad Safety Program (RSP) promulgates, administers, and enforces the Federal laws and regulations designed to promote safety on the Nation's railroads. FRA's oversight focuses on five safety "disciplines" -- track, equipment, operating practices, signals, and hazardous materials.

Evidence: FRA's authorizing statutes include: Title 49 USC Chs. 201-213 (railroad safety program); Title 49 USC Ch. 51 (transporting of hazardous materials); and 49 CFR Sec 1.49 delegates authority to the Federal Railroad Administration (FRA).

Does the program address a specific and existing problem, interest, or need?

Explanation: The hazards posed by trains are very real, with thousands of rail-related injuries and fatalities occurring annually to railroad employees, car and truck passengers, and rail trespassers.

Evidence: 1) FRA's "Railroad Safety Statistics." See FRA Safety Website at http://www.fra.dot.gov/safety

Is the program designed so that it is not redundant or duplicative of any Federal, state, local or private effort?

Explanation: The program is the only Federal regulator of railroad safety nationwide, though FRA works regularly with other relevant Federal transportation agencies, including the Federal Highway Administration, Transportation Security Administration, and the Occupational Health and Safety Administration. FRA works closely with the private sector since the RSP ensures safety of the rail system by overseeing the safety programs of individual private railroads. Additionally, FRA collaborates with States by certifying their rail safety inspectors for those States with safety programs (30 States have rail safety programs).

Evidence:  

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The program is designed to encourage regulatory compliance by the railroads and promote safety improvements. Through inspections, railroad audits, and enforcement tools that include civil penalties, the program balances enforcement action with cooperative approaches. The program also supports educational outreach efforts through its highway-rail grade crossing program. FRA's primary challenge in managing the safety program is to give the appropriate weighting to its cooperative efforts with the railroads and to its regulatory enforcement work.

Evidence: 1) FRA's Safety Assurance and Compliance Program (SACP). See FRA website at www.fra.dot.gov/safety/sacp.htm. 2) DOT IG Follow-up Audit of Safety Assurance and Compliance Program (2-8-02 and 4-18-02). 3) FRA's Railroad Safety Advisory Committee (RSAC). 4) Compliance Agreements. 5) Focused Enforcement, e.g., FRA's Switching Operations Fatality Analysis. See FRA website at www.fra.dot.gov/safety/sofa/index.htm

Is the program effectively targeted, so program resources reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: Through focused inspections, safety audits, and a variety of enforcement tools, RSP resources directly target serious safety problems. FRA uses its Accident/Incident data along with its inspection findings to plan inspections as specified in its focused enforcement guidance document. FRA's main resource management challenge is to monitor the condition of the nation's entire rail system with a limited number of rail inspectors. Another challenge is to effectively target highway-rail grade crossing resources, including collecting better data on how States currently spend their Federal grade crossing dollars.

Evidence: 1) IG Memorandum, Follow-up audit of safety and assurance and compliance program, April 18, 2002.2) For SACP, see FRA website at www.fra.dot.gov/safety/sacp .3) For grade crossing, see FRA website at www.fra.dot.gov/safety/90hrc.htm.4) For OLI, see FRA website at www.fra.dot.gov/safety/oli .5) For RSAC, see FRA website at www.fra.dot.gov/safety/rsac .6) For safety data, see FRA website at www.fra.dot.gov/safety/safetydata .

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The RSP has five specific long-term performance measures that reflect its mission and support the critical outcome goals established by DOT to improve transportation safety nationwide. These measures concern reducing rail-related fatalities, injuries, train accidents, highway-rail grade crossing accidents, and rail hazmat releases.

Evidence: 1) DOT Strategic Plan 2003-2008. See website http://Stratplan.dot.gov .2) DOT FY 2002 Performance & Accountability Report. See website www.dot.gov.3) DOT Performance Plan - FY 2004. See website www.dot.gov/perfplan2004/index.htm.4) FRA FY 2004 Budget.5) FRA Strategic Plan 2000-2003. See FRA website at www.fra.dot.gov/about/FRAstrategic_plan_.htm.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: FRA adjusts its targets annually based on past performance, projected program resources, and the expectation of being able to meet targets.

Evidence: 1) DOT FY 2004 Performance Plan. 2) DOT FY 2002 Performance & Accountability Report. 3) FRA Strategic Plan 2000-2003.

Does the program have a limited number of specific annual performance measures that demonstrate progress toward achieving the program's long-term measures?

Explanation: The RSP has five distinct, quantifiable, annual performance measures that demonstrate its progress toward achieving the long-range goals set by DOT, as noted above.

Evidence: 1) DOT Strategic Plan 2003-2008. 2) DOT FY 2004 Performance Plan. 3) DOT FY 2002 Performance & Accountability Report. 4) FRA FY 2004 Budget. 5) FRA Strategic Plan 2000-2003.

Does the program have baselines and ambitious targets and timeframes for its annual measures?

Explanation: FRA adjusts its targets annually based on past performance, projected program resources, and the expectation of being able to meet targets.

Evidence: 1) FRA FY 2003 and 2004 Budgets. 2) DOT FY 2002 Performance & Accountability Report. 3) FRA Strategic Plan 2000-2003.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, etc.) commit to and work toward the annual and/or long-term goals of the program?

Explanation: FRA partners with several stakeholders in the public and private sectors. States work with the FRA to enforce Federal rail safety regulations (30 States with rail safety programs employ 160 safety inspectors). Operation Lifesaver, a national, nonprofit education and awareness program, receives FRA funds for rail crossing safety and railroad rights-of-ways campaigns. At the Federal level, FRA and FHWA cooperatively manage the highway rail-grade crossing program, which is funded from FHWA resources. Further, the freight railroads and labor unions help evaluate private companies' safety programs and develop strategies for system-wide improvements through SACP and RSAC.

Evidence: 1) Before participation can begin, each State agency must enter into an agreement with FRA for the exercise of specified authority. This agreement may delegate investigative and surveillance authority regarding all or any part of Federal railroad safety laws. 2) Operation Lifesaver, Inc., Contract/Agreement.

Are independent and quality evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: FRA has not arranged for independent evaluations of the safety program's design and effectiveness. However, FRA has some mechanisms for receiving feedback on its performance such as RSAC, which reviews major regulatory programs in collaboration with industry stakeholders. FRA has also hired an independent contractor to perform human capital analysis of FRA's five railroad safety inspection disciplines.

Evidence: 1) RSAC minutes. See: http://rsac.fra.dot.gov/rsac_doc.htm. 2) DOT Human Capital Plan. See: http://dothr.ost.dot.gov/About_Us/Human_Capital_Plan/human_capital_plan.html.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The FRA FY 2004 Budget request to Congress directly linked its five annual performance goals to budget data. FRA could improve the transparency of the budget request by distinguishing funding for the rail safety program including SACP, RSAC, the rail crossing program, and rail inspectors.

Evidence: 1) FRA FY 2004 Budget. See: www.dot.gov/bib2004/fra.html

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The RSP annually reviews and adjusts or updates its performance goals and measures. It is currently assessing the effects of reducing its long-range and annual goals from five to three to more meaningfully support the DOT Safety Strategic Goal.

Evidence: 1) FRA Strategic Plan 2000-2003. See www.fra.dot.gov/about/fra_strategic_plan.htm.2) DOT Strategic Plan 2000-2005.3) DOT Strategic Plan 2003-2008. See http://stratplan.dot.gov .

Are all regulations issued by the program/agency necessary to meet the stated goals of the program, and do all regulations clearly indicate how the rules contribute to achievement of the goals?

Explanation: Each regulation clearly articulates a problem statement for a specific safety issue and explains how the rule elements will solve the stated problem. When feasible, each of the rule's elements are priced separately and its benefits identified. The consensus process within the Railroad Safety Advisory Committee (RSAC) also ensures a sharp focus on results. Individual working groups within RSAC are assigned to certain regulatory tasks. Each working group is comprised of knowledgeable representatives of stakeholder organizations and FRA, and the products of their deliberations may be reported to the full committee only by consensus.

Evidence: 1) Report on DOT Significant Rulemakings. See http://regs.dot.gov/report2.htm.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: FRA collects monthly accident and fatality data from the nation's railroads, including data by county on the location of trespassing deaths, which is the leading type of rail fatality. The data is posted on FRA's website. DOT uses the data to improve program performance, including through a focused inspection methodology that has successfully reduced injuries and fatalities of train and engine service employees. Further, using HAZMAT data, FRA created a Safety Action Plan for shippers with the highest number hazard material incidents. Also, FRA managers develop annual Action Plans for every class 1 railroad based on safety information. Despite this work, FRA could improve its data collection and management efforts. For example, the OIG found that FRA SACP team leaders frequently didn't use accident and fatality data when developing profiles of individual railroads, and FRA didn't always use data on railroad safety inspections for program management. Also, DOT currently does not collect information on how States use their FHWA/FRA highway rail grade crossing funds.

Evidence: 1) FRA's "Rail Safety Statistics." See FRA website at http://safetydata.fra.dot.gov/officeofsafety. 2) OIG Audit, Rail-Highway Grade Crossing Safety, September 30, 1999. 3) OIG Audit Report, Safety Assurance and Compliance Program, Sept 30, 1998. See FRA website at www.fra.dot.gov/safety/sacp.htm. 4) FRA primary databases are: Accident/incidents, Inspection data and the Highway-rail grade crossing inventory. See FRA website at: http://safetydata.fra.dot.gov/officeofsafety.

Are Federal managers and program partners (grantees, subgrantees, contractors, cost-sharing partners, etc.) held accountable for cost, schedule and performance results?

Explanation: Program managers and FRA's Regional Administrators are held accountable for the success of the program and the proper administration of the regional offices. Importantly, safety performance data contributes to personnel evaluations of regional administrators, inspectors, specialists, and SACP managers. Further, FRA regional offices develop annual Regional Action Plans outlining how they will achieve their performance goals. FRA management compiles monthly Regional Performance Measures data to monitor the progress of the regions in achieving Agency goals. Additionally, State inspectors must complete a specified number of inspections each year. Ultimately, railroads are responsible for their safety records and are held accountable by civil penalties imposed by FRA. In 2002, FRA collected more than $7.8 million in penalties from railroads and hazardous materials shippers.

Evidence: 1) Annual Regional Action Plans. 2) Quarterly Assessments by FRA Administrator and Deputy Administrator of FRA Regional Administrators. 3)Inspection Day Policy.

Are all funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: The FRA budget officer and the FRA Administrator conduct reviews semi-annually to ensure that funds have been obligated in a timely manner and for their intended purpose. Also, an extensive Annual Advance Procurement process requires every program manager to identify funding needs for any project greater than $100k, which is then reviewed by the contracting officer and the Administrator to ensure linkage to the Strategic Plan and the budget's intended purpose.

Evidence: 1) Annual Advance Procurement Plan. 2) FRA budget officer and administrator mid-year and end-of-year financial reviews.

Does the program have procedures (e.g., competitive sourcing/cost comparisons, IT improvements, approporaite incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: FRA's safety program does not routinely measure program execution efficiency or effectiveness. Still, through its competitive sourcing efforts and information technology initiatives, FRA is attempting to improve the productivity of its safety inspectors. FRA has committed to develop an efficiency or effectiveness measure for its 2004 performance plan.

Evidence: 1) IT/competitive sourcing initiative. See: www.eps.gov/spg/DOT/FRA/OAGS/DTFR53-03-R-00004/SynopsisP.htm .

Does the program collaborate and coordinate effectively with related programs?

Explanation: FRA works with several Federal programs on a regular basis, particularly in the development of rail safety regulations. For example, FRA and RSPA developed regulations for transporting hazardous materials by rail, which FRA then enforces. FRA and DHS are working together to delineate their responsibilities for rail security issues. Going forward, FRA will work with FWHA, FMCSA, and NHTSA to improve DOT's data collection on highway-rail grade crossings.

Evidence: 1) State Rail Safety Participation Program. See FRA website www.fra.dot.gov/safety .2) State Railroad Safety Technical Training Funding Agreement.3) Operation Lifesaver, Inc., Grant Agreement.4) RSPA relationship yields current, consistently enforced hazardous materials regulations.5) FRA/FTA Joint Statement of Agency Policy July 10, 2000.6) OIG Audit Report - Safety Assurance and Compliance Program, Sept 30, 1998.

Does the program use strong financial management practices?

Explanation: The Department's IG audit of FRA's financial statements and the Department's consolidated financial statements found that the financial statements were fairly presented in all material respects, and conformed with U.S. generally accepted accounting principles. No material internal control weaknesses were cited or discovered. FRA was the pilot agency within DOT for implementing the department's new financial accounting system, DELPHI.

Evidence: DOT FY 2002 Performance & Accountability Report; DOT IG Report FI-2003-018 (Jan.27, 2003).

Has the program taken meaningful steps to address its management deficiencies?

Explanation: For example, responding to GAO and OIG report recommendations that the program be more data driven, in 2001 FRA instituted a focused inspection methodology to reduce injuries and fatalities of train and engine service employees. Using data that disclosed specific tasks that railroad employees are engaged in when such fatalities occurred, inspectors focused their human factor monitoring activities on these tasks. Since the initiative began, there has been a reduction in injuries and fatalities related to those tasks. FRA is now developing similar strategies for improving other aspects of railroad safety.

Evidence: 1) "Safety Assurance and Compliance Program" forums between FRA managers and railroad officials. 2) Leadership Development Programs. 3) Regular use of Federal Executive Institute. 4) "360" Management Reviews. 5) "Coaching" programs for senior program managers. 6) Responsibility-Based Safety Enforcement Policy.

Did the program seek and take into account the views of all affected parties (e.g., consumers; large and small businesses; State, local and tribal governments; beneficiaries; and the general public) when developing significant regulations?

Explanation: FRA's has two programs specifically intended to elicit input on FRA's rail safety program from key stakeholders. The Railroad Safety Advisory Committee (RSAC) involves all segments of the rail community on significant safety regulatory issues. The RSAC consists of 27 voting entities representing hundreds of companies and organizations. The SACP is a partnering effort between FRA and individual railroads to collaboratively identify and correct root causes of problems across a railroad. The program supplements the enforcement efforts of FRA safety inspectors.

Evidence: Information on RSAC regulatory issues, meetings, contacts, etc., is located on FRA's public web site and distributed via notices in the Federal Register. See http://rsac.fra.dot.gov/rsac_doc.htm.

Did the program prepare adequate regulatory impact analyses if required by Executive Order 12866, regulatory flexibility analyses if required by the Regulatory Flexibility Act and SBREFA, and cost-benefit analyses if required under the Unfunded Mandates Reform Act; and did those analyses comply with OMB guidelines?

Explanation: All significant safety regulations under the RSP prepare regulatory impact analyses, if required under EO 12866; regulatory flexibility analyses, if required by the Regulatory Flexibility Act and SBREFA; and cost-benefit analyses, if required under the Unfunded Mandates Reform Act. All analyses complied with OMB guidelines.

Evidence: See the electronic docket DMS at DMS.DOT.GOV. FRA posts the Regulatory Evaluations/Regulatory Impact Analysis and Cost Benefits in the electronic docket for public access. Two examples are FRA-1999-6439-12: Regulatory Evaluation for the Train Horn Rule (NPRM) and FRA-2001-11068-13: Regulatory Evaluation of Drug and Alcohol Use.

Does the program systematically review its current regulations to ensure consistency among all regulations in accomplishing program goals?

Explanation: In accords with DOT's Regulatory Policies and Procedures, E.O. 12866 (Regulatory Planning and Review), and Sec. 610 of the Regulatory Flexibility Act, FRA conducts reviews of its rules that (1) have been published within the last 10 years and (2) have a significant economic impact on small entities. FRA publishes a semiannual list in the Federal Register of any such rules that it will review during the year. The DOT's Regulations Council reviews each agenda and DOT has created an Internet site that provides general information for the public about its rulemaking responsibilities and activities.

Evidence: 1) DOT Semiannual Regulatory Agenda (May 27, 2003).

Are the regulations designed to achieve program goals, to the extent practicable, by maximizing the net benefits of its regulatory activity?

Explanation: The FRA, working with interested industry parties, formulates regulations that direct resources towards efficiently achieving safety improvement goals. FRA's cost/benefit analyses demonstrate that in every case it has chosen the regulatory option that produces the greatest benefit at the lowest cost among the reasonably available options. FRA safety regulations are designed to significantly reduce railroad operation hazards to railroad employees and the general public.

Evidence: FRA performs a cost/benefit analysis for every rule, and solicits comments from all stakeholders on its assumptions, calculations, and conclusions, including FRA's efforts to achieve program goals and maximize net benefits. FRA considers all serious comments and makes appropriate changes in the Final Rule and its accompanying cost/benefit analysis. In addition, OMB reviews each FRA rule and accompanying cost/benefit analysis to ensure that FRA gives high priority to the selection of the most cost-beneficial option. For example, FRA received over 3,000 comments on a proposed regulation to establish standards for the use of train horns at highway-rail grade crossings. The proposed rule permitted 'quiet zones' where communities could prohibit the sounding of train horns if they instituted supplemental safety measures. Many commenters noted the high cost of implementing supplemental safety measures at grade crossings that have a historically low risk of accidents. As a result, FRA modified the proposed rule to make accident history a criterion for determining the need for supplemental safety measures.

Has the program demonstrated adequate progress in achieving its long-term outcome performance goals?

Explanation: Overall, safety trends have been positive. Since the late 1980s, FRA's five primary safety indicators show significant improvements, despite increases in rail traffic. For example, rail related injuries per million train miles has fallen from 45 in 1987 to 15 in 2002 and rail hazmat releases per billion hazmat ton miles has fallen from 20 in 1996 to 11 in 2001. During this time train miles increased from 581 million to 728 million.

Evidence: 1) DOT Strategic Plan 2003-2008. See website http://Stratplan.dot.gov .2) DOT FY 2002 Performance & Accountability Report. See website www.dot.gov.3) DOT Performance Plan - FY 2004. See website www.dot.gov/perfplan2004/index.htm.4) FRA FY 2004 Budget.5) FRA Strategic Plan 2000-2003. See FRA website at www.fra.dot.gov/about/FRAstrategic_plan_.htm.

Does the program (including program partners) achieve its annual performance goals?

Explanation: In recent years, FRA has not met all of its annual performance goals, though has come close in most cases. Since the late 1980s, safety indicators have shown significant improvement, but in recent years gains have tapered off. Because railroads by now have taken obvious steps to improve safety, marginal improvements are today harder to achieve.

Evidence: 1) FRA FY 2004 Budget, in which the collected data was used to set new performance goals. 2) DOT FY 2004 Performance Plan. 3) DOT FY 2002 Performance & Accountability Report. 4) "Switching Operations Fatality Analysis" data. 5) FRA's "Railroad Safety Statistics." See FRA Safety Website at www.fra.dot.gov/safety . Click on Safety Data.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program performance goals each year?

Explanation: FRA currently lacks established efficiency and effectiveness measures for program delivery, but has committed to developing efficiency and effectiveness measures for FY 2004. FRA has taken steps to improve program delivery, particularly through recent IT investments that have made rail safety inspectors more productive by reducing the time required to process and retrieve data from FRA's data bases.

Evidence: Examples of IT investments include a safety data website that allows rail safety inspectors to access data instantly rather than having to wait weeks. Also, FRA has custom software that allows inspectors to submit inspection reports from a PC rather than by paper, which has reduced input errors and processing time. The system edits that data entered, does cross validation checks, and provides immediate correction of inspection data. The system was launched nationwide in 1996. Further, FRA inspectors now use Palm PDAs to record track conditions and upload data to a PC. Track inspectors are furnished with a CD disk that is downloaded into their Palm, allowing them to quickly locate defects recorded by track geometry vehicles. This advance has reduced the amount of time inspectors spend locating defects.

Does the performance of this program compare favorably to other programs, including government, private, etc., that have similar purpose and goals?

Explanation: The RSP is the primary Federal regulator of railroad safety nationwide. It is uniquely charged with ensuring that the Nation's railroads operate safely. Therefore, it is difficult to find a suitable comparison. No other Federal agency is designed by law or regulation to cover the broad safety operations of railroads.

Evidence: 1) Title 49 USC Chs. 201-213 (railroad safety program). 2) Title 49 USC Ch. 51 (transport hazardous materials). 3) 49 CFR Sec 1.49 delegates authority to the Federal Railroad Administration. 4) DOT FY 2002 Performance & Accountability Report.

Do independent and quality evaluations of this program indicate that the program is effective and achieving results?

Explanation: A full review of FRA's safety program has not been conducted for several years. The most recent indicate that FRA needs to make better use of data in managing its programs, but FRA also is making efforts to improve. A 1999 report noted progress in reducing grade-crossing accidents and fatalities, and improvements in some aspects of the Safety Assurance and Compliance Program. Also, DOT's IG noted last year that, "FRA is moving in the right direction in better using the information developed in the SACP process," and that there is improved accountability and consistency in the SACP. The IG noted the evolving nature of SACP, "with improvements being added to FRA's safety program."

Evidence: 1) The Rail-Highway Grade Crossing Safety Report was issued on Sept. 30, 1999. See: www.oig.dot.gov/item_details.php?item=223 . The report on the Safety and Compliance Program was issued on Sept. 30, 1998. 2) April 2002 Follow-up Audit by DOT IG of Safety Assurance and Compliance Program.

Were programmatic goals (and benefits) achieved at the least incremental societal cost and did the program maximize net benefits?

Explanation: FRA receives qualitative feedback on the impact of its regulations through the RSAC program. Industry stakeholders including organized labor and railroads discuss the effectiveness and design of current and new regulations. FRA uses this information when updating its regulations and designing its enforcement efforts.

Evidence: The RSAC meets quarterly. Meeting minutes are available at http://rsac.fra.dot.gov/rsac_doc.htm.