Detailed Information on the
FAA Aviation Safety Assessment

Conduct look-back study in FY 2005 to determine if rule maximized net benefits.

Develop an efficiency measure

Measure: U.S. commercial air carrier fatalities per 100 million persons on board

Explanation:Beginning in FY2008, the FAA introduced a new performance metric for commercial air carrier safety??Fatalities per 100 Million Persons on Board. This new metric is more relevant to the flying public, as it better measures the individual risk. All fatalities, including passengers, crewmembers, ramp workers, and ground fatalities, are considered equally. The measure is calculated as the number of fatalities (including ramp accidents and other fatalities as a result of the accident) divided by number of passengers and crew on board flights. The agency aims to cut this risk in half by 2025. Before FY 2008, the commercial fatality rate was a rolling three-year average of the accident rate. The three-year average was calculated by dividing the number of accidents for the previous 36 months by the number of departures.

Measure: Reduce the number of general aviation fatal accidents

Explanation:This measure counts the number of general aviation and non-scheduled Part 135 fatal accidents during the fiscal year. The measure includes all civil (non-military) general aviation aircraft operations that are not Part 121 or Part 135 (commercial aviation). FAA and the general aviation community developed General Aviation Fatal Accidents measure as an overall measure of the impact of improved safety. Beginning in FY 2009, this measure will be converted to a rate. The new safety metric will track the general aviation fatal accident rate as the performance measure rather than the number of fatal accidents. The current measure is not rate-based and does not reflect fleet activity levels and their relationship to the number of fatal accidents. The new performance measure will be a true rate-based metric that tracks changes in the fatal accident rate for a fixed volume of flight hours. The performance target baseline will cover the period from 5/1/05 through 4/30/08. This 3-year period captures the safest years ever recorded for General Aviation so consequently, the baseline and targets should be ambitious. FAA's goal is to reduce general aviation fatal accidents over the next ten years to no more than one accident per 100,000 flight hours.

Measure: Reduce the number of accidents in Alaska

Explanation:The total number of general aviation accidents and Part 135 accidents. NOTE: This measure includes ALL general aviation accidents, bothnot just fatal and non-fatalaccidents. This measure is NOT a subset of the Reduce General Aviation Fatal Accidents. The first baseline of 130, against which future targets were set, was established based on data from the years 2000 to 2002. The FAA plans to convert the current measure to a true rate-based measure in FY 2010. This rate will be based on accidents involving fatalities and serious injuries. This will shift from the focus away from the total number of accidents and allow us to target our safety initiatives on accidents that pose the greatest risk to the flying public. The use of a rate will also take into account fluctuation in activity levels from year to year caused by economic conditions, the price of fuel, weather conditions, and other variables.

Measure: Average cost of safety oversight for Part 145 repair stations

Explanation:This measure monitors the average cost of FAA's certification and surveillance activities for Part 145 domestic and foreign repair stations. Part 145 repair stations conduct maintenance, repair and renovations of aircraft and air carrier fleets. Located in the United States and worldwide, these repair stations vary greatly in size and scope. FAA conducts oversight of repair stations in two ways: by sending its own inspectors to review repair station operations and by making airlines and air cargo companies responsible for ensuring that repair stations are following proper procedures. This measure is computed as the total labor costs of FAA certification and surveillance activities for repair stations divided by the number of repair stations. FAA's Aviation Safety office will use this measure to evaluate total safety oversight spent on this critical sector of aviation. FAA plans to use the measure to benchmark regional performance and drive organizational efficiencies. The targets initially grow, but then decrease starting in 2010. The Aviation Safety office expects costs to rise in the short term, but then decrease as the organization begins to implement new efficiencies.

Is the program purpose clear?

Explanation: FAA's Mission Statement: 'FAA provides a safe, secure, and efficient global aerospace system that contributes to national security and the promotion of US aerospace safety.' US Code 49, Subtitle VII (Aviation Programs), Chapter 447 states, 'The Administrator of the Federal Aviation Administration shall promote safe flight of civil aircraft in air commerce...'

Evidence: USC 49, Subtitle VII, Aviation Programs. FAA Mission Statement - www.faa.gov/aboutfaa/Mission.cfmFAA Strategic Plan, pp. 6-10. See www.faa.gov/avr/FlightPlan/page56.cfm

Does the program address a specific and existing problem, interest or need?

Explanation: USC 49, Subtitle VII, § 44701 call for the FAA to provide regulations in order to promote safety and reduce and eliminate aviation accidents. The FAA continues to address safety improvements within the civil aviation industry. FAA is the sole certification authority for the United States civil aviation community. No other organization - public or private - exists that can do this job. From "A Brief History of the FAA'" - "The approaching introduction of jet airliners, and a series of midair collisions, spurred passage of the Federal Aviation Act of 1958. This legislation transferred CAA's [Civil Aeronautics Authority] functions to a new independent body, the Federal Aviation Agency, which had broader authority to combat aviation hazards. The act took safety rulemaking from CAB [Civil Aeronautics Board] and entrusted it to the new FAA. It also gave FAA sole responsibility for developing and maintaining a common civil-military system of air navigation and air traffic control, a responsibility CAA had shared with others."

Evidence: USC 49, Subtitle VII, § 44701. Federal Aviation Act of 1958"A Brief History of the Federal Aviation Administration and Its Predecessor Agencies" - www.faa.gov/aboutfaa/History_Brief.cfmAdministrator's Fact Book, December 2003, pp. 2-6 (see www.atctraining.faa.gov/factbook) details aviation and airspace accidents by type.Links to international organizations, treaty, etc.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: Yes, the program is a solely unique FAA function. FAA is the sole certification authority for the United States aviation community. No other Federal or non-Federal entity overlaps with the AVR mission.

Evidence: USC 49, Subtitle VII, § 44701. FAA Mission Statement - www.faa.gov/aboutfaa/Mission.cfmFAA Strategic Plan, pp. 6-10. See www.faa.gov/avr/FlightPlan/page56.cfm

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: There is no evidence that another approach or mechanism would be more efficient or effective to achieve the intended purpose. Grants or other direct federal programs would not be as efficient at achieving the safety record that the FAA has to date. The Federal Aviation Act of 1958 is designed to maximize net benefits.

Evidence: USC 49, Subtitle VII, § 44701

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: Field offices for the Regulation and Certification (AVR) program, including Flight Standards District Offices (FSDO's), Aircraft Certification Directorates, and others, are located near, or next to many of FAA's major customers - mfgs, airports. In addition, notices, rules and other AVR actions are sent directly to the AVR field offices and other customers.

Evidence: Flight Standards District Offices - http://www.faa.gov/avr/afs/fsdo/Aircraft Certification Service Offices - http://www.faa.gov/certification/aircraft/map.htm#TopFAA Regional Offices and Centers - http://www.faa.gov/aboutfaa/Regional.cfmFlight Standards Designees - http://www.faa.gov/avr/afs/index.cfm and http://afs600.faa.gov/default.htmAircraft Certification Designees - http://www.faa.gov/certification/aircraft/ Aerospace Medicine Designees - http://www.cami.jccbi.gov/aam-400/ameinfo.html and http://www.faa.gov/avr/aam/Game/Version_2/03amemanual/home/home.htm and http://www.faa.gov/avr/aam/order8520-2e.htm

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The Regulation and Certification (AVR) program has long-term performance measures that directly support the program's purpose.

Evidence: FAA Flight Plan ' www.faa.gov/avr/FlightPlan AVR Business Plan ' www.faa.gov/aboutfaa/BusinessPlans/AVR.pdf

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The Regulation and Certification (AVR) program has quantified targets and timeframes for the long-term measures.

Evidence: FAA Flight Plan ' www.faa.gov/avr/FlightPlan AVR Business Plan ' www.faa.gov/aboutfaa/BusinessPlans/AVR.pdf

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: Regulation and Certification (AVR) program has annual goals that directly link to DOT's long-term goals. These goals are quantifiable. Strategic plan includes long term goal through 2008. FAA has also developed an efficiency measure - Cost per Rule - that the Agency is currently baselining. FAA will use this measure to help reduce the cost in dollars and/or time of rulemaking, and to better target rulemaking resources to those rules and policies that are the most important.

Evidence: FAA Flight Plan ' www.faa.gov/avr/FlightPlan AVR Business Plan ' www.faa.gov/aboutfaa/BusinessPlans/AVR.pdf

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Regulation and Certification (AVR) program has developed baselines for each of its goals.

Evidence: FAA Flight Plan ' www.faa.gov/avr/FlightPlan AVR Business Plan ' www.faa.gov/aboutfaa/BusinessPlans/AVR.pdf Department of Transportation FY 2004 Performance Plan and Report ' www.dot.gov/PerfPlan2004/index.html

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: AVR and the industry worked together to develop the targets for its long-term goals. Industry also weighed in with comments on the FAA's Flight Plan (Five-year strategic plan)

Evidence: FAA Websites

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The International Civil Aviation Authority (ICAO) audited the FAA and found that 'the United States has a comprehensive and complete regulatory framework for safety oversight.' AVR has also received numerous audits by the GAO and Inspector General's offices - on topics such as Air Transportation Oversight System (ATOS) and Repair Stations.

Evidence: The ICAO assessment of FAA can be found at www.faa.gov/avr/iasa/ Air Transportation Oversight System -April 8, 2002 (AV-2002-88) Review of Air Carriers' Use of Airc raft Repair Stations -July 8, 2003 0 (AV-2003-47)

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The draft FY 2006 Congressional Justification request ties resource requirements to accomplishment of annual and long-term goals.

Evidence: Draft FY 2006 FAA Congressional Justification

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: AVR created a biweekly Dashboard to monitor performance against goals, as well as to identify problem areas that we can take corrective action on. In addition, the FAA approved an order for AVR to create an Integrated Planning Team to focus AVR's efforts on strategic and annual planning. FAA initiated a review in 2001 of major processes being used in the U.S. to certify, operate, and maintain commercial transport airplanes called the Commercial Airplane Certification Process Study (CPS) Study. This led to a Customer Service Initiative that gives customers the right to ask for review on any inspector's decision made in the regulatory or certification process without fear of retribution.

Evidence: FAA Flight Plan ' www.faa.gov/aboutfaa/FlightPlan.cfm AVR Business Plan ' www.faa.gov/aboutfaa/BusinessPlans/AVR.pdf www.faa.gov/avr/customerservice/index.cfm

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: AVR tracks the goals in the Flight Plan and AVR Business Plan monthly at a minimum. AVR tracks many other projects performance measures biweekly in the AVR Dashboard meetings. AVR regularly uses performance data to adjust. Examples include ATOS, which allows AVR to refocus inspectors where they are most needed based on data from industry and inspectors. The Aircraft Certification Systems Evaluation Program (ACSEP) collects annual data from industry on the ACSEP audit team and program performance. AVR works with the Commercial Aviation Safety Team to implement initiatives designed to mitigate or eliminate causal factors in commercial aviation accidents. AVR reviews NTSB accident data daily. The Safety Performance Analysis System (SPAS) is used by Flight Standards aviation safety inspectors to monitor the performance of certificate holders and to identify those that pose a greater-than-normal safety risk. The Program Tracking and Reporting Subsystem (PTRS) and Vital Information Subsystem (VIS) uses inspector and carrier data enhance out basic National Program Guidelines (NPG) by using a system safety approach.

Evidence: FAA Quarterly Performance Report ' www.faa.gov/aboutfaa/Performance.cfmFAA Performance and Accountability Report ' www.faa.gov/aba/html_fm/files_pdf/2003_PAR.pdf ATOS ' Do we have reports on web?CAST ' WebsiteACSEP ' www.faa.gov/certification/aircraft/ACSEP%20Program.htmSPAS ' WebsitePTRS/VIS - Website

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: FAA uses a Performance Management System (PMS) to tie pay to performance. This system exempts the Administrator from the government-wide GS system of pay. AVR senior management and core compensation employee's pay is tied to performance through Short-Term Incentive payments or Superior Contribution Increases, and/or Organizational Success Increase. Managers and employees are evaluated against service, AVR and FAA goals. AVR takes actions against airlines, manufaturers, and pilots who do not meet standards. AVR also increases scrutiny of airlines when airlines declare bankruptcy to ensure that safety standards are not cut when financial pressures may provide an incentive to cut corners.

Evidence: FAA Executive Compensation and Core Compensation Plans ' www.faa.gov/ahr/employee1.cfm www.faa.gov/ahr/pms/pms.cfm

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: The FAA Budget Office ensures that all program funds are obligated in a timely manner in accordance with the program plan. In addition, AVR develops and reviews a Quarterly Funding Plan. AVR was in violation last year of a immaterial process-related Anti-Deficiency Act violation. No money was spent that was not obligated. AVR and the FAA have provided training on apportionment rules to prevent any similar violation in the future.

Evidence: FAA Performance and Accountability Report ' www.faa.gov/aba/html_fm/files_pdf/2003_PAR.pdf

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: AVR is developing efficiency metrics, including Cost per Rule. Baselining will occur this year. The Cost per Rule measure used Labor Distribution Reporting (LDR) to record labor costs. Basically, the measure is the total amount of labor dollars spent divided by the number of rules. AVR is also tracking the costs and time for each rule. This measure is important because rulemaking is one of the FAA's most important functions that crosses organizational lines to complete. By using Cost Per Rule, FAA will be able to most efficiently apply its resources in the rulemaking process. FAA is currently baselining the measure, and will develop targets in October 2004.

Evidence: FAA Budget Documents

Does the program collaborate and coordinate effectively with related programs?

Explanation: Although AVR and the FAA are the only organizations that regulate the civil aviation industry, AVR does collaborate and coordinate effectively with other programs. AVR works internationally with other regulators and ICAO to harmonize safety standards. This harmonization effort reduced the number of rules carriers had to follow by comparing similar rules and regulations and selecting the safest to become the standard for both the FAA and other regulatory agencies. This created both a safer aviation environment while reducing costs to the aviation industry.

Evidence: Final harmonization rules completed include: Miscellaneous Flight Requirements; Powerplant Installation Requirements; Public Address System; Trim Systems and Protective Breathing Equipment; and Powerplant controls FAA-2002-13859, July 2, 2004). Electrical Equipment and Installations, Storage Battery Installation; Electronic Equipment; and Fire Protection of Electrical System Componenents on Transport Category Airplanes (FAA-2001-9634, FAA-2001-9633, FAA-2001-9638, FAA-2001-9637, March 16, 2004).

Does the program use strong financial management practices?

Explanation: FAA received a clean audit with no material weaknesses in FY 2003.

Evidence: FAA Performance and Accountability Report, pg. 48, Independent Auditors' Report ' www.faa.gov/aba/html_fm/files_pdf/2003_PAR.pdf Page

Has the program taken meaningful steps to address its management deficiencies?

Explanation: AVR reviews its performance biweekly in its Dashboard meetings. AVR also participates in monthly FAA Flight Plan review meetings. In addition, AVR has developed an Integrated Planning Team to more efficiently and effectively coordinate performance, planning and resource management across the organization. AVR has also responded positively to IG and GAO audits. Finally, AVR is undergoing ISO 9000 certification as a result of the Certification Process Study. The CPS study in 2001 was created to address the role that processes play in accident prevention. ISO 9000 certification is a way to standardize these processes. AVR has also developed a SWAT team to more quickly close out various rulemaking issues.

Evidence: AVR DashboardFAA Quarterly Performance Report ' www.faa.gov/aboutfaa/Performance.cfm AVR Planning Order ' Website when completedGAO/IG Audit ListISO 9000 ' Site or info? aia-aerospace.org/issues/subject/faa/faa_cert_study.pdf www.faa.gov/ipg/pif/iCMM/iCMMandISO9001-Final-25Feb2004.doc

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: ?? Commercial Air Carrier Fatal Accident Rate ' The FAA has achieved this goal every year since FY 2000. ?? General Aviation Fatal Accidents ' The FAA has achieved this goal from when we made this an official goal. ?? Alaska Accidents is a new goal that FAA began measuring against a ceiling this fiscal year. FAA is on track to meet long-term safety goals.

Evidence: FAA Performance and Accountability Report ' www.faa.gov/aba/html_fm/files_pdf/2003_PAR.pdf

Does the program (including program partners) achieve its annual performance goals?

Explanation: The FAA has met annual safety goals for its established performance goals, and is on track to achieve its annual performance goals for FY2004.

Evidence: FAA Performance and Accountability Report ' www.faa.gov/aba/html_fm/files_pdf/2003_PAR.pdf

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: AVR has not developed historic efficiency data

Evidence:  

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: AVR's standards are recognized as a 'gold standard' worldwide. Many ICAO safety standards were adopted from AVR. In addition, as a result of FAA's efforts, the United States has the second smallest rate of hull loss accidents from 1994 to 2003 (0.4 hull loss accidents per million departures.) Only the Australia/Oceana region had less.

Evidence: ICAO report at www.faa.gov/avr/iasa/ The model regs can be found at www.faa.gov/avr/iasa/calr.htm July 8, 2004 CAST Presentation - International Outreach

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Audits indicate that the AVR program is effective and achieving results but the IG and GAO still believe there is room for improvement. For example, IG and GAO staff have expressed concern about AVR's data quality and oversight of repair stations. AVR has undergone numerous audits over the past three years. Currently, AVR is undergoing nine audits with the IG/GAO. AVR is working to resolve the outstanding recommendations from the published audits.

Evidence: ICAO report at www.faa.gov/avr/iasa/ Air Transportation Oversight System -April 8, 2002 (AV-2002-88) Review of Air Carriers' Use of Airc raft Repair Stations -July 8, 2003 0 (AV-2003-47)