Detailed Information on the
Air Force Depot Maintenance Assessment

Work to improve financial management of the depot maintenance program.

Use the program metrics to more closely align funding decisions for Air Force depot maintenance to the program outcomes.

Work with individual contractors to put in place Lean processes in order to meet the contract due date performance standard in FY2006

Implement Lean and Six Sigma at all our organic depots to help meet the defect rate goal for 2005.

Measure: Fighter Mission Capable Rate (F-15E). This measures the percent of aircraft that are capable of performing their designated mission. The Air Force defines mission capable standards by aircraft model. Mission capable rates for F-15Es are used as an example.

Explanation:This measures the percent of aircraft that are capable of performing their designated mission. The Air Force defines mission capable standards by aircraft model. Mission capable rates for F-15Es are used as an example.

Measure: Organic Due Date Performance - This measures the percent of maintenance actions performed in government depots that are completed on schedule.

Explanation:

Measure: Contract Due Date Performance

Explanation:

Measure: Organic Aircraft Quality Defect Rate (defects per aircraft) - This measures the number of defects per aircraft repaired in government depots.

Explanation:A lower measure is better

Measure: Contract Aircraft Quality Defect Rate (defects per aircraft)

Explanation:A lower measure is better

Is the program purpose clear?

Explanation: The Aircraft Depot Maintenance program provides for airframe, engine, and component rework to meet established Air Force readiness goals.

Evidence: The Department of Defense is required by 10 U.S.C. Sec. 2464 to perform organic maintenance on its materiel. DoD Directive 4151.18 states that "Maintenance programs are structured for meeting readiness and sustainability objectives (including mobilization and surge capabilities) of national defense strategic and contingency requirements."

Does the program address a specific and existing problem, interest or need?

Explanation: There is a need to ensure that all aircraft are properly maintained to perform the required missions of the Air Force and support to combatant commanders

Evidence: Technical Order 00-25-4 (Depot Maintenance of Aerospace Vehicles and Training Equipment) and the Maintenance Requirements Review Board Development and Review Process have laid out the technical reasons why Air Force aircraft need maintenance actions funded through the depot maintenance program.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: No other government agencies outside the Department of Defense provide for the maintenance of the Air Force's airframes, engines, and components.

Evidence: The Air Force has Title 10 responsibility for the maintenance of its materiel assets. While there is some similarity with Naval air depot activities, collaboration through the Joint Depot Maintenance Activity Group ensures that the program is not unnecessarily redundant or duplicative of other Federal programs.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The Air Force continuously reviews its maintenance procedures and models to increase their efficiency. Such processes determine workload requirements, costing of workload, and performance measurement. The Air Force Depot Maintenance program allows U.S. aircraft to operate in a high degree of readiness and has contributed to the Air Force's ability to project American air superiority around the world. Moreover, Air Force Depots are ISO 9001:2000 registered, which means that independent audits have found that they are managed according to the best practices used in the private sector.

Evidence: The Air Force has provided guidance that the program be funded sufficiently to prevent backlogs of equipment that would decrease materiel readiness and inhibit the warfighter's capabilities. The Air Force's recent accomplishments in Iraq and Afghanistan are testiments to the program's good design.

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: The program is well-designed. Maintenance of its aircraft is a high priority for the Air Force and funds are specifically targetted for it. Funds are programmed to needs by using the Maintenance Planning and Execution (MP&E) data system, which produces cost and schedule information. Additionally, the Office of the Secretary of Defense conducts a "mid-year" review to ensure that funds are being properly executed within the program.

Evidence: All funds in this program are contained within a sub-activity group (Congressional Special Interest). Congressional approval is required prior to moving more than $15M from this program funding line.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The Air Force has numerous performance measures that tie in with its long-term goals. Maintenance requirements are estimated across the Future Years Defense Plan (FYDP). Maintenance requirements are intended to give the Air Force aircraft a high mission capable rate.

Evidence: The Air Force maintains a MERLIN database that has readiness indicators for aircraft. There are also Black Book and Red Book metrics that contain annual performance data. Long term maintenance requirements are addressed in the Program Objective Memorandum (POM) and Defense Planning Guidance (DPG).

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The Air Force has very ambitious goals for its depot maintenance program, set out in the POM. It expects its aerial assets to have a high mission capable rate.

Evidence: See 2.1. Annual performance data are presented to the Air Force Chief of Staff in a quarterly review.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The Air Force has several annual performance measures that demonstrate progress toward achieving the program's long term goals. They include the aircraft mission capable rates as well as measures of the speed and quality of maintenance provided.

Evidence: The Air Force maintains a MERLIN database that has readiness Indicators for aircraft. There are also Black Book and Red Book metrics that contain annual performance data. Annual performance data are presented to the Air Force Chief of Staff in a quarterly review.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The Air Force depot maintenance program has very ambitious targets for its annual performance metrics. Goals vary by aircraft type, as each type has different maintenance requirements. For example, the baseline mission capable rate for fighter aircraft in FY 2001 was 73.9%

Evidence: The Air Force maintains a MERLIN database that has readiness Indicators for aircraft. There are also Black Book and Red Book metrics that contain annual performance data.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The Air Force works toward program goals with the other military services through the Joint Depot Maintenance Activity Group. Contractors are held accountable to work toward program goals via the Contractor Performance Assessment Reporting System. Annual performance data are presented to the Air Force Chief of Staff in a quarterly briefing.

Evidence: Contractors and other partners are expected to work toward the program goals. The ultimate responsibility for maintenance placed on the Air Logistics Center.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The GAO and Air Force Audit Agency (AFAA) conduct periodic audits of the Air Force's depot maintenance programs. Furthermore, Air Force Depots are registered as ISO 9001:2000 compliant, which requires independent audits to verify that their management meets best practices.

Evidence: Examples of recent audits include: Management Improvements Needed for Backlog of Funded Contract Maintenance Work and Financial Reporting of Deferred Maintenance Information on Air Force Weapons Systems for FY 2002.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Air Force budgets continually compare planned expenses to projected requirements, and where applicable refinements take place to ensure maximum cost effectiveness lies within the depot maintenance program. Budget requests can be traced to individual maintenance actions.

Evidence: Individual maintenance activities can be tied to the OP-30 exhibit in the Department's budget justification materials.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The Air Force has taken steps to reduce backlogs in contractor maintenance work by beginning a phased process to remove contract depot maintenance from the Working Capital Fund. This has allowed the Air Force to more accurately assess its end of the year carryover balances when budgeting for future years.

Evidence: The GAO has noted inconsistencies in the AIr Force's old business processes and particularly singled out backlogs in funded contract maintenance work as an impediment to sound budget planning. The Air Force has taken meaningful steps to resolve its deficiencies by changing the funding mechanisms for problem areas.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The Air Force depot maintenance program regularly collects information on performance and incorporates it into the strategic management of the depot maintenance program to improve efficiency.

Evidence: The Air Force maintains a MERLIN database that has readiness Indicators for aircraft. There are also Black Book and Read Book metrics that contain annual performance data. Annual performance data are presented to the Air Force Chief of Staff in a quarterly briefing.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: The maintenance of military materiel is an inherently governmental function. Partners are other geovernmental programs and contractors. The Air Force uses the Contractor Performance Assessment Reporting System to ensure that sufficient levels of accountabililty exist for cost, schedule, and performance results.

Evidence: The Air Force has several mechanisms to hold managers and partners accountable for performance results. Contract Maintenance Programs for Depot Maintenance Business Area documentation specifies that contracting Air Logistics Centers are responsible for contractor performance. Contractor performance is assessed using the Contractor Performance Assessment Reporting System (CPARS). This has resulted in performance awards to contractors that have surpassed performance expectations.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Funds are obligated in a timely manner. While some carryover of maintenance funds is inevitable, the Navy maintains this amount within Defense Department guidelines. The Office of the Secretary of Defense conducts a "mid-year" review to determine whether funds are being executed according to requirements. All funds are monitored to ensure the fiscal integrity of depot maintenance programs.

Evidence: The Air Force budget office receives monthly DD1002 reports on current year obligations. Obligation data are also presented in the Air Force's budget justification materials.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program performs cost benefit analyses and cost comparisons when determining where to source depot maintenance actions.

Evidence: When other organic depots can conduct repairs more efficiently than Air Force depots, the Air Force enters into Depot Maintenance Interservice Agreements for its repairs. Likewise, the Air Force uses competitive sourcing when determining whether contractors should perform depot-level maintenance. The Air Force is bound by 10 U.S.C. 2466 to expend no less than 50% of the funds allocated for depot-level maintenance at public sector depots.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The Air Force depot maintenance program collaborates with other services and agencies to maximize cost effectiveness for maintenance through the Joint Depot Maintenance Activity Group.

Evidence: Through the Joint Depot Maintenance Group, the Air Force enters into Depot Level Maintenance Interservice Support Agreements with other services if they can provide the same maintenance for Air Force materiel at a lower cost. An example is the Air Force's collaboration with the U.S. Army for the maintenance of H-1 helicopters.

Does the program use strong financial management practices?

Explanation: The Defense Department's financial management weaknesses are well-documented. While DoD continues to make efforts to improve them, the Department has yet to obtain an unqualified audit opinion. The Air Force does not have audit reports demonstrating that its depot maintenance program is free from internal weaknesses.

Evidence: Numerous audit reports document the Department's financial management weaknesses. Because of the magnitude of its problems, DoD is unlikely to obtain an unqualified audit for some time. GAO has specifically identified logistics and inventory control as one of the Department's weaknesses.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The program is implementing best management practices from the private sector. It is implementing Lean thinking to reduce waste and increase efficiency.

Evidence: The introduction of Lean thinking, Theory of Constraints, and Six Sigma have proven to increase efficiency in other services' depots. These management techniques have reduced cycle times and costs. The Air Force is also applying them to its depot programs and expects to realize efficiency gains. This should allow the Air Force to reduce defect rates and increase due date performance, which are captured in some of their performance metrics.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Annual mission capable rates for Air Force fighters have increase from 73.9 percent in FY 2001 to a projection of 75.6 percent in FY 2004.

Evidence: Mission capable rates are available in Depot Maintenance Performance Measures and Readiness Indicators in the MERLIN Database.

Does the program (including program partners) achieve its annual performance goals?

Explanation: Aircraft mission capable rates have improved since FY 2001 and public-sector depot maintenance is meeting performance goals for due date performance and defect rates. But depot maintenance contractors are not meeting goals set for due date performance and defect rates.

Evidence: Information regarding due date performance and defect rates are provided to the Air Force Chief of Staff in a quarterly briefing. The program is achieving its goals even though depot-level work requirements have doubled since 1991.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The Air Force continues to improve efficiency each year through process improvements (such as implementing Lean thinking) or alternate sourcing. The program has been more effective in achieving its goals since FY 2001.

Evidence: Requirements for aircraft depot maintenance vary from year to year and can change based on the operational tempo of Air Force assets. Since FY 2001 (the baseline) Air Force assets have operated at a high operational tempo due to military operations in Iraq and Afghanistan. In spite of the higher operational tempo, the depot maintenance program has improved the mission capable rates of the fighter fleet from 73.9 % in FY 2001 to 75.6% in FY 2004. For example, by introducing best management practice, the Oklahoma City Air Logistics Center has reduced the number of flow days needed to maintain KC-135s from 440 to 230 days increasing the due date performance rate.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The Air Force's aircraft are operating at a high level of mission capability, similar to those of the Navy. There are no other federal programs with a similar purpose.

Evidence: Air Force assets have achieved a high mission capable rate that compares favorably with assets of other services, such as the Navy. Maintenance indicators and processes are shared with similar programs to achieve similar results.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: The GAO and Air Force Audit Agency (AFAA) periodically audit the Air Force depot maintenance program. DOD routinely certifies Air Force assessments of how well their training, equipment, and personnel can meet the requirements of real-world military missions. This assessment of Air Force readiness is reported to Congress in classified form on a quarterly basis. Ongoing combat operations and these reports show that Air Force units are ready to meet their real-world combat missions.

Evidence: Audits have shown that the Air Force depot maintenance program faces some management challenges to increase its efficiency. Nonetheless, the program is successful at providing the Air Force with a mission capable fleet of aerial assets, as evidenced by the Air Force's performance in recent military actions.