The Analysis of Alternatives (AoA) is an important element of the defense acquisition process. An AoA is an analytical comparison of the operational effectiveness, suitability, and life-cycle cost (or total ownership cost, if applicable) of alternatives that satisfy established capability needs. Initially, after the Materiel Development Decision, the AoA is initiated to examine potential materiel solutions with the goal of identifying the most promising option, thereby guiding the Materiel Solution Analysis phase (see section 3.3.2). Subsequently, an update to the AoA is initiated at the start of the Technology Development Phase and is reviewed at Milestone B (which usually represents the first major funding commitment to the acquisition program). The update to the AoA is used to refine the proposed materiel solution, as well as reaffirm the rationale, in terms of cost-effectiveness, for initiation of the program into the formal systems acquisition process. For Major Defense Acquisition Programs at Milestone A, the Milestone Decision Authority (MDA) must certify in writing to the Congress that the Department has completed an AoA consistent with study guidance developed by the Director, Cost Assessment and Program Evaluation (DCAPE), in addition to meeting other certification criteria (10 U.S.C. § 2366a). For Major Defense Acquisition Programs at Milestone B, the Milestone Decision Authority (MDA) must certify in writing to the Congress that the Department has completed an AoA with respect to the program in addition to meeting other certification criteria (10 U.S.C. 2366b). Pursuant to DoDI 5000.02, the AoA is updated as needed at Milestone C.

In practice, AoA issues vary somewhat between AoAs for weapon and other tactical systems and AoAs for major automated information systems. Sections 3.3.2, 3.3.3, and 3.3.4 provide discussion about AoAs that may be of general interest, although much of the discussion is focused on weapon systems. Section 3.3.5 discusses the AoA process for major automated information systems.

3.3.2. Role of the Analysis of Alternatives (AoA) as Part of the Materiel Solution Analysis

The analysis of alternatives process is expected to play a key role in support of the Materiel Solution Analysis Phase. After a program has an approved Materiel Development Decision, the analysis of alternatives process is expected to contribute to the selection of a preferred materiel solution that satisfies the capability need documented in the approved Initial Capabilities Document (ICD).

The Director, Cost Assessment and Program Evaluation (DCAPE), develops and approves study guidance for the AoA. The guidance is developed with the input of other DoD officials. Prior to the MDD review, DCAPE provides the AoA study guidance to the DoD Component designated by the MDA. Following receipt of the AoA study guidance, the DoD Component prepares an AoA study plan that describes the intended methodology for the management and execution of the AoA. The AoA study plan is coordinated with the MDA and approved by DCAPE prior to the MDD review. A suggested template for the AoA study plan is provided in section 3.3.3.

The study guidance shall require, at minimum, full consideration of possible trade-offs among cost, schedule, and performance objectives for each alternative considered. The study guidance shall also require an assessment of whether or not the joint military requirement can be met in a manner that is consistent with the cost and schedule objectives recommended by the JROC. The AoA study guidance and resulting AoA plan should build upon the prior analyses conducted as part of the Joint Capabilities Integration and Development System (JCIDS). The JCIDS process is briefly described in section 1.3, and is fully described in CJCS Instruction 3170.01. The JCIDS analysis process that leads to an approved Initial Capabilities Document (ICD) is built upon the analysis known as the Capabilities-Based Assessment (CBA). The CBA provides recommendations (documented in the ICD) to pursue a materiel solution to an identified capability gap that meets an established capability need. The CBA does not provide specific recommendations as to a particular materiel solution, but rather provides a more general recommendation as to the type of materiel solution (such as Information Technology system, incremental improvement to an existing capability, or an entirely new "breakout" or other transformational capability). In this way, the ICD can be used to establish boundary conditions for the scope of alternatives to be considered in the subsequent AoA. The AoA study guidance should be crafted to provide a fair balance between focusing the AoA and ensuring that the AoA considers a robust set of novel and imaginative alternatives.

The final AoA supporting a Milestone A decision is provided to the DCAPE not later than 60 days prior to the milestone decision review meeting. The evaluation criteria to be addressed in this assessment are provided in DoD Instruction 5000.02, Enclosure 7, paragraph 5, and are discussed further in section 3.3.4.1.

3.3.2.1. Role of the Analysis of Alternatives (AoA) in Evolutionary Acquisition

The AoA is used to identify the most promising end-state materiel solution, but the AoA also can play a supporting role in crafting a cost-effective and balanced evolutionary acquisition strategy. The alternatives considered in the AoA may include alternative evolutionary paths, each path consisting of intermediate nodes leading to the proposed end-state solution. In this way, the Materiel Solution Analysis can help determine the best path to the end-state solution, based on a balanced assessment of technology maturity and risk, and cost, performance, and schedule considerations (as shown in Figure 3.3.2.1.F1). The rationale for the proposed evolutionary acquisition strategy would be documented as part of the Technology Development Strategy.

Figure 3.3.2.1.F1. Establishment of an Evolutionary Acquisition Strategy

3.3.3. Analysis of Alternatives (AoA) Study Plan

The first major step leading to a successful AoA is the creation and coordination of a well-considered analysis plan. The study plan should establish a roadmap of how the analysis will proceed, and who is responsible for doing what. At minimum, the study plan should facilitate full consideration of possible trade-offs among cost, schedule, and performance objectives for each alternative considered, as well as an assessment of whether or not the joint military requirement can be met in a manner that is consistent with the cost and schedule objectives recommended by the JROC.

A recommended outline for the AoA plan would resemble the following:

Introduction
- Background
- Purpose
- Scope
Ground Rules
- Scenarios
- Threats
- Environment
- Constraints and Assumptions
- Timeframe
- Excursions
Alternatives
- Description of Alternatives
- Nonviable Alternatives
- Operations Concepts
- Sustainment Concepts
Determination of Effectiveness Measures
- Mission Tasks
- Measures of Effectiveness
- Measures of Performance
Effectiveness Analysis
- Effectiveness Methodology
- Models, Simulations, and Data
- Effectiveness Sensitivity Analysis
Cost Analysis
- Life-Cycle Cost Methodology
- Additional Total Ownership Cost Considerations (if applicable)
- Fully Burdened Cost of Delivered Energy (if applicable)
- Models and Data
- Cost Sensitivity and/or Risk Analysis
Cost-Effectiveness Comparisons
- Cost-Effectiveness Methodology
- Displays or Presentation Formats
- Criteria for Screening Alternatives
Organization and Management
- Study Team/Organization
- AoA Review Process
- Schedule

Of course, every AoA is unique, and the above outline may need to be tailored or streamlined to support a given situation. Each point in the above outline is discussed further in the next several sections.

3.3.3.1. Analysis of Alternatives (AoA) Study Plan—Introduction

The introduction to the AoA plan describes the developments that led to the AoA, including prior relevant analyses (such as the Capabilities-Based Assessment. It should reference the applicable capability needs document(s) and other pertinent documents, and highlight the capability gaps being addressed through the applicable capability needs. The introduction should describe the applicable AoA study guidance and any other terms of reference. It also should provide a broad overview of the planned AoA that describes in general terms the level of detail of the study, and the scope (breadth and depth) of the analysis necessary to support the specific milestone decision.

3.3.3.2. Analysis of Alternatives (AoA) Study Plan—Ground Rules

The ground rules described in the analysis plan include the scenarios and threats, as well as the assumed physical environment and any constraints or additional assumptions. The scenarios are typically derived from defense planning scenarios and associated joint operational plans, augmented by more detailed intelligence products such as target information and enemy and friendly orders of battle. Environmental factors that impact operations (e.g., climate, weather, or terrain) are important as well. In addition, environment, safety, and occupational health factors associated with the use of chemical and/or biological weapons may need to be considered as excursions to the baseline scenario(s).

The study plan should describe what future timeframe, or timeframes, will be considered in the analysis. Often, the time period(s) selected will be determined by the time period(s) assumed in the DoD-approved planning scenario. However, there is some flexibility on this point, especially if something significant—such as the deployment of a new capability, or the retirement of a legacy system—is projected to occur one or two years after one of the time periods in the scenario. A common and desirable practice is to consider two time periods of interest, say "near-term" and "far-term," separated by a decade or so.

The AoA study plan should describe the planned analytic excursions to the baseline scenarios and other major ground rules. Such excursions are strongly encouraged in order to explore any impact of changing threat levels, warning times, involvement of allied forces, political constraints on basing or overflights, just to name a few issues. These excursions can be used to see if there any major issues that are critical to the relative cost-effectiveness of the alternatives considered in the AoA.

3.3.3.3. Analysis of Alternatives (AoA) Study Plan—Range of Alternatives

The analysis plan also should document the range of alternatives to be addressed in the analysis. In many cases, there will be a minimum set of alternatives required by the initial analysis guidance. Additional direction during subsequent AoA reviews may insert yet other alternatives. Practically, the range of alternatives should be kept manageable. Selecting too few or too many are both possibilities, but experience has shown that selecting too many, exceeding the available resources of the AoA study team, is the greater concern. The number of alternatives can be controlled by avoiding similar but slightly different alternatives and by early elimination of alternatives (due to factors such as unacceptable life-cycle cost or inability to meet Key Performance Parameters). In many studies, the first alternative (base case) is to retain one or more existing systems, representing a benchmark of current capabilities. An additional alternative based on major upgrades and/or service-life extensions to existing systems also may be considered.

For each alternative, evaluating its effectiveness and estimating its life-cycle cost (or total ownership cost, if applicable) requires a significant level of understanding of its operations and support concepts. The operations concept describes the details of the peacetime, contingency, and wartime employment of the alternative within projected military units or organizations. It also may be necessary to describe the planned basing and deployment concepts (contingency and wartime) for each alternative. The sustainment concept for each alternative describes the plans and resources for system training, maintenance, and other logistics support.

It is important that the alternatives considered in the AoA should address alternative concepts for maintenance, training, supply chain management, and other major sustainment elements. In this way, the AoA can identify the preferred materiel solution not only in terms of traditional performance and design criteria (e.g., speed, range, lethality), but also in terms of support strategy and sustainment performance as well. In other words, the AoA should describe and include the results of the supportability analyses and trade-offs conducted to determine the most cost-effective support concept as part of the proposed system concept.

3.3.3.4. Analysis of Alternatives (AoA) Study Plan—Effectiveness Measures

The analysis plan should describe how the AoA will establish metrics associated with the military worth of each alternative. Military worth often is portrayed in AoAs as a hierarchy of mission tasks, measures of effectiveness, and measures of performance. Military worth is fundamentally the ability to perform mission tasks, which are derived from the identified capability needs. Mission tasks are usually expressed in terms of general tasks to be performed to correct the gaps in needed capabilities (e.g., hold targets at risk, or communicate in a jamming environment). Mission tasks should not be stated in solution-specific language. Measures of effectiveness are more refined and they provide the details that allow the proficiency of each alternative in performing the mission tasks to be quantified. Each mission task should have at least one measure of effectiveness supporting it, and each measure of effectiveness should support at least one mission task. A measure of performance typically is a quantitative measure of a system characteristic (e.g., range, weapon load-out, logistics footprint, etc.) chosen to enable calculation of one or more measures of effectiveness. Measures of performance are often linked to Key Performance Parameters or other parameters contained in the approved capability needs document(s). Also, measures of performance are usually the measures most directly related to test and evaluation criteria.

3.3.3.5. Analysis of Alternatives (AoA) Study Plan—Effectiveness Analysis

The analysis plan spells out the analytic approach to the effectiveness analysis, which is built upon the hierarchy of military worth, the assumed scenarios and threats, and the nature of the selected alternatives. The analytic approach describes the level of detail at various points of the effectiveness analysis. In many AoAs involving combat operations, the levels of effectiveness analysis can be characterized by the numbers and types of alternative and threat elements being modeled. A typical classification would consist of four levels: (1) system performance, based on analyses of individual components of each alternative or threat system, (2) engagement, based on analyses of the interaction of a single alternative and a single threat system, and possibly the interactions of a few alternative systems with a few threat systems, (3) mission, based on assessments of how well alternative systems perform military missions in the context of many-on-many engagements, and (4) campaign, based on how well alternative systems contribute to the overall military campaign, often in a joint context. For AoAs involving combat support operations, the characterization would need to be modified to the nature of the support. Nevertheless, most AoAs involve analyses at different levels of detail, where the outputs of the more specialized analysis are used as inputs to more aggregate analyses. At each level, establishing the effectiveness methodology often involves the identification of suitable models (simulation or otherwise), other analytic techniques, and data. This identification primarily should be based on the earlier selection of measures of effectiveness. The modeling effort should be focused on the computation of the specific measures of effectiveness established for the purpose of the particular study. Models are seldom good or bad per se; rather, models are either suitable or not suitable for a particular purpose.

It also is important to address excursions and other sensitivity analyses in the overall effectiveness analysis. Typically, there are a few critical assumptions that often drive the results of the analysis, and it is important to understand and point out how variations in these assumptions affect the results. As one example, in many cases the assumed performance of a future system is based on engineering estimates that have not been tested or validated. In such cases, the effectiveness analysis should describe how sensitive the mission or campaign outcomes are to the assumed performance estimates.

3.3.3.6. Analysis of Alternatives (AoA) Study Plan—Cost Analysis

The AoA plan also describes the approach to the life-cycle cost (or total ownership cost (see section 3.1.5, if applicable) analysis. The cost analysis normally is performed in parallel with the operational effectiveness analysis. It is equal in importance as part of the overall AoA process. It estimates the total life-cycle cost (or total ownership cost) of each alternative, and its results are later combined with the operational effectiveness analysis to portray cost-effectiveness comparisons. What is important to emphasize is that the cost analysis will be a major effort that will demand the attention of experienced, professional cost analysts.

The principles of economic analysis apply to the cost analysis in an AoA. Although the cost estimates used in an AoA originally are estimated in constant dollars, they should be adjusted for discounting (time value of money), accounting for the distribution of the costs over the study time period of interest. In addition, the cost estimates should account for any residual values associated with capital assets that have remaining useful value at the end of the period of analysis. Further guidance on economic analysis is provided in DoD Instruction 7041.3, "Economic Analysis for Decisionmaking."

The cost analysis should also describe the planned approach for addressing the Fully Burdened Cost of Energy, for those AoAs where this issue is applicable. See section 3.3.4.1 for further information on this topic.

Further information on the recommended analytic approach for cost estimates is provided in section 3.7.

3.3.3.7. Analysis of Alternatives (AoA) Study Plan—Cost-Effectiveness Comparisons

Typically, the next analytical section of the AoA plan deals with the planned approach for the cost-effectiveness comparisons of the study alternatives. In most AoAs, these comparisons involve alternatives that have both different effectiveness and cost, which leads to the question of how to judge when additional effectiveness is worth additional cost. Cost-effectiveness comparisons in theory would be best if the analysis structured the alternatives so that all the alternatives have equal effectiveness (the best alternative is the one with lowest cost) or equal cost (the best alternative is the one with greatest effectiveness). Either case would be preferred; however, in actual practice, in many cases the ideal of equal effectiveness or equal cost alternatives is difficult or impossible to achieve due to the complexity of AoA issues. A common method for dealing with such situations is to provide a scatter plot of effectiveness versus cost. Figure 3.3.3.7.F1 presents a notional example of such a plot.

Figure 3.3.3.7.F1. Sample Scatter Plot of Effectiveness versus Cost

Note that the notional sample display shown in Figure 3.3.3.7.F1 does not make use of ratios (of effectiveness to cost) for comparing alternatives. Usually, ratios are regarded as potentially misleading because they mask important information. The advantage to the approach in the figure above is that it reduces the original set of alternatives to a small set of viable alternatives for decision makers to consider.

3.3.3.8. Analysis of Alternatives (AoA) Study Plan—Organization and Management

Finally, the AoA plan should address the AoA study organization and management. Often, the AoA is conducted by a working group (study team) led by a study director and staffed appropriately with a diverse mix of military, civilian, and contractor personnel. Program offices or similar organizations may provide assistance or data to the AoA study team, but (per DoD Instruction 5000.02, Enclosure 7) the responsibility for the AoA may not be assigned to a program manager, and the study team members should not reside in a program office. In some cases, the AoA may be assigned to an in-house analytic organization, a federally funded research and development center, or some other similar organization.

The AoA study team is usually organized along functional lines into panels, with a chair for each panel. Typical functional areas for the panels could be threats and scenarios, technology and alternatives (responsible for defining the alternatives), operations and support concepts (for each alternative), effectiveness analysis, and cost analysis. In many cases, the effectiveness panel occupies the central position and integrates the work of the other panels. The study plan also should describe the planned oversight and review process for the AoA. It is important to obtain guidance and direction from senior reviewers with a variety of perspectives (operational, technical, and cost) throughout the entire AoA process.

The analysis plan is fundamentally important because it defines what will be accomplished, and how and when it will be accomplished. However, the plan should be treated as a living document, and updated as needed throughout the AoA to reflect new information and changing study direction. New directions are inevitably part of the AoA process, and so the analysis should be structured so as to be flexible. Frequently, AoAs turn out to be more difficult than originally envisioned, and the collaborative analytical process associated with AoAs is inherently slow. There are often delays in obtaining proper input data, and there may be disagreements between the study participants concerning ground rules or alternatives that lead to an increase in excursions or cases to be considered. Experience has shown that delays for analyses dealing with Special Access materials can be especially problematic, due to issues of clearances, access to data, storage, modeling, etc. It is often common for the study director to scale back the planned analysis (or at least consider doing so) to maintain the study schedule.

3.3.4. Analysis of Alternatives Final Results

3.3.4.1. Analysis of Alternatives (AoA) Final Results and Assessment

Normally, the final results of the AoA initially are presented as a series of briefings. For potential and designated major defense acquisition programs (Acquisition Category (ACAT) I) and major automated information systems (ACAT IA), the final AoA results are provided to the Office of the Director, Cost Assessment and Program Evaluation (CAPE), no later than 60 days prior to the milestone decision meeting (Defense Acquisition Board or Information Technology Acquisition Board review). Providing emerging results to CAPE prior to the final briefing is wise to ensure that there are no unexpected problems or issues. For other programs, the AoA results should be provided to the DoD Component entity equivalent to CAPE, if applicable. In any case, the AoA final results should follow all of the important aspects of the study plan, and support the AoA findings with the presentation. In particular, all of the stated AoA conclusions and findings should follow logically from the supporting analysis.

Having received the final AoA briefing(s), the CAPE evaluates the AoA and provides an independent assessment to the Head of the DoD Component (or the Principal Staff Assistant) and to the Milestone Decision Authority. DoD Instruction 5000.02, Enclosure 7, provides the evaluation criteria for this assessment. According to the Instruction, the CAPE, in collaboration with the OSD and Joint Staff, shall assess the extent to which the AoA:

Illuminated capability advantages and disadvantages;
Considered joint operational plans;
Examined sufficient feasible alternatives;
Discussed key assumptions and variables and sensitivity to changes in these;
Calculated costs; and,
Assessed the following:

Technology risk and maturity;
Alternative ways to improve the energy efficiency of DoD tactical systems with end items that create a demand for energy, consistent with mission requirements and cost effectiveness; and
Appropriate system training to ensure that effective and efficient training is provided with the system.

The recommended template for the AoA study plan provided in Section 3.3.3 provides considerable guidance for conducting an AoA that would be responsive to the first five assessment criteria.

For the issue of technology risk and maturity, Section 3.3.2.1 provides a suggested approach where the AoA can help craft a cost-effective evolutionary acquisition strategy that is based on a balanced assessment of technology maturity and risk, as well as cost, performance, and schedule considerations.

For the issue of energy efficiency (applicable to tactical systems with end items that create a demand for delivered fuel or other forms of energy), Section 3.1.6 describes the analytic construct known as the Fully Burdened Cost of Delivered Energy; the Department now intends for this construct to play a major role in applicable AoAs.

For the issue of system training, the AoA should consider alternatives that provide for the individual, collective, and joint training for system operators, maintainers, and support personnel. The training system includes simulators and other training equipment, as well as supporting material such as computer-based interactive courseware or interactive electronic technical manuals. Where possible, the alternatives should consider options to exploit the use of new learning techniques, simulation technology, embedded training (i.e., training capabilities built into, strapped onto, or plugged into operational systems) and/or distributed learning to promote the goals of enhancing user capabilities, maintaining skill proficiencies, and reducing individual and collective training costs. Further information on system training is provided in Section 6.3.3. In addition to addressing the assessment criteria explicitly identified in DoD Instruction 5000.02, Enclosure 7, the AoA should also address alternative concepts for maintenance, supply chain management, and other sustainment elements (see Chapter 5 of this Guidebook).

3.3.4.2. Analysis of Alternatives (AoA) Final Report

Usually, in addition to a final briefing, the AoA process and results are documented in a written final report. The report typically is not published formally by the time of the program milestone decision review, due to schedule constraints. However, the report nevertheless may be important to the historical record of the program, since the report serves as the principal supporting documentation for the AoA. The report also may serve as a reference source for analysts conducting future AoAs. The final report can follow the same format as the study plan, with the addition of these sections:

Effectiveness Analysis
- Effectiveness Results
Cost Analysis
- Life-Cycle Cost (or Total Ownership Cost, if applicable) Results
Cost-Effectiveness Comparisons
- Cost-Effectiveness Results
- Assessment of Preferred Alternative(s)

By following the same format, much of the material from the (updated) study plan can be used in the final report.

3.3.5. Analysis of Alternatives (AoA) Considerations for Major Automated Information Systems (MAIS)

DoD Instruction 5000.02, Enclosure 4, Table 2-1 and Table 3, requires an AoA for MAIS programs at milestone decisions. Much of the discussion on AoAs provided in the earlier sections of the Guidebook is more applicable to weapon systems, and needs to be modified somewhat for MAIS programs. This section discusses AoA issues for MAIS programs. The AoA should include a discussion of whether the proposed program (1) supports a core/priority mission or function performed by the DoD Component, (2) needs to be undertaken because no alternative private sector or governmental source can better support the function, and (3) supports improved work processes that have been simplified or otherwise redesigned to reduce costs, improve effectiveness, and make maximum use of commercial off-the-shelf technology. The analysis should be tied to benchmarking and business process reengineering studies (such as analyses of simplified or streamlined work processes, or outsourcing of non-core functions).

For all MAIS program AoAs, one alternative should be the status quo alternative as used in the Economic Analysis, and one alternative should be associated with the proposed MAIS program. Other possible alternatives could be different system, network, and/or data architectures, or they might involve different options for the purchase and integration of commercial-off-the-shelf products, modifications, and upgrades of existing assets, or major in-house development.

Most likely, the effectiveness analysis in a MAIS program AoA will not involve scenario-based analysis as is common for the weapon system AoAs. The effectiveness analysis for an MAIS program should be tied to the organizational missions, functions, and objectives that are directly supported by the implementation of the system being considered. The results of the AoA should provide insight into how well the various alternatives support the business outcomes that have been identified as the business goals or capabilities sought. In some cases, it may be possible to express the assessment of effectiveness across the alternatives in monetary terms, and so effectiveness could be assessed as benefits in the framework for the Economic Analysis. In other cases, the effectiveness might be related to measurable improvements to business capabilities or better or timelier management information (leading to improved decision-making, where it can be difficult or impossible to quantify the benefits). In these cases, a common approach is to portray effectiveness by the use of one or more surrogate metrics. Examples of such metrics might be report generation timeliness, customer satisfaction, or supplier responsiveness. In addition to management information, the effectiveness analysis also should consider information assurance and interoperability issues.

The cost analysis supporting the AoA should follow the framework of the Economic Analysis. The life-cycle cost estimates of the alternatives considered in the AoA should be consistent with and clearly linked to the alternatives addressed in the Economic Analysis. Both the effectiveness analysis and the cost analysis should address the risks and uncertainties for the alternatives, and present appropriate sensitivity analysis that describes how such uncertainties can influence the cost-effectiveness comparison of the alternatives.

The appropriate sponsor or domain owner should lead the development of the AoA for a MAIS program. Experience has shown that the MAIS programs for which the sponsor or domain owner engages with the Office of the Director, Cost Assessment and Program Evaluation (CAPE) early in the process are much more likely to be successful than those that select a preferred alternative before contacting CAPE or before completing the AoA.

The DoD Component performing the AoA should develop a study plan that addresses the AoA study guidance, as applicable. At a minimum, the study plan should address the following topics:

AoA Study Plan – Outline

Introduction (Background, Purpose & Scope)
Ground Rules: Constraints and Assumptions
Description of Alternatives
Determination of Effectiveness Measures
1. Measures of Effectiveness (MOEs) – operationally relevant & measurable
2. Measures of Performance – technical characteristics required to satisfy MOEs and are measurable & employed as an operational test criteria
Effectiveness Analysis Methodology
Cost Analysis
Cost-Effectiveness Comparisons
Risk & Sensitivity Analysis
1. Mission
2. Technology
3. Programmatic, to include funding
Study Organization and Management
Schedule, with associated deliverables