3.6. Major Automated Information Systems Economic Analysis
3.6.1. Introduction
3.6.2. Office of Cost Assessment and Program Evaluation Review Procedures
3.6.2.1. Kick-Off Meeting
3.6.2.2. Use of the Cost Analysis Requirements Description (CARD) for Major Automated Information System (MAIS) Programs
3.6.2.3. Office of Cost Assessment and Program Evaluation’s CARD Review and Assessment
3.6.1. Introduction
An automated information system (AIS) is a system of computer hardware, computer software, data and/or telecommunications that performs functions such as collecting, processing, storing, transmitting and displaying information; however, systems that are an integral part of a weapon or weapon system are excluded from this definition. AIS programs that meet the specified dollar thresholds in DoD Instruction 5000.02, Enclosure 3, Table 1, qualify as major automated information system (MAIS) programs. MAIS programs that are subject to review by OSD at the Defense Acquisition Board (DAB) are designated Acquisition Category (ACAT) IAM. Other MAIS programs, delegated to the head of the DoD Component or the appropriate DoD Component Acquisition Executive, are designated ACAT IAC. In some cases, an ACAT IA program also meets the definition of a Major Defense Acquisition Program (MDAP). In these cases, the Secretary of Defense may designate that the program be treated only as a MAIS program or only as a major defense acquisition program (MDAP). Generally, a program that requires the development of customized hardware shall be treated only as a MDAP, and a program that does not require the development of customized hardware shall be treated only as a MAIS program.
DoD Instruction 5000.02, Enclosure 4, Table 2-1, requires that an Economic Analysis be performed in support of the Milestone A, Milestone B, and full-rate production decision (or equivalent) reviews. The purpose of the Economic Analysis is to determine the best MAIS program acquisition alternative by assessing the net costs and benefits of the proposed MAIS program relative to the status quo. In general, the best alternative will be the one that meets validated capability needs at the lowest life-cycle cost (measured in net present value terms), and/or provides the most favorable return on investment.
Whenever an Economic Analysis is required, the DoD Component responsible for the program also may be required to provide a DoD Component Cost Analysis, which is an independent estimate of program life-cycle costs. Normally, the Economic Analysis is prepared by the MAIS program office, and the DoD Component Cost Analysis is prepared by an office or entity not associated with the program office or its immediate chain of command. The need for a DoD Component Cost Analysis at Milestone A is evaluated for each program in tailoring the oversight process.
The Economic Analysis should be accomplished in accordance with DoD Instruction 7041.3, "Economic Analysis for Decisionmaking." Normally, the DoD Component submits a Final Cost/Benefit Position that resolves the differences between the Economic Analysis and the Component Cost Analysis. Also, the Component and the MDA should address any differences between the Final Cost/Benefit Position and the funding in the current Future Years Defense Program.
In addition to an Economic Analysis, independent cost estimates are occasionally required for MAIS programs. Per 10 U.S.C. § 2445c, MAIS programs where the MDA is USD(AT&L) (ACAT IA) that experience critical program changes must undergo an independent cost estimate (ICE) prepared by the Director of Cost Assessment and Program Evaluation (DCAPE). ICEs will also be conducted for MAIS programs at any other time considered appropriate by DCAPE, or upon request by USD(AT&L) (see 10 U.S.C. § 2334). Additionally, DCAPE develops an ICE for MAIS Defense Business Systems when the Deputy Chief Management Officer or DoD Chief Information Officer is the MDA and a critical change, as defined in 10 U.S.C. § 2445c, has occurred.
3.6.2. Office of Cost Assessment and Program Evaluation Review Procedures
For Acquisition Category IAM programs, both the Economic Analysis and the DoD Component Cost Analysis are subject to independent review and assessment by the Director, Cost Assessment and Program Evaluation (DCAPE).
The purpose of the DCAPE’s assessment is to provide the Milestone Decision Authority with an independent determination that (1) the estimates of life-cycle costs and benefits are reasonable, traceable, and reflect DoD policy and DCAPE guidance on the consideration of life-cycle costs, (2) the return on investment calculation is valid, and (3) the cost estimates are built on realistic program and schedule assumptions.
During the review process, DCAPE staff may engage in discussion with the DoD Components regarding any discrepancies related to MAIS cost estimates and comment on deficiencies regarding the methodology or execution of cost estimates. Furthermore, DCAPE staff are authorized to concur with the choice of a cost estimate used to support the acquisition program baseline (APB) as well as in the selection of a proper confidence interval for the MAIS program.
DCAPE and the Secretary of the Military Department concerned are required by Congress to report certain elements of program cost risk for MAIS programs. For such programs, DCAPE and the Secretary of the Military Department concerned (or the head of the Defense Agency concerned) must state the confidence level used in establishing a cost estimate, the rationale for selecting the confidence level, and ensure that the confidence level provides a high degree of confidence that the program can be completed without the need for significant adjustment to program budgets.
The confidence level disclosure shall be included in the ADM approving the APB and in any other cost estimates for MAIS programs prepared in association with this section.
3.6.2.1. Kick-Off Meeting
The review process normally begins with a kick-off meeting held with DCAPE staff, representatives from the Major Automated Information System (MAIS) program office, the DoD Component Cost Analysis Team, and any DoD Component functional or headquarters sponsors. The purpose of the meeting is to reach a common understanding on the expectations for the upcoming activities and events leading to the Information Technology Acquisition Board milestone review. As a starting point, the DoD Component staff and/or sponsors' representatives should review the contents of the most recently approved capability needs documents, and explain any prior analysis (such as a Capabilities-Based Assessment) used to justify the need for a materiel solution (that will be met by the MAIS program).
At the kick-off meeting, the DoD Component staff and/or sponsors' representatives also should be prepared to explain the planned approach for the upcoming Economic Analysis. To facilitate this dialogue, the MAIS program office should prepare and provide a brief Economic Analysis development plan. The development plan should document the organizational responsibilities, analytic approach, ground rules and assumptions, and schedule for the economic analysis. The development plan should identify the specific alternatives that will be compared in the Economic Analysis. Normally, at least one alternative should be associated with the proposed MAIS program, and one alternative should be associated with the status quo (no modernization investment). It may well be the case that the status quo alternative represents an unacceptable mission posture—it may cost too much to sustain, be unable to meet critical capability needs, or be unsupportable due to technological obsolescence. Nevertheless, the status quo concept, applied over the same time frame (Life Cycle) as the proposed MAIS program, is used for comparative purposes in the Economic Analysis. The Economic Analysis development plan should document the DoD Component Cost Analysis approach and schedule as well.
As part of the Economic Analysis development plan, the program office should propose the cost element structure that will be used to organize and categorize cost estimates in the Economic Analysis. The cost element structure provides a hierarchal framework of defined cost elements that in total comprise the program life-cycle cost. The cost element structure should include phase-out costs associated with the status quo (legacy or predecessor) system. These costs would be incurred in managing, preserving, and maintaining the operations of the status quo system as it runs parallel to the phasing in of the new system. The status quo phase-out cost elements are not used in the estimate of the status quo alternative. A sample of a generic cost element structure is available from DCAPE staff. DCAPE can also provide advice on a consistent approach to net present value and return on investment computations.
3.6.2.2. Use of the Cost Analysis Requirements Description (CARD) for Major Automated Information System (MAIS) Programs
As soon as possible after the kick-off meeting, the draft Cost Analysis Requirements Description (CARD) is provided to DCAPE staff for review. The CARD is used to define and describe the MAIS program for purposes of preparing both the Economic Analysis and the DoD Component Cost Analysis. For a MAIS program, the CARD typically would address the following elements:
- Program description;
- Program operational concept;
- Program data management requirements;
- Program quantity requirements;
- Program manpower requirements;
- Program fielding strategy;
- Program milestone schedule; and
- Program acquisition plan or strategy.
Procedures for the preparation of the CARD are described in DoD Instruction 5000.02, Enclosure 7, paragraph 2. Additional guidelines on CARD preparation are found in DoD 5000.4 M, "DoD Cost Analysis Guidance and Procedures," Section 1. However, these guidelines are for the most part oriented toward weapon systems and may need to be tailored somewhat for automated information systems. The system description in the CARD should address both hardware and software elements. The CARD should describe each major hardware item (computers, servers, etc.), noting those items that are to be developed, and those items that are off-the-shelf. The CARD also should describe each software configuration item (including applications as well as support software) and identify those items that are to be developed. For software items to be developed, the CARD should provide (1) some type of sizing information (such as counts of source lines of code, function points, or Reports, Interfaces, Conversions and Enhancements (RICE)—Forms and Workflows (FW) (RICE-(FW) objects) suitable for cost estimating, and (2) information about the programming language and environment. In addition, the CARD should describe any special (physical, information, or operations) system security requirements, if applicable.
Clearly, much of the information needed for the CARD is often available in other program documents. The CARD should stand-alone as a readable document, but can make liberal use of appropriate references to the source documents to minimize redundancy and effort. In such cases, the CARD should briefly summarize the information pertinent to the Economic Analysis in the appropriate section of the CARD, and provide a reference to the source document.
3.6.2.3. Office of Cost Assessment and Program Evaluation’s CARD Review and Assessment
To facilitate the DCAPE review and assessment, the DoD Component's Economic Analysis and Cost Analysis teams should provide written documentation early enough to permit a timely report to the Overarching Integrated Product Team (OIPT) and Information Technology Acquisition Board. The timeline for document submission is the same as the timeline set forth in Section 3.4.3.1 for major defense acquisition programs. The documentation serves as an audit trail of source data, methods, and results. The documentation should be easy to read, complete and well organized to allow any reviewer to understand the estimate fully. The documentation also serves as a valuable reference for future cost analysts, as the program moves from one acquisition milestone to the next.
After review of the documentation, DCAPE staff provides feedback to the program office and DoD Component staff. Subsequently, DCAPE staff prepares a written report containing the findings of their independent assessment to the Milestone Decision Authority. Depending on the circumstances, the report may contain recommended cost and benefits positions, and it may raise funding or schedule issues. The expectation is that any issues raised have already emerged in prior discussions and that the final DCAPE report should not contain any surprises.
DAG Tutorial 
