Skip Navigation

This memorandum on the Department of Defense's Advanced Concept Technology Demonstration (ACTD) program was prepared by the Congressional Budget Office (CBO) in response to a request from the former Ranking Minority Member of the House National Security Committee. Jofi Joseph and Rachel Schmidt of CBO's National Security Division prepared the memorandum under the general supervision of Christopher Jehn and R. William Thomas. The authors would like to thank Delia Welsh for her research assistance and Evan Christman, Geoff Forden, Ray Hall, Arlene Holen, Eric J. Labs, Marvin Phaup, Philip Webre, Cindy Williams, and Lauri Zeman for their contributions. Thanks also to the Office of the Deputy Under Secretary of Defense for Advanced Technology and the program offices of various ACTD projects for providing information. However, full responsibility for the final product lies with the authors and CBO.

Leah Mazade edited the memorandum, and Melissa Burman proofread it. Cindy Cleveland prepared the report for publication, and Laurie Brown prepared the electronic versions for CBO's World Wide Web site (http://www.cbo.gov). Questions about the memorandum may be addressed to Rachel Schmidt.
 
 

SUMMARY

INTRODUCTION

• ACTDs and the Defense Acquisition Process
• How ACTDs Are Selected and Conducted
• What Happens After the ACTD?
• Transition Issues

THE ACTD PROGRAM BUDGET

• Spending by Year of Project Selection
• Sources of Funding for ACTDs
• Spending by Class of ACTD

POTENTIAL REWARDS AND RISKS OF ACTDs

• Possible Benefits of ACTDs
• Risks of ACTDs
• Conclusions

APPENDIXES

A - ACTDs Selected to Date
B - ACTDs by Class and Probable Exit Path
 

TABLES
1.	Transition Paths for ACTDs Completed by June 1998
2.	Funding for ACTDs and Other Categories of Defense Spending
A-1.	Summary Information for ACTDs Selected to Date
B-1.	ACTDs by Class and Probable Exit Path
FIGURES
1.	How ACTDs Fit into the DoD Acquisition Process
2.	Actual and Planned Spending on ACTDs, by Year of Project Selection
3.	Planned Size of ACTDs, by Year of Project Selection
4.	Actual and Planned Spending on ACTDs, by Source of Funding and Year of Project Selection
5.	Actual and Planned Spending on ACTDs, by Project Class and Year of Project Selection

 
 

SUMMARY

The Department of Defense's traditional approach to developing and building weapon and information systems has been criticized for taking too long, costing too much, and not adequately involving those who ultimately use the equipment. To address those problems, the Department of Defense (DoD) initiated the Advanced Concept Technology Demonstration (ACTD) program in 1994. As ACTDs were envisioned, military services or defense agencies would adapt new but mature technologies to build prototype equipment that met a critical military need. The systems would then go to a unified command or service for evaluation in the field. The ACTD project would last two to four years. After that time, the system would enter the formal acquisition process if larger quantities were needed.

From 1995 through 1998, DoD has spent $3.2 billion on 46 ACTDs. The $3.2 billion represents about 2 percent of DoD's entire budget for research and development during that time. If the Congress appropriates the amount requested by the Administration for the 1999-2003 period, DoD will spend an additional $1.7 billion on the ACTDs that it has already selected. (Because comparatively large projects were picked in 1995 and 1996, over two-thirds of the $4.9 billion total will support ACTDs chosen in the first two years.) The Administration also included $0.4 billion in its budget to support future projects that have not yet been selected.

Most ACTDs appear to be relevant to operations that combine the efforts of military services. Yet although several services may participate in an ACTD in some manner, only nine of the 46 projects selected thus far are financed by more than one service. Defense agencies have provided much of the funding for the other 37 ACTDs, perhaps because each military service may be reluctant to contribute to projects that reduce their individual autonomy. Support from defense agencies may also ensure that a project's funding will be more stable than it would be if it had to rely on the cooperation of more than one service.

Among the 46 ACTDs that have been funded, nearly half will continue as operational prototypes--DoD does not intend to move them into formal acquisition. Leaving an ACTD with its user/sponsor as a residual operational capability may be appropriate in instances in which multiple copies of a system are not needed. For example, U.S. forces stationed on the Korean Peninsula face a unique threat from North Korean artillery, which the Precision/Rapid Counter-MRL (Multiple Rocket Launcher) ACTD addressed. In such circumstances, developing and fielding a tailored capability through an ACTD may be quicker than using the formal acquisition process, which is designed to ensure that larger quantities of a weapon system are built with consistent quality. However, some components of those operational prototypes, such as software systems, are more broadly applicable and may become part of other technology projects used throughout DoD.

The ACTD process requires less analysis of the future costs of procurement, operation, and support than does formal acquisition because the primary focus in an ACTD is on demonstrating technologies quickly. By comparison, under the formal acquisition process, program managers for a new system must conduct studies of all likely "ownership" costs. The Office of the Deputy Under Secretary of Defense for Advanced Technology has issued guidelines suggesting that ACTD managers use integrated product teams to develop estimates of future costs. However, too few projects have been completed to know whether that process is working.

Planning for the continued operation of residual systems is also a concern. ACTDs may provide important military capabilities for operating forces, but little is known about the future ownership costs of those systems. The continuing challenge of ACTDs lies in balancing the goal of developing operational prototypes quickly against the need to plan for the cost of that equipment in later years.
 

INTRODUCTION

In 1994, the Clinton Administration began a program aimed at changing the way in which the Department of Defense (DoD) develops and fields certain military equipment. The initiative is made up of projects known as advanced concept technology demonstrations (ACTDs), which are designed to take relatively mature technologies and let battlefield commanders determine, in an operational setting, whether the new systems address their needs. The program grew out of the perception that the military's traditional approach to acquisition--a series of stages used to develop and procure equipment--had grown overly cumbersome and prevented DoD from exploiting the benefits of emerging technologies quickly. The rationale for ACTDs was that they would offer more flexibility, allowing battlefield commanders to evaluate a new concept over a two- to four-year period before DoD invested the time and money required for the formal acquisition process.

Over the 1995-1998 period, spending on the ACTD program--$3.2 billion, or slightly more than 2 percent--is a relatively small share of the $143 billion DoD has allocated to defense research, development, testing, and evaluation. Yet the program's proponents argue that its potential payback could be substantially greater. Some of the ACTDs are operational prototypes for what will ultimately become acquisition programs with costs many times greater than those of the demonstration project. Successful demonstration of a prototype may allow DoD to help control costs for those larger programs by evaluating the equipment earlier in the acquisition process.

That strategy is a break from DoD's historic approach to acquisition, and some critics find it problematic. They contend that the ACTD program allows DoD to circumvent legitimate oversight, which could lead to wasteful spending. Moreover, several of the projects now being carried out seem ill suited to the ACTD process. In addition, some analysts suggest that DoD has not been paying sufficient attention to the future costs of building, operating, and supporting the equipment developed during an ACTD.

Funding for the ACTD program is hard to identify because it is spread among the budgets of the military services, defense agencies, and the Office of the Secretary of Defense. This Congressional Budget Office (CBO) memorandum provides details about recent spending on ACTDs and enumerates some of the risks and potential benefits of the demonstration program.

ACTDs and the Defense Acquisition Process

DoD's traditional approach to developing and buying weapons is marked by four phases: exploring various weapon concepts, defining what the specific weapon system will look like, refining plans through engineering and manufacturing development, and then producing the equipment in larger quantities and operating and supporting it in the field (see Figure 1).⁽¹⁾ Before a program can proceed to the next stage of acquisition, defense officials review its progress to evaluate whether risk is under control, as measured by cost, schedule, and the ability to meet performance goals. Despite recent changes, the process remains fairly formal in order to maintain oversight of a system that guides many billions of dollars in federal spending each year.
 

FIGURE 1. HOW ACTDs FIT INTO THE DoD ACQUISITION PROCESS

SOURCE: Congressional Budget Office based on data from the Department of Defense.

NOTE: ACTDs = advanced concept technology demonstrations.

An ACTD offers defense planners flexibility by allowing them to conduct some of the steps of the formal process simultaneously. Managers of the demonstrations build prototype systems and let commanders evaluate the technology in the field. If those prototypes do not work well or are not as useful as other technological approaches, the project may be returned to the lab or canceled. If the ACTD provides an important capability but needs more work, officials can send the project to an appropriate point in the formal acquisition process--for example, engineering and manufacturing development or an early stage of production at low rates. After the ACTD is over, commanders in most cases retain the equipment that they used to demonstrate the technology's usefulness; that equipment is known as residual operational capability.

How ACTDs Are Selected and Conducted

The Office of the Deputy Under Secretary of Defense for Advanced Technology (hereafter referred to as the AT Office) selects new ACTDs each year, provides a small amount of budgetary support, and helps oversee each project. In the fall preceding the fiscal year in which a project is scheduled to begin, the AT Office issues a general request for proposals to the military services, theater commands, Office of the Joint Chiefs of Staff, and defense agencies. In the past, the AT Office has eliminated roughly one-half to two-thirds of the initial candidates through a review that determines whether an application fits the general parameters laid out for the ACTD program. (Those parameters include whether the proposal fills a critical military need, involves relatively mature technologies, and has a substantial possibility of effectiveness.) No more than about 25 applications survive that hurdle.

The proposals that remain are forwarded to the "Breakfast Club" and the Office of the Joint Chiefs of Staff for more evaluation. The Breakfast Club is a group of senior officials from the AT Office, the Office of the Director for Defense Research and Engineering, and the Deputy Assistant Secretary of Defense for Command, Control, Communications, and Intelligence; the club also includes representatives from the Office of the Joint Chiefs of Staff, the Defense Advanced Research Projects Agency, the Ballistic Missile Defense Organization, and the science and technology and operational requirements offices of each military service. This advisory group reviews each application and assesses the level of risk associated with the maturity of the technology and the project's management strategy. It also considers whether the program is affordable and whether the ACTD approach is suitable for that particular candidate technology. If the uncertainties about how a technology can be developed into a useful system are too great, they may outweigh other considerations because of the risk of wasting scarce budgetary resources.

The role of the Office of the Joint Chiefs of Staff and theater commanders in the ACTD selection process is to evaluate whether a candidate project meets an urgent military need. In 1995 and 1996, those assessments were communicated to the AT Office informally. The next year, the office introduced a more formal process by asking the Joint Requirements Oversight Council (JROC) to explicitly rank the ACTD candidates. (The JROC is made up of the Vice Chiefs of Staff of the Air Force and Army, the Vice Chief of Naval Operations, the Assistant Commandant of the Marine Corps, and the Vice Chairman of the Joint Chiefs of Staff. It is designed to give battlefield commanders more input into DoD's acquisition decisions.) The JROC bases its rankings on input from regional commanders, the military services, and Joint Warfare Capability Assessment teams set up by the Office of the Joint Chiefs of Staff to evaluate 10 aspects of the military's ability to fight conflicts. The JROC also receives information from the AT Office and may request additional briefings from applicants.

After receiving the Breakfast Club's input, the Deputy Under Secretary of Defense for Advanced Technology forwards the JROC's ranked list to be approved by the Under Secretary of Defense for Acquisition and Technology. The number of projects initiated each year depends on how much funding the Congress appropriates for the ACTD program and the extent to which selections from previous years need continued funding. The AT Office aims to announce its final ACTD selections in mid-October, after the new fiscal year has begun. That means that the Congress, which reviews DoD's funding requests for the upcoming year during the spring and summer, is asked to finance the ACTD program before the new projects for that year have been chosen. For ACTDs beginning in 1999, however, Congressional staff received information about candidates at an earlier point in the defense authorization and appropriation process.

A lead military service or defense agency manages each ACTD and acts as the key developer. Funding is usually, but not necessarily, provided by that lead group, as well as by other participating services or defense agencies and the AT Office. In addition, the ACTD has a user/sponsor--often the commander-in-chief of a unified command--whose role is to evaluate the technology and begin to develop a concept of operations for using it. (Appendix A lists all ACTDs selected to date, along with their lead service, user/sponsor, and proposed levels of funding.)

The AT Office requires that staff of each project prepare an implementation directive and a management plan, which are both intended to build consensus among key decisionmakers about the project's goals and participants' responsibilities. The implementation directive is a one- or two-page agreement that describes the military capability to be demonstrated during the ACTD, assigns responsibility for planning and conducting the demonstration, and provides approximate schedules and budgets. The directive must be completed before the AT Office provides funding for a project. The management plan outlines the ACTD's overall goals and strategy in more detail. Ideally, all major participants in an ACTD should sign the plan within 90 days of the project's start. In practice, however, the overwhelming majority of ACTDs have missed that deadline. (As of July 1998, for example, all parties had still not signed the management plans for four of the 12 projects selected in 1996.)

ACTDs generally last two to four years, but some take longer to complete. For the first one to two years, projects usually integrate component technologies and build the system, design appropriate software, and plan demonstrations of the prototype's capabilities. During the latter two years, the user/sponsor conducts one or more field demonstrations and evaluates the results. Typically, a demonstration is integrated with regularly planned exercises by a regional command. Managers of the Joint Countermine ACTD, for example, incorporated demonstrations of various mine detection and breaching technologies into two exercises by U.S. Atlantic Command forces--one at a joint force exercise at Camp Lejeune, North Carolina, in August 1997 and another at a combined Canadian and North Atlantic Treaty Organization exercise at Stephenville, Newfoundland, in June 1998.

What Happens After the ACTD?

After the ACTD is completed, the lead service or defense agency continues to fund the project for two years, during which any equipment built for the demonstration remains in the hands of its users. That follow-on period allows users to keep evaluating the system's capabilities and, if appropriate, provides time for the military service or defense agency to plan for the transition to formal acquisition.

Since only nine out of 46 ACTDs have been completed thus far, there is little experience by which to evaluate the transition process (see Table 1). In its literature about the initiative, the AT Office suggests that compared with traditional acquisition, ACTDs will move technologies into procurement more quickly. However, nearly half of the ACTDs initiated between 1995 and 1998 are expected to continue permanently as residual operational capability--that is, without moving into the formal acquisition process (Appendix B lists the expected ending status of all projects). Four such ACTDs have been completed; AT officials plan a similar route for 17 others. In addition to the Predator unmanned aerial vehicle and Consequence Management projects--the only completed ACTDs thus far to have entered procurement--the AT Office expects 15 other ACTDs to move into the formal acquisition process.
 

The ACTDs that are making the transition to formal acquisition merit attention because they could create a spike in the future demand for procurement dollars. In addition, the ACTDs that remain as residual operational capability are unique systems that will require support to keep operating. Logistics, sustainability, and training all play a role in ensuring that residual systems provide battlefield commanders with continued capability. Costs related to those factors can produce significant levels of spending over the service life of a system.

Transition Issues

One change in the ACTD process emerged from the military's experience with the Predator unmanned aerial vehicle, a 1995 selection. When the ACTD initiative first began, the AT Office did not provide guidance about when defense officials should select a project's lead service. In the case of the Predator, the AT Office and the JROC waited until late in the project to designate the Air Force as the lead, although that service had not participated in developing the system's concept of operations or in conducting its early demonstrations. The Air Force subsequently added new operational requirements for the Predator to suit its operational procedures. To try to prevent such development glitches in the future, in 1997 the AT Office began designating a lead service for a new ACTD at the start of the project.

Similarly, during 1995 and 1996, DoD did not require the military services or the defense agencies that sponsored ACTDs to provide funding for the two-year follow-on period. But it takes two years to develop DoD's annual budget request.⁽²⁾ As a result, a military service would need to reprogram funding from other priorities in order to quickly plan for and support an ACTD technology. In the case of the Predator, for example, the Air Force had to find $350 million within its six-year budget to support the program's operation, maintenance, personnel, and military construction costs.⁽³⁾

When DoD established the ACTD program in 1994, it did not include a formal approach to address any concerns that might arise about the transition stage of the projects. However, in December 1996, it published guidelines in the Defense Acquisition Deskbook for ACTDs involving military platforms (such as unmanned aerial vehicles), the category of systems that are most likely to move into formal procurement.⁽⁴⁾ Many of those recommendations were based on DoD's experience with the Predator program.

For ACTDs that began in and after 1996, the AT Office made one important change by requiring project managers to establish transition integrated product teams (IPTs) where applicable. Transition IPTs examine whether a program will be affordable, what the requirements for operating and supporting the residual capability will be, what strategies are needed for testing and evaluation, and whether there is adequate documentation for maintenance, training, or moving a system into procurement. (DoD has also incorporated the IPT approach in its formal acquisition process.)
 

THE ACTD PROGRAM BUDGET

Thus far, the Administration has spent an average of $800 million per year on ACTDs during the 1995-1998 period. A small share of that funding ($77 million in 1998) is provided by the Office of the Deputy Under Secretary of Defense for Advanced Technology. The rest comes from the defense agencies and military services.

The ACTD program is comparatively small (see Table 2). DoD's spending for science and technology, a category that includes funding for basic and applied research and certain technology demonstrations, is over nine times as large as the nearly $3.2 billion that DoD has spent on ACTDs since the program's inception.
 

Spending by Year of Project Selection

Of the $3.2 billion already appropriated for the ACTD program, two-thirds has gone toward projects initiated in 1995 and more than a quarter toward projects begun in 1996 (see Figure 2). Because DoD selected relatively large-scale projects during those first two years, they will continue to account for a large share of new appropriations. If the Congress funds the program at the levels requested by the Administration in 1998, total spending on ACTDs will amount to $5.4 billion over the 1995-2003 period. That figure includes $0.4 billion that the Administration has designated as the AT Office's contribution toward projects that have not yet been selected. Two-thirds of the $5.4 billion will support projects selected in 1995 and 1996.
 

FIGURE 2. ACTUAL AND PLANNED SPENDING ON ACTDs, BY YEAR OF PROJECT SELECTION

SOURCE: Congressional Budget Office based on data from the Department of Defense.

NOTE: ACTDs = advanced concept technology demonstrations.

The number of new projects approved each year has remained relatively constant. Eleven ACTDs were initiated in 1995, 12 began in 1996, nine started in 1997, and 14 were picked in 1998. Thus, the anticipated spending for each succeeding group of ACTDs has declined; the projects initiated in 1998 average about $50 million compared with $230 million for those started in 1995 (both figures are in 1998 dollars; see Figure 3).
 

FIGURE 3. PLANNED SIZE OF ACTDs, BY YEAR OF PROJECT SELECTION

SOURCE: Congressional Budget Office based on data from the Department of Defense.

NOTES: ACTDs = advanced concept technology demonstrations; UAV = unmanned aerial vehicle.

Sources of Funding for ACTDs

The military services are ultimately responsible for financing the larger-scale purchases of equipment that may result from an ACTD. Consequently, unless a military service feels vested in the new system being developed during the ACTD, even the most useful project may never move into the formal acquisition process. It is somewhat surprising, then, that collectively, DoD's defense agencies have provided a little more than half of the funding for the ACTD program, compared with about 40 percent from the military services combined (see Figure 4). For projects selected in 1997 and 1998, however, defense agencies provided proportionately less than before.
 

FIGURE 4. ACTUAL AND PLANNED SPENDING ON ACTDs, BY SOURCE OF FUNDING AND YEAR OF PROJECT SELECTION

SOURCE: Congressional Budget Office based on data from the Department of Defense.

NOTE: ACTDs = advanced concept technology demonstrations.

Defense Agency Support. DoD anticipates that the defense agencies will provide between 45 percent and 63 percent of the actual and planned funding for ACTDs selected in 1995, 1996, and 1997. That share falls to 30 percent for projects picked in 1998.

The Defense Advanced Research Projects Agency (DARPA) and the Defense Airborne Reconnaissance Office (DARO) have been major sources of funding for ACTDs. DARPA has provided a large share of the funding for eight projects, including the Battlefield Awareness and Data Dissemination, Joint Logistics, and Semi-Automated Imagery Intelligence Processing ACTDs. DARO owes its role in the ACTD initiative entirely to three unmanned aerial vehicle projects initiated in 1995 and 1996: the Predator, the high-altitude endurance program (which is developing two vehicles, the Global Hawk and the Darkstar), and the Outrider tactical unmanned aerial vehicle. DARO has been responsible for all of the funding for those three ACTDs--they receive no budgetary support from the military services or the AT Office.

Several other defense agencies also provide funding for ACTDs, albeit on a smaller scale than DARPA and DARO. They include the Ballistic Missile Defense Organization, the Defense Intelligence Agency, the Defense Special Weapons Agency, the National Imagery and Mapping Agency, and the Defense Information Systems Agency, among others.

Military Services. Although funding from defense agencies has been a key component of the ACTD program's support, the science and technology budgets of the military services also play an important role. Their contributions vary widely. The Army accounts for 28 percent of appropriated and planned funding for all ongoing ACTDs over the 1995-2003 period. By comparison, the Departments of the Navy and Air Force together represent 12 percent of the total.

Over the 1995-1998 period, two or more military services will support nine of the 46 ACTDs that have been selected. In addition, six other projects involve funding from a military service and one or more defense agencies.

Funding is just one indicator of interest in ACTDs--a military service can also assume responsibility for planning the transition of individual ACTDs to formal procurement. By that measure, the Army is or was the lead service for 17 ACTDs, the Navy has acted or currently acts as lead for 10 projects, the Marine Corps is or was the lead for two projects, and the Air Force, for 11. For some projects, a defense agency serves in that capacity. The services can also be involved by developing and reviewing internal proposals for ACTDs before sending them on to navigate the broader selection process run by the AT Office. All three military departments have established an internal review mechanism designed to raise candidate projects that they believe would be the strongest contenders in the DoD-wide competition.

Funding Provided by the AT Office. ACTDs draw relatively little funding from the Office of the Deputy Under Secretary of Defense for Advanced Technology.⁽⁵⁾ Nevertheless, the office's share of total ACTD spending has grown from 2 percent in 1995 to about 15 percent for projects selected in 1998.

AT Office funds are directed toward those activities that distinguish an ACTD from traditional research and development programs. Specifically, the office uses its support to integrate different technologies into a unified framework, to buy multiple versions of systems for the purpose of testing, and to provide technical support during the two-year follow-on period of a project.

Although the Congress has not provided as much funding as the Administration has requested, the AT Office's annual appropriation has increased steadily from its 1995 level of $32 million to $77 million for 1998. For 1999, the Administration has requested $116.3 million to support the AT Office. Eventually, as new projects debut and replace completed ones, the Administration hopes to reach a steady-state level of about $130 million (in 1998 dollars) for the ACTD program.

Spending by Class of ACTD

Defense officials divide ACTDs into three classes on the basis of their underlying technology and DoD's ultimate plans for the new systems. Class I ACTDs develop software systems that are designed to run on commercial workstations. The Advanced Joint Planning ACTD is an example: it integrated existing and new software so that the user, the U.S. Atlantic Command, could improve its command-and-control capabilities. That framework may then be transferred to other unified commands or to other information systems that are used more widely throughout DoD. Key issues specific to this group of ACTDs are verifying, validating, and accrediting software developed under the project as well as planning for long-term needs such as updating code for the systems and training users.

The 16 Class I ACTDs account for the smallest share of funding, representing only 20 percent of total actual and projected funding for projects selected thus far (see Figure 5).
 

FIGURE 5. ACTUAL AND PLANNED SPENDING ON ACTDs, BY PROJECT CLASS AND YEAR OF PROJECT SELECTION

SOURCE: Congressional Budget Office based on data from the Department of Defense.

NOTE: ACTDs = advanced concept technology demonstrations.

Class II ACTDs have much in common with traditional defense acquisition programs. Typically, they demonstrate a weapon platform or primary sensor that DoD will ultimately purchase in larger quantities. The most notable examples of Class II ACTDs are unmanned aerial vehicles: the Predator, the Global Hawk and Darkstar high-altitude endurance project, and the Outrider tactical vehicle. This category comprises 18 projects and accounts for 39 percent of actual and projected funding for all selected ACTDs.

Class III ACTDs integrate various technologies into a comprehensive framework, or "system-of-systems." Although just 12 projects fall into Class III, at 41 percent they make up the largest share of total funding. An example of a Class III project is the Joint Countermine ACTD. It incorporates Army, Navy, and Marine Corps countermine technologies into a framework that routes sensor data through a single software system to create a more complete overview of mine threats on the battlefield.
 

POTENTIAL REWARDS AND RISKS OF ACTDs

Analysts who study the defense acquisition process have found it difficult to formulate generalizations based on the experience of individual weapon systems because the technologies involved and the conditions under which they are developed vary a great deal.⁽⁶⁾ The same is true of ACTDs--their numbers are not large and they vary significantly in their size and scope. Moreover, very few of the projects have reached the stage at which they can be evaluated. For those reasons, this memorandum cannot definitively answer whether ACTDs are achieving the goal of saving time and money in the acquisition process. Instead, CBO enumerates here some of the possible rewards of ACTDs and, because of their implications for defense spending, some potential risks.

Possible Benefits of ACTDs

Advocates of the ACTD process believe that it can provide new military capabilities more quickly than the formal acquisition process and thereby save time and money. Several specific benefits are detailed below.

Greater Involvement of Users in Developing Systems. Critics of the traditional approach to developing weapons argue that those who will ultimately use the equipment play too limited a role; the process is largely handled by a separate community of program managers and engineers who design and build the systems. In particular, some analysts believe that too little input from battlefield commanders can lead to unrealistic expectations about what a system can do. ACTDs are designed to let users evaluate an early prototype in an operational setting and develop tactics for using the equipment.

The prospect of residual operational capability may offer a strong incentive for commanders to become involved in ACTDs. Agreeing to sponsor an ACTD means detailing personnel to participate in the project and, during joint exercises, dedicating time and resources to operational demonstrations. Since those resources might otherwise be used in mission training, they reflect a real cost and indicate the value commanders place on the capability.

Quicker Movement of Equipment to Battlefield Commanders. A related goal of ACTDs is to get mature technologies into the hands of users faster than the formal acquisition process allows. Because battlefield commanders keep the residual equipment that is developed during the ACTD, they thereby gain some additional military capability, albeit in limited numbers.

As one example, the Precision/Rapid Counter-MRL (Multiple Rocket Launcher) ACTD improved the ability of the user/sponsor, the U.S. Forces in Korea, to counter the threat posed by North Korea's 240-millimeter multiple rocket launchers. Probably the most difficult challenge of the ACTD was integration: combining the data provided by various sensors and quickly transmitting it to the guns, missiles, and planes that would return fire against a North Korean threat. Program managers credit the ACTD with reducing the average time of response to a simulated attack by enemy MRLs from 15 to 20 minutes to 3 to 4 minutes, while significantly improving the effectiveness of the response.

The Precision/Rapid Counter-MRL is a case in which battlefield commanders have kept residual operational capability without planning to acquire more systems through formal acquisition. That approach is appropriate because the prototype countered a threat specific to the circumstances facing U.S. forces near the Demilitarized Zone. In such a context, an ACTD may provide a quicker way to develop and field a unique capability than does the standard acquisition process, which is designed to ensure that larger quantities of a weapon system are built with consistent quality.

Lower Acquisition Costs. Proponents of the ACTD initiative argue that it will save time and money because users evaluate the equipment earlier in the development process. That evaluation might allow defense officials to identify and cancel less effective projects more quickly than they would in the formal acquisition process or to make changes in a system's design at less cost than that required to retrofit the first units off the production line. Some supporters also suggest that ACTDs could streamline any acquisition program that followed the demonstration by providing the same information as the program definition phase but in a more compressed time frame.⁽⁷⁾

The Predator medium-altitude unmanned aerial vehicle (UAV), which was designated an ACTD in 1995, is one of the two ACTDs that has moved into formal acquisition and procurement. As a demonstration project, the Predator lasted a total of 30 months, during which it was deployed twice to the Balkans. In August 1997, the acting Under Secretary of Defense for Acquisition and Technology formally approved the Predator program for production as an Air Force system. As the first completed ACTD to have led to procurement, the AT Office has learned from that experience.

Did the ACTD process shorten the time taken to develop the Predator and thereby save money for DoD? There is no clear answer. Project officials argue that because of the ACTD process, they were able to put a limited number of operational Predators into the hands of users more quickly than would otherwise have been possible. Compared with other UAV projects that have also been run as ACTDs, the Predator's technologies appear to have been relatively mature. The underlying air vehicle was based on the Gnat-750, a commercially available UAV, which may have made the Predator a good candidate for the ACTD approach.⁽⁸⁾

Advocates point out that by comparison with other UAV programs, the Predator ACTD was relatively inexpensive (the demonstration effort cost $128 million). Yet it is not clear that the acquisition process for the Predator will be completed any faster than it would have been as a formal acquisition program. The Air Force, which became the Predator's lead service late in the project period, had to backtrack to support the procurement process by preparing its operational requirements document and plans for maintenance, logistical support, and training.⁽⁹⁾ Moreover, the air vehicles built for demonstrations during the Predator ACTD differ significantly from the ultimate version that the Air Force is purchasing in larger quantities. Thus, it is not clear that DoD has saved time and money in this case by using the ACTD approach.

Limiting Requirements Creep. ACTDs may help to limit what is known as requirements creep--a phenomenon in which incremental demands are placed on a system as it goes through the acquisition process. Those additional demands can be problematic because they may lead to growth in costs and delay in the schedule. Requirements creep has posed a problem for UAV systems other than the Predator, such as the Army's Aquila.⁽¹⁰⁾

Managers of the Predator program believe that control of requirements creep was a clear outcome of running the project as an ACTD. Demonstrations may help to mitigate the problem because their compressed schedule does not offer as much opportunity to add requirements. Also, if users/sponsors agree to an operational concept for a prototype or a cap on costs early in the ACTD, there may be less room for revision later.

Nevertheless, ACTDs still allow the user to interact with the technology and suggest changes. In the case of the Predator, its extended deployment in Bosnia revealed at least two major problems: ice formed on the wings in cold weather, and the vehicle's sensors had difficulty "seeing" through dense fog. As a result, the Air Force, among other changes, is upgrading the Predator's de-icing features and modifying its radar to allow it to peer through heavy cloud cover.

Promotion of "Jointness." Supporters believe that ACTDs are a good vehicle for fostering joint projects among the military services (rather than leaving certain development programs in the hands of each service).⁽¹¹⁾ Yet defense agencies and the AT Office provide all of the funding for about half of the ACTD projects selected in 1995, 1996, and 1997. Part of DoD's rationale for relying on funding by the defense agencies to such an extent may stem from a perception that the services are reluctant to finance combined projects because they believe the result would limit their individual autonomy.

Some defense analysts argue that DoD has much to gain from integrating technologies that each military service already has on hand and from changing the structure of forces.⁽¹²⁾ Several ACTDs--especially Class III, or "system-of-systems," projects--aim to combine technologies as described in Joint Vision 2010, a publication of the Office of the Joint Chiefs of Staff. The document is a template that the office developed for integrating sensors, command-and-control systems, weapon platforms, and logistics centers more fully among the services.⁽¹³⁾

But such joint development also has its risks--for example, if cooperation leads to a system that has too many requirements because of the need to win budgetary support from several services. In such a situation, the requirements may ultimately be at odds with one another rather than promoting the system's operational effectiveness overall.⁽¹⁴⁾ By financing joint ACTDs through a defense agency rather than through the services, DoD may provide more stable funding and better identify those requirements that would make the new system most effective.

Risks of ACTDs

Because ACTDs involve less oversight than the formal acquisition process, an ever-present risk is that DoD's choices of projects might not be the best use of its scarce resources.

Questionable Project Selections. Some of the sharpest criticisms directed toward ACTDs thus far center on early project selections. Among the program's early critics was the Office of the DoD Inspector General, which found in a 1997 study that among nine ACTD projects it examined in detail, the majority were "questionable" choices.⁽¹⁵⁾ That criticism raised concern about the AT Office's approach to selecting ACTDs. Specifically, if the office picks technologies that are not mature enough, the ACTD framework may not permit officials to manage risk as well as the formal process would allow--which could lead to growth in costs.

Although the AT Office has subsequently addressed the Inspector General's recommendations, certain projects selected in the program's first two years have come under particular scrutiny as inappropriate:

• Although many observers consider the Predator UAV a successful demonstration, the High-Altitude Endurance UAV ACTD has been seen as less so because of the crash of a Darkstar prototype. Moreover, both the high-altitude endurance project and a 1996 selection, the Outrider tactical UAV, have experienced significant slips in their development schedules. The task of designing certain subsystems and then integrating them into a platform has proved difficult and has led some Members of Congress to question whether those systems were indeed ready for an accelerated demonstration program.⁽¹⁶⁾
  
  
• Another 1995 selection, the Low-Life-Cycle-Cost Medium-Lift Helicopter ACTD, compared the cost-effectiveness of leasing commercial helicopters with the costs of using military aircraft for support missions for the Military Sealift Command. Although the concept of purchasing resupply services for ships at sea is a new one, it is unclear why the project was carried out as an ACTD, given that it did not demonstrate a new technology.
  
  
• One 1996 ACTD, the Tactical High-Energy Laser, bypassed the selection process altogether. President Clinton and former Israeli Prime Minister Shimon Peres agreed to cooperate in developing a laser system to defend northern Israel against artillery attacks originating in Lebanon. Then Secretary of Defense William Perry subsequently created an ACTD to develop the tactical high-energy laser. The program never went through the formal selection process for ACTDs--it simply represented a command decision.

Ambiguous Criteria in the Selection Process. According to DoD, candidates for ACTDs must be sufficiently advanced that the project can explore the use of the technologies rather than develop their underlying know-how. The projects must also address urgent requirements identified by unified commanders. In the past, the AT Office's interpretation of those two parameters has led to criticism.

How do defense officials assess a technology's maturity in the formal acquisition process? At the start, the developers and future users of the equipment draw up an operational requirements document, which lists minimum levels of performance that a project must achieve on certain technical measures. For example, a project to develop a propulsion system might require a certain thrust-to-weight ratio, level of specific fuel consumption, and mean time between failures. Progress toward those thresholds gives an indication of a new system's risk.

During an ACTD, however, the demonstration of the combination of technologies takes place before commanders draw up an operational requirements document. In fact, one goal of involving users in the demonstration is to provide experience that will help battlefield operators decide which requirements are most important. ACTD managers have begun to draw up explicit measures of effectiveness and performance that they use to evaluate a technology during its demonstration. However, those evaluations are performed toward the end of the project rather than at the point when the AT Office is looking at proposals.

Initially, the AT Office did not issue guidelines for deciding whether a technology was mature, and as a result, different definitions emerged. For example, officials in the Army's science and technology community consider a project mature only if it can be readily put in the field. Their Navy and Air Force counterparts take a more flexible approach, citing as mature any technology currently in an Advanced Technology Development program (part of budget category 6.3) or lying outside the very early phases of the research and development (R&D) process. (DoD categorizes R&D funding as 6.1 through 6.7 to signify whether the work is closer to understanding underlying science [6.1], well down the road toward engineering a new piece of equipment [6.4], or modifying systems that are already being operated in the field [6.7].) Most sources of funding for ACTDs fall under categories 6.3 and 6.4; however, much of DARPA's early funding for ACTDs selected in 1996 fell under budget category 6.2, applied research. Until recently, the services have occasionally contributed a small amount of 6.2 funding toward certain projects.

In the past, one specific aspect of the technological maturity question has generated substantial criticism for the AT Office: the degree to which certain ACTDs use computer models either to simulate the capabilities of technologies that are not yet developed or to show how technologies will interact.⁽¹⁷⁾ That critique raises the issue of whether a project's technological maturity should be judged solely in terms of its component subsystems or also in terms of the risks associated with integrating those parts.

For candidate projects in 1998 and 1999, the AT Office declared that an ACTD's "core systems," or the technologies that are part of the project's "performance critical path," must be sufficiently mature when the project is being considered for funding; that is, they must be past the early 6.1 and 6.2 phases. The office also stated that by the time the user/sponsor begins to evaluate an ACTD's utility in its exercises, the project should have demonstrated technical feasibility--in other words, its integration tasks should have been successfully completed. That definition still leaves a loophole: blanket exceptions are allowed at the discretion of the Deputy Under Secretary for Advanced Technology. However, AT officials have yet to invoke any exceptions.

Despite the AT Office's attempts to address the issue of technological maturity, the subject remains problematic. Some defense officials argue that developing an explicit definition of it is virtually impossible; ACTDs simply incorporate too wide a range of technologies. Proponents also suggest that since the Breakfast Club is composed of officials who are well seasoned in defense acquisition, it possesses enough expertise to assess the technical maturity of candidate projects.

Another frequent criticism of the ACTD program is that at its start, defense officials did not establish clear procedures by which to evaluate whether candidate projects would meet an urgent military need. For 1995 and 1996, the AT Office was under no obligation to follow the Joint Requirements Oversight Council's recommendations in its final choice of projects. Among nine ACTDs reviewed by DoD's Inspector General, only four were identified "as having high potential in meeting the Joint Staff's need."⁽¹⁸⁾ The AT Office subsequently asked the JROC to rank candidates, and it now follows those rankings in its funding decisions. However, the JROC does not play a role in the initial screening of candidates to assess whether those proposals fit the general parameters laid out for the ACTD program.

Would military commanders prefer to have more influence in the selection process? Officials from the Office of the Joint Chiefs of Staff indicate that they are satisfied with their current level of involvement. DoD officials also contend that the initial screening of candidates by the AT Office centers on technical risk alone; thus, no reason exists for JROC involvement at that stage.

Too Little Oversight. Although ACTDs offer DoD flexibility in developing new systems, that freedom also means less oversight and the risk of a misuse of funds. The Congress has been especially sensitive to that danger: the conference report to the 1998 National Defense Authorization Act warned DoD not to use ACTDs as a means of evading the formal acquisition process.⁽¹⁹⁾

Some concerns have centered on the possibility that DoD might try to use R&D funding for procurement by building prototypes and leaving them in the field. For example, the House Appropriations Committee sharply criticized one ACTD--the Rapid Force Projection Initiative--for its plan to buy more than 250 advanced missiles. At the time, DoD defended the approach as a legitimate use of residual operational capability. The Committee, however, reprimanded the Army for bypassing many of the fundamental steps "in place to ensure DoD only procures safe, cost-effective, operationally suitable, and supportable weapons systems."⁽²⁰⁾ Following Congressional guidance, the AT Office issued a policy that DoD will not use R&D funding to buy any more equipment than is needed to support operational demonstrations.

Another question raised about ACTDs is whether DoD is using less stringent contracting procedures for the projects than it uses for other work. No single contracting strategy applies to all ACTDs, and there is no evidence that DoD has systematically used special authority to avoid oversight.⁽²¹⁾ However, some critics point to the case of the Predator, which differed in some respects from what might have happened if the prototypes had been developed in a formal acquisition program. For example, project managers did not write options for additional hardware purchases into the initial contracts with General Atomics, the Predator's developer--thereby neglecting a common procurement strategy that helps to avoid future negotiations over price after DoD has already committed the government to using a contractor as a sole source of production.⁽²²⁾ More recent guidance from the AT Office gives ACTD managers examples of contracting strategies to anticipate future needs, such as obtaining priced options or soliciting cost information from contractors today that would help DoD to negotiate future prices on development and production contracts.

Future Demands for Defense Spending. There is a paradox that comes into play when battlefield commanders decide that ACTD prototypes are worth buying in larger numbers. Because the ACTD initiative was designed to conduct operational demonstrations, ACTDs involve less regulation than formal acquisition. No one knows at the start whether DoD will ultimately buy the system; officials can always return a project to the lab or terminate it. To assume that project managers should plan for acquisition may violate the spirit of ACTDs. But without such planning, ACTDs could present a competing demand for funding during the last stages of preparing DoD's budget request.

Several solutions to this paradox have been proposed. Under one funding alternative, DoD would include a so-called wedge in its budget--an amount not yet distributed among specific projects that could be used to acquire systems that might be demonstrated successfully within one or two years. Another option would be to postpone further work on a project until funding could be incorporated into the next budget proposal--a two-year delay. Or, in a case in which the lead service was convinced of the importance of a new capability, service officials could include funding for acquisition earlier in an ACTD in anticipation of its success.

The AT Office suggests that each ACTD's management plan include some discussion of possible acquisition costs. In practice, however, few management plans contain such estimates, probably because so little is known about a project's future at its start. About halfway through an ACTD, new guidelines recently established by the AT Office call for a project's transition integrated product team to begin estimating how much funding may be needed for procurement. Yet such plans are necessarily sketchy, if they are present at all. In general, details about ongoing ACTDs are so incomplete that the Congressional Budget Office cannot arrive at an independent estimate of future acquisition costs.

Even when an ACTD moves into the formal acquisition process, some estimates of costs may still be lacking. DoD's procurement process requires program managers to analyze the type and cost of logistical support for a system in order to draw up estimates of its life-cycle costs. But in the case of the Predator--one of the two ACTDs that has moved into the formal system for acquisition--estimates were never made because the demonstration vehicles were pressed into use in Bosnia before planning was completed for their support, logistics, and training.⁽²³⁾ The contractor, General Atomics, provided flight operations manuals for piloting the vehicles but no repair manuals for maintenance personnel and no list of required spare parts. Moreover, project managers had no data on which to judge the system's basic reliability.⁽²⁴⁾ The managers did form a transition IPT to plan the Predator's move into formal acquisition, but the team tended to focus on meeting its schedule and on performance criteria rather than on issues related to costs.⁽²⁵⁾

ACTDs that continue permanently as residual operational capability could also create new demands for funding. Although each project covers the cost of developing and building residual systems, the expenses of operating them, purchasing spare parts, training the operators, and the like continue for as long as the equipment is used. Those costs fall to the user and sponsoring service or defense agency.

Little is known as yet about the magnitude of such costs. In the case of the Precision/Rapid Counter-MRL ACTD, project officials estimate that they will need about $700,000 annually to operate and support the software and workstations left with the U.S. Forces in Korea. That figure includes the cost of spare parts and technicians; in addition, other contractor personnel will be on hand to support the four annual exercises that the command plans to conduct, and project managers anticipate that they will also help to provide support. Military personnel who operate the residual equipment will receive training through their interaction with contractors as well as from the training materials and software developed during the ACTD.

The AT Office argues that establishment of its transition IPTs will lead to better planning for the costs of sustaining ACTD systems in future years. Given that those teams are made up of representatives from all relevant areas associated with an ACTD, that optimistic scenario may prove to be the case. But the fact that ACTD residual systems represent innovative technologies that have been fielded for only a very short time introduces uncertainty into such cost estimates.

The case of the Precision/Rapid Counter-MRL project may be instructive. As one of the first ACTDs, the project developed its plans for operating and supporting the technology on a more informal basis than recent guidelines would recommend. However, project officials contend that the lessons they learned from that experience are being used in a 1998 project run by the same program office (Theater Precision Strike Operations), with the result that officials are making plans earlier in the ACTD process.

Conclusions

Several of the risks associated with ACTDs became evident during the initiative's first two years. For example, critics point to several projects involving unmanned aerial vehicles as situations in which the AT Office selected ACTDs that had a high degree of technical risk associated with integrating their component technologies. In one instance, the Congress questioned whether the lesser degree of oversight for ACTDs had led to DoD's using R&D funds to procure a larger number of weapon systems than needed for operational demonstrations. Critics also note that ACTD project managers have devoted less time to planning for the future procurement, operation, and support costs of new equipment than would have been the case under the traditional acquisition process. The AT Office has addressed many of those concerns by emphasizing more planning for the transition of ACTDs to procurement and by making changes in its process for selecting and managing projects.

Yet so few ACTDs have been completed thus far that it would be premature to evaluate the initiative as a whole. Projects such as the Predator and the Precision/Rapid Counter-MRL appear to have achieved the goal of involving battlefield users more directly in the process of developing new systems, and they have also provided commanders with important new military capabilities quickly. But generalizing those results to other projects is questionable: those two ACTDs dealt only with small numbers of systems or with unique circumstances in which it was more appropriate to leave prototype equipment behind. Thus, it is impossible to say whether the ACTD initiative has led to savings of time and money for defense acquisition programs. On the one hand, the relative shortness of the Predator's development may have reduced the opportunity to add new requirements to the program for the procurement phase. On the other, the Air Force has had to backtrack in preparing the documentation it needs for formal procurement. The continuing challenge is to improve the ACTD process so that it balances the benefits of earlier input from battlefield users with the need to plan for the ownership costs of the new technologies.
 

APPENDIX A: ACTDs SELECTED TO DATE

The following table lists all of the projects in the Department of Defense's Advanced Concept Technology Demonstration program. The table includes information on total expected project costs and each demonstration's user/sponsor.
 

APPENDIX B: ACTDs BY CLASS AND PROBABLE EXIT PATH

Table B-1 shows the likely exit paths and class designations for the Department of Defense's advanced concept technology demonstrations.
 

1. DoD Directive 5000.1, "Defense Acquisition," and DoD Instruction 5000.2-R, "Mandatory Procedures for Major Defense Acquisition Programs and Major Automated Information Systems," both dated 1996, present guidelines for the acquisition process.

2. The process begins when the Secretary of Defense provides drafts of the Defense Planning Guidance, a document that gives broad direction about available resources and designates the responsibilities of each military service and defense agency. About six months later, in the spring, those organizations submit their program objectives memoranda, which detail their plans, to the Office of the Secretary of Defense (OSD) for review. After receiving input from OSD, the military departments and defense agencies then forward budget estimates to OSD in the fall of the first year. With some final changes, those numbers are incorporated in the President's budget, which is submitted to the Congress in February of the second year. The Congress reviews the proposal and appropriates funding before the start of the new fiscal year in October.

3. Statement of Lt. Gen. Brett M. Dula, Vice Commander, Air Combat Command, before the House Committee on National Security, April 9, 1997.

4. The Defense Acquisition Deskbook is a CD-ROM-based reference containing the DoD documents that guide the acquisition process. It is widely used by defense program managers.

5. The AT Office provides staff support directly to the Under Secretary of Defense for Acquisition and Technology. The recent Defense Reform Initiative proposes that the AT Office report to the Directorate for Defense Research and Engineering. Some defense officials contend that such a reorganization might be more efficient if, for example, the AT Office's staff was merged with that of the Advanced Technology Directorate. However, others are concerned that transferring authority for ACTDs to an organization that stresses basic research and development may diminish the ACTD program's emphasis on mature technology and military utility.

6. See, for example, Jeffrey A. Drezner, The Nature and Role of Prototyping in Weapon System Development, R-4161-ACQ (Santa Monica, Calif.: RAND, 1992).

7. John D. Moteff, DoD's Advanced Concepts and Technology Demonstrations, CRS Report for Congress 95-283SPR (Congressional Research Service, February 16, 1995).

8. Congressional Budget Office, Options for Enhancing the Department of Defense's Programs for Unmanned Aerial Vehicles, CBO Paper (forthcoming).

9. Michael Thirtle, Robert Johnson, and John Birkler, The Predator ACTD: A Case Study for Transition Planning to the Formal Acquisition Process, MR-899-OSD (Santa Monica, Calif.: RAND, 1997), pp. 69-70.

10. Statement of Louis J. Rodrigues, Director, Defense Acquisition Issues, National Security and International Affairs Division, before the Subcommittees on Military Research and Development and on Military Procurement of the House Committee on National Security, published as General Accounting Office, Unmanned Aerial Vehicles: DoD's Acquisition Efforts, GAO/T-NSIAD-97-138 (April 9, 1997), p. 3.

11. Statement of Thomas Perdue, Acting Deputy Under Secretary of Defense for Advanced Technology, before the Subcommittee on Research and Technology of the House Committee on National Security, February 27, 1997.

12. James R. Blaker, Understanding the Revolution in Military Affairs, Defense Working Paper 3 (Washington, D.C.: Progressive Policy Institute, January 1997).

13. Department of Defense, Office of the Joint Chiefs of Staff, Joint Vision 2010 (1995).

14. Stan Crock and Neal Sandler, "Pilotless Planes: Not Cleared for Takeoff?" Business Week (December 8, 1997), p. 108.

15. Office of the Inspector General of the Department of Defense, Advanced Concept Technology Demonstration, Audit Report 97-120 (April 1997), p. 7.

16. For a more detailed look at DoD's UAVs, see Congressional Budget Office, Options for Enhancing the Department of Defense's Program for Unmanned Aerial Vehicles.

17. Office of the Inspector General of the Department of Defense, Advanced Concept Technology Demonstration, p. 6.

18. Ibid., p. 11.

19. U.S. House of Representatives, National Defense Authorization Act for Fiscal Year 1998, conference report to accompany H.R. 1119, Report 105-340 (October 23, 1997), p. 675.

20. U.S. House of Representatives, Committee on Appropriations, Department of Defense Appropriations Bill, 1998, Report 105-206 (July 25, 1997), p. 168.

21. Under an extension of section 845 of P.L. 103-160, the defense authorization bill for fiscal year 1994, the military departments and other designated officials are granted relief from parts of the federal acquisition regulations. The Congress made that change to encourage more commercial companies to do business with DoD. Although justification for section 845 authority may be understandable, systematic use of such agreements for ACTDs might raise the issue of whether those projects were receiving sufficient oversight. CBO observed that only one ACTD selected thus far, the High-Altitude Endurance UAV, uses section 845 authority.

22. Thirtle, Johnson, and Birkler, The Predator ACTD, p. 42.

23. Statement of Lt. Gen. Brett M. Dula, April 9, 1997.

24. Thirtle, Johnson, and Birkler, The Predator ACTD, p. 46.

25. Ibid., p. 43.