This is the accessible text file for GAO report number GAO-08-263 
entitled 'Department of Homeland Security: Better Planning and 
Assessment Needed to Improve Outcomes for Complex Service Acquisitions' 
which was released on May 8, 2008.

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to Congressional Requesters: 

United States Government Accountability Office: 
GAO: 

April 2008: 

Department of homeland security: 

Better Planning and Assessment Needed to Improve Outcomes for Complex 
Service Acquisitions: 

GAO-08-263: 

GAO Highlights: 

Highlights of GAO-08-263, a report to congressional requesters. 

Why GAO Did This Study: 

The Department of Homeland Security (DHS) has relied on service 
acquisitions to meet its expansive mission. In fiscal year 2006, DHS 
spent $12.7 billion to procure services. To improve service acquisition 
outcomes, federal procurement law establishes a preference for a 
performance-based approach, which focuses on developing measurable 
outcomes rather than prescribing how contractors should perform 
services. 

GAO was asked to (1) evaluate the implementation of a performance-based 
approach in the context of service acquisitions for major, complex 
investments, and (2) identify management challenges that may affect 
DHS’s successful acquisitions for major investments, including those 
using a performance-based approach. 

GAO reviewed judgmentally selected contracts for eight major 
investments at three DHS components totaling $1.53 billion in fiscal 
years 2005 and 2006; prior GAO and DHS Inspector General reviews; 
management documents and plans; and related data, including 138 
additional contracts for basic services. 

What GAO Found: 

All service contracts for the eight major, complex investments GAO 
reviewed had outcome-oriented requirements; however, four of these 
contracts did not have well-defined requirements, a complete set of 
measurable performance standards, or both. These service contracts 
experienced cost overruns, schedule delays, or did not otherwise meet 
performance expectations. In contrast, service contracts for the other 
four investments GAO reviewed had well-defined requirements linked to 
measurable performance standards. Contractors had begun work on three 
of these four contracts and performed within budget meeting the 
standards. This finding is consistent with prior GAO work on service 
acquisitions, which has highlighted the criticality of sound 
acquisition planning to develop well-defined requirements and 
measurable performance standards to achieving desired outcomes. In the 
four cases that had negative outcomes, program officials identified the 
contractor performance weaknesses through quality assurance 
surveillance and took corrective actions. Prior GAO work has found that 
if acquisitions, including those that are performance-based, are not 
appropriately planned, structured, and monitored, there is an increased 
risk that the government may receive products or services that are over 
budget, delivered late, and of unacceptable quality. 

In managing its service acquisitions, including those that are 
performance based, DHS has faced workforce and oversight challenges. 
Prior GAO work has highlighted the importance of having the right 
people with the right skills to achieve successful acquisition 
outcomes. Contracts for two major investments with negative cost and 
schedule outcomes did not have the staff needed to adequately plan and 
execute the contracts. Further, while representatives for several of 
the contracts GAO reviewed indicated that contracting and program staff 
worked well together, some senior acquisition representatives at the 
component level indicated that a lack of collaboration between these 
key stakeholders has been a challenge when developing and managing 
complex service acquisitions. In terms of oversight, component 
contracting and program officials said they used a performance-based 
approach to the maximum extent practicable; however, DHS does not have 
reliable data to facilitate required reporting, informed decisions, and 
analyzing acquisition outcomes. GAO’s review also found that about half 
of an additional 138 contracts for basic services identified as 
performance-based did not have any of the elements intended to foster 
good outcomes: a performance work statement, measurable performance 
standards, and a quality assurance surveillance plan. DHS’s Chief 
Procurement Officer (CPO)—who is responsible for departmentwide 
oversight of acquisitions—has several efforts under way to address some 
of these workforce and oversight issues. One initiative is an 
acquisition oversight program that is intended to assess (1) compliance 
with federal acquisition guidance, (2) contract administration, and (3) 
business judgment. However, this oversight program has not yet included 
an evaluation of the outcomes of contracting methods such as 
performance-based service acquisition. 

What GAO Recommends: 

DHS generally concurred with GAO’s recommendations that DHS develop 
measurable standards consistently linked to well-defined requirements, 
evaluate acquisition outcomes for major investments, and improve data 
quality to help identify and assess contracting methods and outcomes. 

To view the full product, including the scope and methodology, click on 
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-263]. For more 
information, contact John Hutton (202) 512-4841 or huttonj@gao.gov. 

[End of section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

Definition of Requirements and Performance Standards Influenced 
Contract Outcomes: 

DHS Has Faced Workforce and Oversight Challenges in Managing Its 
Service Acquisitions: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Scope and Methodology: 

Appendix II: Comments from the Department of Homeland Security: 

Appendix III: GAO Contact and Staff Acknowledgments: 

Tables: 

Table 1: Key Characteristics of Eight Performance-Based Service 
Contracts: 

Table 2: Less Than Expected Performance Identified through Surveillance 
and Action Taken: 

Table 3: Review of Performance-Based Elements on Selected Contracts: 

Figure: 

Figure 1: Percent of Contracts and Obligations by Contract Type for 42 
DHS Performance-Based Service Acquisitions (Fiscal Years 2005 and 
2006): 

Abbreviations: 

ACE: Automated Commercial Environment: 

CBP: Customs and Border Protection: 

CPO: Chief Procurement Officer: 

DHS: Department of Homeland Security: 

FAR: Federal Acquisition Regulation: 

FEMA: Federal Emergency Management Agency: 

FPDS-NG: Federal Procurement Data System-Next Generation: 

ICE: Immigration and Customs Enforcement: 

NPI: National Prime Integration: 

OFPP: Office of Federal Procurement Policy: 

SBInet: Secure Border Initiative Network: 

TSA: Transportation Security Administration: 

TWIC: Transportation Worker Identification Credential: 

[End of section] 

United States Government Accountability Office:
Washington, DC 20548: 

April 22, 2008: 

The Honorable Joseph I. Lieberman: 
Chairman: 
The Honorable Susan M. Collins: 
Ranking Member: 
Committee on Homeland Security and Governmental Affairs: 
United States Senate: 

The Honorable Daniel K. Akaka: 
Chairman: 
The Honorable George V. Voinovich: 
Ranking Member: 
Subcommittee on Oversight of Government Management, the Federal 
Workforce and the District of Columbia: 
Committee on Homeland Security and Governmental Affairs: 
United States Senate: 

The Honorable Bennie G. Thompson: 
Chairman: 
Committee on Homeland Security: 
House of Representatives: 

In fiscal year 2006, the Department of Homeland Security (DHS) spent 
$12.7 billion on services, representing more than 80 percent of its 
total procurement spending for that year. DHS has relied on service 
acquisitions to meet its expansive homeland security mission. However, 
prior GAO work has found that if acquisitions are not appropriately 
planned, structured, and monitored, there is an increased risk that the 
services provided will not fulfill intended acquisition outcomes or, 
ultimately, meet government needs.[Footnote 1] 

To help improve service acquisition outcomes, federal procurement 
policy calls for agencies to use a performance-based approach to the 
maximum extent practicable. This approach includes: a performance work 
statement that describes outcome-oriented requirements, measurable 
performance standards, and quality assurance surveillance. These 
characteristics, if properly implemented, can help ensure that the 
services meet cost, schedule, and performance requirements. Since this 
concept was introduced in the 1990s, it has become widely accepted as a 
sound acquisition approach. 

Despite its governmentwide acceptance, concerns have been raised about 
how well agencies are using a performance-based approach, particularly 
for complex or major investments. Given the significant amount of DHS 
spending on services and its critical mission, you asked us to (1) 
evaluate the implementation of a performance-based approach in the 
context of service acquisitions for major, complex investments; and (2) 
identify management challenges that may affect DHS's successful 
implementation of service acquisitions for major investments, including 
those using a performance-based approach. 

To conduct our work, we focused on the Coast Guard, Customs and Border 
Protection (CBP), Immigrations and Custom Enforcement (ICE), and the 
Transportation Security Administration (TSA)--four of the five DHS 
components reporting the highest obligations for performance-based 
service acquisitions in fiscal years 2005 and 2006.[Footnote 2] To 
evaluate DHS's implementation of a performance-based approach in the 
context of service acquisitions for major investments,[Footnote 3] we 
judgmentally selected eight major investments at the Coast Guard, CBP, 
and TSA[Footnote 4] based on size and complexity, and reviewed actions 
for one performance-based service contract associated with each 
investment.[Footnote 5] These contracts had a combined estimated value 
of $1.53 billion and included services provided in the acquisition and 
sustainment phases of the investments.[Footnote 6] We assessed these 
contracts against acquisition regulations and policy and interviewed 
contracting and program management staff. Performance-based service 
acquisitions were difficult to identify and were not clearly distinct 
from other service acquisitions because they did not always include all 
required elements. Therefore, we reviewed prior GAO reports and DHS 
Inspector General reports to provide a broader context on DHS's major 
service acquisitions. 

To identify management challenges that may affect DHS's successful 
implementation of service acquisitions for major investments, we 
analyzed management documents and plans; staffing data; and oversight 
mechanisms, including the Federal Procurement Data System-Next 
Generation (FPDS-NG), the governmentwide database for procurement 
spending. From FPDS-NG, we judgmentally selected 138 contracts for 
primarily basic services coded as performance-based and awarded in 
fiscal years 2005 or 2006 at Coast Guard, CBP, ICE, and TSA with 
obligations greater than $1 million dollars and asked the components to 
verify whether these contracts included the required performance-based 
elements. Finally, we interviewed contracting and program management 
staff and DHS Chief Procurement Office (CPO) representatives and met 
with representatives from the Office of Management and Budget's Office 
of Federal Procurement Policy (OFPP). For more information on our scope 
and methodology, see appendix I. 

We conducted this performance audit from January 2007 to April 2008 in 
accordance with generally accepted government auditing standards. Those 
standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. 

Results in Brief: 

All service contracts for the eight major, complex investments we 
reviewed had outcome-oriented requirements, but the requirements were 
not always well-defined. Four of the eight contracts we reviewed did 
not have well-defined requirements, or a complete set of measurable 
performance standards, or both at the time of contract award or start 
of work. These service contracts experienced cost overruns, schedule 
delays, or did not otherwise meet performance expectations. In 
contrast, service contracts for the other four investments had well- 
defined requirements linked to measurable performance standards. 
Contractors had begun work on three of these four contracts and 
performed within budget meeting the standards. This finding is 
consistent with our prior work on service contracting, which has 
highlighted the criticality of sound acquisition planning to develop 
stable, well-defined requirements and measurable performance standards 
to achieving desired outcomes. Components conducted quality assurance 
surveillance for the contracts we reviewed. For the four contracts that 
had negative outcomes, surveillance helped DHS components to identify 
contractor performance weaknesses and corrective actions were taken. 
Prior GAO work has found that if acquisitions, including those that are 
performance-based, are not appropriately planned, structured, and 
monitored, there is an increased risk that the government may receive 
products or services that are over budget, delivered late, and of 
unacceptable quality. 

In managing its service acquisitions, including those that are 
performance-based, DHS has faced workforce and oversight challenges. 
Our prior work has highlighted the importance of having the right 
people with the right skills to achieve successful acquisition 
outcomes. Contracts for two of the eight major investments we reviewed 
that had negative cost or schedule outcomes did not have the staff 
needed to adequately plan and execute the contracts. Further, while 
representatives for several of the contracts we studied indicated 
contracting and program staff worked together effectively, at the 
component level, senior acquisition representatives indicated that a 
lack of collaboration between the program and contracting offices in 
general has been challenging. In terms of oversight, DHS component 
contracting and program staff told us they used a performance-based 
approach to the maximum extent practicable; however, the department 
does not have reliable data to facilitate reporting or perform 
management assessments of these acquisitions. Moreover, while the CPO 
has implemented a departmentwide acquisition oversight program, the CPO 
has not performed management assessments of performance-based service 
acquisitions. Our review of an additional 138 contracts for primarily 
basic services found that about half of the contracts identified as 
performance-based in FPDS-NG did not have any of the three elements: a 
performance work statement, measurable performance standards, and a 
quality assurance surveillance plan, which are intended to foster good 
outcomes. Inaccurate data limit DHS's visibility over service 
acquisitions and the department's ability to make informed acquisition 
management decisions. DHS's CPO has several efforts under way to 
address departmentwide workforce and oversight challenges. One 
initiative is an acquisition oversight program that is intended to 
assess (1) compliance with federal acquisition guidance, (2) contract 
administration, and (3) business judgment. However, this oversight 
program has not yet included an evaluation of the outcomes of 
contracting methods such as performance-based service acquisition. 

To increase DHS's ability to improve outcomes for its service 
acquisitions, including those that are performance-based, we are 
recommending that the Secretary of Homeland Security take several 
actions. These actions include improving acquisition planning for 
requirements for major, complex investments to ensure they are well- 
defined, and developing consistently measurable performance standards 
linked to those requirements; systematically evaluating the outcomes of 
major investments and relevant contracting methods at a departmentwide 
level; and improving the quality of FPDS-NG data so that DHS can more 
accurately identify and assess the use and outcomes of various 
contracting methods. In written comments on a draft of this report, DHS 
generally concurred with our recommendations and provided some 
information on efforts under way to improve acquisition management. The 
department's comments are reprinted in appendix II. OMB did not comment 
on the findings or conclusions of this report. DHS and OMB each 
provided technical comments, which we incorporated as appropriate and 
where supporting documentation was provided. 

Background: 

Over the last decade, the use of federal service contracting has 
increased and now accounts for over 60 percent of federal procurement 
dollars spent annually. Prior GAO work has found that if acquisitions 
are not appropriately planned, structured, and monitored, there is an 
increased risk that the services provided will not fulfill intended 
acquisition outcomes or, ultimately, meet agency needs. 

A performance-based approach, if successfully implemented, can help 
improve service acquisition outcomes. This concept--introduced during 
the 1990s--represented a shift in emphasis from specifying methods in 
which contracted work should be performed to specifying acquisition 
outcomes. In 2000, federal procurement law established a performance- 
based approach as the preferred acquisition method for services. 
[Footnote 7] The FAR requires all performance-based service 
acquisitions to include: 

* a performance work statement that describes outcome-oriented 
requirements in terms of results required rather than the methods of 
performance of the work; 

* measurable performance standards describing how to measure contractor 
performance in terms of quality, timeliness, and quantity; and: 

* the method of assessing contract performance against performance 
standards, commonly accomplished through the use of a quality assurance 
surveillance plan.[Footnote 8] 

In addition, federal procurement law established a preference for using 
firm fixed-price contracts or task orders--where a specified price is 
paid regardless of the contractor's incurred costs--when using a 
performance-based approach for service acquisitions.[Footnote 9] A 
performance-based acquisition approach also calls for greater 
collaboration among procurement and program representatives throughout 
the contract period and often requires more time for acquisition 
planning and management than other contracting methods. Agencies may 
also involve the contractor in the multi-functional team responsible 
for developing performance-based acquisition planning documents to 
leverage the contractor's technical expertise. Including stakeholders 
with varied knowledge and skills enables acquisition teams to determine 
at the earliest point possible that all aspects of the acquisition are 
necessary, executable, and tailored to the transaction's risk level. 
Regardless of the contracting method, focusing on outcomes and 
collaboration among multiple stakeholders in the contracting process 
has been acknowledged as sound contract management. Other factors, such 
as pressure to get programs up and running, additional external 
requirements, and technological challenges also impact the ability to 
achieve good acquisition outcomes. 

A 1998 OFPP study on performance-based contracts--based largely on 
contracts for basic services, such as janitorial or maintenance 
services--showed that a number of anticipated benefits had been 
achieved: reduced acquisition costs, increased competition for 
contracts, and improved contractor performance.[Footnote 10] However, 
our prior work and the work of others has found that implementing a 
performance-based approach is often more difficult for complex 
acquisitions because agencies begin with requirements that are less 
stable, making it difficult to establish measurable outcomes. OFPP has 
noted in policy that certain types of services, such as research and 
development, may not lend themselves to outcome-oriented 
requirements.[Footnote 11] CPO representatives also have noted that 
defining outcome-oriented requirements and measurable performance 
standards may be more challenging for certain types of services, such 
as research and development or professional and management support 
services. Further, complex service contracts, such as those for 
information technology, may need to have requirements and performance 
standards continually refined throughout the life-cycle of the 
acquisition for a contractor to deliver a valuable service over an 
extended period of time. 

To encourage agencies to apply a performance-based approach to service 
acquisitions, the Office of Management and Budget established 
governmentwide performance targets: 20 percent of eligible service 
contract dollars for fiscal year 2002, 40 percent for fiscal year 2005 
and fiscal year 2006, 45 percent for fiscal year 2007, and 50 percent 
for fiscal year 2008. 

In 2007, the congressionally mandated Acquisition Advisory Panel 
[Footnote 12] reported on the implementation of a performance- based 
approach and concluded that agencies were not clearly defining 
requirements, not identifying meaningful quality measures, not 
effectively managing the contract, and had limited data to make 
informed decisions. The panel made recommendations to improve 
application of the performance-based approach in three areas: goal 
setting and management, guidance for acquisition staff, and data 
quality.[Footnote 13] For example, the panel recommended that the 
Office of Management and Budget adjust the governmentwide target to 
reflect individual agency assessments and plans. 

In May 2007, OFPP issued a memo directing agencies, at a minimum, to 
meet targets established and report on them in their management plans. 
In response, DHS's CPO, who is responsible for creating departmentwide 
policies and processes for managing and overseeing the acquisition 
function, established a performance-based target of 25 percent for 
fiscal year 2007, increasing to 40 percent by fiscal year 2010, that 
was included in DHS's Performance-Based Management Plan. Meeting these 
targets is a responsibility shared between the CPO and the component 
heads, in accordance with DHS' system of dual accountability for 
managing the acquisition function.[Footnote 14] 

Definition of Requirements and Performance Standards Influenced 
Contract Outcomes: 

Requirements definition and performance standards influenced outcomes 
for performance-based contracts associated with eight major, complex 
investments. While all eight contracts for the investments we reviewed 
had outcome-oriented requirements, the requirements were not always 
well-defined.[Footnote 15] Further, contracts for half of the 
investments did not have a complete set of measurable performance 
standards. Major investments with contracts that did not have well- 
defined requirements or complete measurable performance standards at 
the time of contract award or start of work experienced either cost 
overruns, schedule delays, or did not otherwise meet performance 
expectations. Conversely, contracts with well-defined requirements 
linked to measurable performance standards delivered results within 
budget and provided quality service. This finding is consistent with 
our body of work on service contracting, which has emphasized the 
importance of sound acquisition planning to develop well-defined 
requirements and measurable performance standards to achieve desired 
results. Table 1 summarizes our analysis of the requirements, 
performance standards, and outcomes for the eight performance-based 
contracts we reviewed. Component program staff conducted surveillance 
for these contracts, and in the four cases that had negative outcomes, 
identified less than expected performance and took action. 

Table 1: Key Characteristics of Eight Performance-Based Service 
Contracts: 

Major investment by component: Coast Guard: Response Boat Medium; 
Service: Research, analysis, and financial and information management; 
Well-defined requirements[A]: contract met or mostly met the criteria; 
Measurable performance standards[B]: contract met or mostly met the 
criteria; 
Outcomes: Contractor submitted all required documentation on time; met 
project management quality standards; and maintained electronic 
archiving and restoration standards. 

Major investment by component: Customs and Border Protection: Automated 
Commercial Environment; 
Service: Trade systems software development (task order 23); 
Well-defined requirements[A]: contract did not meet the criteria; 
Measurable performance standards[B]: contract partially met the 
criteria; 
Outcomes: Costs increased by 40 percent ($21.1 million). More than a 
year behind schedule; unplanned software redesign. 

Major investment by component: Customs and Border Protection: National 
Prime Integration; 
Service: Maintenance of equipment used at border crossings, airports, 
and seaports; 
Well-defined requirements[A]: contract met or mostly met the criteria; 
Measurable performance standards[B]: contract partially met the 
criteria; 
Outcomes: Costs increased by 53 percent ($24 million). Maintenance wait 
times were longer than planned. 

Major investment by component: Customs and Border Protection: SBInet; 
Service: Project 28 border surveillance systems development and 
fielding; 
Well-defined requirements[A]: contract did not meet the criteria; 
Measurable performance standards[B]: contract partially met the 
criteria; 
Outcomes: DHS rejected initial acceptance of Project 28. The project 
was delayed 8 months with final acceptance in February 2008. DHS noted 
that the contractor met the requirements, but the project did not fully 
meet DHS's needs and the technology will not be replicated in future 
SBInet development. 

Major investment by component: Transportation Security Administration: 
Electronic Baggage Screening Program; 
Service: Maintenance for explosive trace detection machines; 
Well-defined requirements[A]: contract met or mostly met the criteria; 
Measurable performance standards[B]: contract met or mostly met the 
criteria; 
Outcomes: Contractor exceeded the performance standard for machine 
downtime with a score 1 hour less than required and operated at cost 
through the second quarter of fiscal year 2007. 

Major investment by component: Transportation Security Administration: 
Screening Partnership Program; 
Service: Passenger screening services at one airport; 
Well-defined requirements[A]: contract met or mostly met the criteria; 
Measurable performance standards[B]: contract met or mostly met the 
criteria; 
Outcomes: Contractor exceeded most performance standards; for example: 
threat detection performance and false alarm rates exceeded the quality 
standards. Contractor had cost underrun of 2.2 percent ($677,000). 

Major investment by component: Transportation Security Administration: 
Secure Flight; 
Service: Maintaining database used to screen airline passenger data; 
Well-defined requirements[A]: contract met or mostly met the criteria; 
Measurable performance standards[B]: contract partially met the 
criteria; 
Outcomes: Initial contractor planning reports were inadequate; system 
experienced operational downtime; surveillance reports identified poor 
contractor performance. Contractor generally met time frames and 
delivered within budget. 

Major investment by component: Transportation Security Administration: 
Transportation Worker Identification Credential; 
Service: Issuing identification credentials to maritime workers; 
Well-defined requirements[A]: contract met or mostly met the criteria; 
Measurable performance standards[B]: contract met or mostly met the 
criteria; 
Outcomes: Outcomes not available at the time of our review. 

Source: GAO analysis. 

[A] Well-defined requirements should provide clear descriptions of 
results to be achieved at the time of the award or start of work and 
primary requirements should not change substantially following contract 
award. 

[B] The set of measurable performance standards for a contract enables 
the government to assess all aspects of the contractor's work in terms 
of quality, timeliness, and quantity. The contract's performance 
standards are also linked to the requirements. 

[End of table] 

Contracts That Lacked Stable, Well-Defined Requirements, Complete 
Measurable Performance Standards, or Both Experienced Negative 
Outcomes: 

Although all contracts we reviewed for the eight major, complex 
investments had outcome-oriented requirements, contracts for four 
investments--two for systems development and two for operations and 
maintenance--did not have stable and well-defined requirements, a full 
complement of measurable performance standards, or both. These four 
contracts experienced schedule delays, cost increases, or other 
negative outcomes. Prior GAO and DHS Inspector General work on other 
DHS major investments have similarly found that when requirements were 
not well-defined, performance did not meet expectations. Our work has 
found that performance-based acquisitions must be appropriately planned 
and structured to minimize the risk of the government receiving 
services that are over cost estimates, delivered late, and of 
unacceptable quality.[Footnote 16] Specifically, we have emphasized the 
importance of clearly defined requirements to achieving desired results 
and measurable performance standards to ensuring control and 
accountability.[Footnote 17] 

Systems Development Efforts: 

Two CBP systems development contracts lacked both well-defined 
requirements and measurable performance standards prior to the start of 
work. The first--ACE Task Order 23 project, a trade software 
modernization effort under a letter contract[Footnote 18]--was 
originally estimated to cost $52.7 million over a period of 
approximately 17 months. DHS contracting representatives told us that 
some technical requirements were not fully defined within the original 
scope of work, and some new software requirements were added after 
contract award. Because many requirements were not fully defined in 
advance of the contract's award, the program was unable to establish 
clear, measurable performance standards and valid cost or schedule 
baselines for assessing contractor performance. The need to redefine 
contract requirements for the ACE program contributed to significant 
schedule delays and cost increases.[Footnote 19] ACE Task Order 23 was 
completed at the end of October 2007 with a project cost increase of 
approximately $21.1 million, or 40 percent, over the original estimate. 
As of April 2008, DHS reported that some portions of the work were 
delayed to better define requirements, and the final portion of the 
software was projected to be completed in June 2009, about 26 months 
later than planned. 

The second, CBP's Secure Border Initiative Network (SBInet) Project 28 
contract, was intended to help secure a section of the United States- 
Mexico border using a surveillance system. Project 28 was planned to be 
SBInet's proof of concept, as well as the first increment of the 
fielded SBInet system. CBP awarded the Project 28 contract before the 
overall SBInet operational requirements and system specifications were 
finalized. Additional design requirements for equipment installation 
were added after contract award. DHS's Inspector General reported in 
February 2007--more than 3 months after the Project 28 contract was 
awarded--that CBP had not properly defined SBInet's operational 
requirements and needed to do so quickly to avoid rework of the 
contractor's systems engineering. Several performance standards were 
not clearly defined to isolate the contractor's performance from that 
of CBP employees, making it difficult to determine whether any problems 
were due to the contractor's system design, CBP employees, or both. 
Other performance standards were difficult to measure because CBP did 
not plan a controlled operational test, which is a test conducted in a 
separate, instrumented test area that resembles the border environment. 
CBP planned to conduct operational testing, which is less rigorous, 
after the system was fully accepted. As a result, it was not clear how 
CBP intended to measure compliance with the Project 28 standard for 
probability of detecting persons attempting to illegally cross the 
border. Another standard stated that the acceptable level of contractor 
performance would be determined in the future. Project 28, planned as 
an 8-month, $20 million firm fixed-price effort,[Footnote 20] did not 
meet the June 2007 delivery date, and in August 2007 CBP notified the 
contractor that it did not approve delivery because the system failed 
the initial acceptance test. DHS conditionally accepted the system in 
December 2007 and fully accepted it in February 2008. DHS noted that 
the contractor met the requirements; however, Project 28 resulted in a 
product that did not fully meet user needs and its design will not be 
used as a basis for future SBInet development.[Footnote 21] In 
addition, DHS officials have stated that much of the Project 28 system 
will be replaced by new equipment and software. 

Operations and Maintenance Efforts: 

Performance-based contracts for two other major investments--CBP's 
National Prime Integration (NPI) contract and a TSA Secure Flight 
contract--included well-defined requirements but did not have a full 
complement of measurable performance standards. These operations and 
maintenance contracts had less than expected contractor performance or 
significant cost increases. 

CBP's NPI contract for maintenance of inspection, detection, and 
surveillance devices at border crossings, airports, and seaports 
included well-defined requirements for conducting equipment deployment, 
maintenance, and replacement as well as engineering and logistics 
support. For example, the requirements for equipment corrective 
maintenance specify the contractor shall conduct maintenance, including 
parts and labor, as a result of equipment or system failure. However, 
the contract did not incorporate performance standards for post-
maintenance inspections and other standards needed to fully assess the 
quality of equipment maintenance performed by the contractor. CBP 
representatives acknowledged the need for additional measurable 
performance standards addressing quality of equipment maintenance to 
help ensure good contractor performance. While CBP does not plan to 
incorporate additional performance measures until September 2008, CBP 
revised the contract documentation in December 2007 to emphasize 
program management. Regarding contract outcomes, during the last year 
of the contract, costs were about $24 million, or 53 percent, higher 
than original estimates, in part because of increases in the quantity 
of equipment being supported. Customer wait times have been much longer 
than expected for equipment maintenance, and wait time standards have 
been revised to between 24 and 96 hours depending on the type of 
equipment. Average wait times from September 2006 to September 2007 
were between 118 and 106 hours, not meeting the revised standard. 

A TSA Secure Flight contract for operations and maintenance of the 
federal terrorist watchlist database, which is used to screen airline 
passenger data, also experienced less than expected outcomes. The 
contract we reviewed contained well-defined requirements and measurable 
performance standards for operation, maintenance, repair, and upgrade 
of the system. For example, the requirements specify that the 
contractor shall provide on-site technical and management support. 
Although the contract had some measurable performance standards, the 
contract did not have the full complement of standards needed to 
monitor performance, according to program representatives. For example, 
one standard required no operational system downtime and the contractor 
was not always able to meet this standard. After we reviewed the 
performance standards, program representatives subsequently 
incorporated some additional, improved standards to provide increased 
visibility over how well the contractor responds to system outages. 
These standards are designed to measure the time required to switch 
from the primary to the secondary system for planned and unplanned 
system outages. Additionally, according to program and contracting 
representatives, the contractor initially delivered inadequate plans 
for staffing and operations, and did not always meet the operational 
system downtime requirement, but was within budget and generally met 
established time frames. 

Prior Reviews Have Similar Findings: 

The inability to manage requirements and meet cost, schedule, and 
performance objectives is not limited to the components and systems we 
reviewed. Several prior GAO and DHS Inspector General reviews of major 
DHS investments using a performance-based approach point to such 
shortcomings. For example: 

* In June 2007, we reported that DHS's performance-based contract for a 
departmentwide financial management system, eMerge2, did not have clear 
or complete requirements.[Footnote 22] Consequently, the program 
experienced schedule delays and less than acceptable contractor 
performance and was ultimately terminated after a $52 million 
investment. 

* In March 2007, we reported that the Coast Guard's performance-based 
contract for replacing or modernizing its fleet of vessels and 
aircraft, Deepwater, had requirements that were set at unrealistic 
levels and then were frequently changed.[Footnote 23] This resulted in 
cost escalation, schedule delays, and reduced contractor 
accountability. The DHS Inspector General has also indicated that 
numerous opportunities exist for DHS to make better use of sound 
practices, such as well-defined requirements. The DHS Inspector General 
recently concluded that Deepwater had poorly defined requirements and 
inadequate oversight that contributed to ineffective or inefficient 
results and increased costs.[Footnote 24] 

* In February 2006, DHS's Inspector General reported that TSA's 
Information Technology Managed Services contract had unclear 
requirements that contributed to negative outcomes, including an 85 
percent increase to nearly $834 million in DHS's 3-year cost estimate 
and a failure to deliver expected information technology capabilities. 
[Footnote 25] 

Contracts with Well-Defined Requirements and Measurable Performance 
Standards Experienced Positive Outcomes: 

Contracts for four other major investments we reviewed--one at the 
Coast Guard and three at TSA--had well-defined requirements and 
measurable performance standards. These requirements did not 
significantly change in scope after contract award. Contracts for 
program management support for the Response Boat-Medium program, 
maintenance of the Electronic Baggage Screening Program, and passenger 
and baggage screening for the Screening Partnership Program met the 
performance standards and were within budget. We were unable to 
evaluate outcomes for the fourth contract associated with the 
Transportation Worker Identification Credential (TWIC) program because 
key work under the contract was delayed. 

Program management support and related services we reviewed for the 
Coast Guard's Response Boat-Medium investment had well-defined 
requirements, and two of three performance standards were measurable. 
The contracted services included research and project support, 
financial and information management, and Web site support. The 
requirements for research and project support clearly stated the 
contractor was to provide project planning and execution support; 
documentation, database, and internet support; and administrative 
support. The performance standards were linked directly to the 
requirements. The two measurable standards were for timely submission 
of documents and electronic records keeping. The Coast Guard did not 
indicate how it would measure contractor performance for the third 
standard addressing research analysis and advisory services. Coast 
Guard representatives said the contractor had fulfilled the 
requirements within budget and on time, but did not document contractor 
performance. 

A maintenance contract for TSA's Electronic Baggage Screening Program 
also had well-defined program management and technical requirements. 
For example, the requirements for maintenance clearly stated the 
contractor should perform preventative and corrective maintenance on 
detection machines, providing specific definitions for each of these 
tasks. The contract also had measurable performance standards linked 
directly to the technical requirements. Specifically, program 
representatives used the 'mean downtime' performance standard to 
calculate the average number of hours a machine was unable to perform 
its mission during a 16-hour day as a means of evaluating how quickly 
the contractor restored the detection equipment to operation. For 
fiscal years 2006 through 2007, the contractor met or exceeded the mean 
downtime performance standard, which has become more rigorous each 
year. Additionally, the contractor has stayed within the budget 
established in the firm fixed-price contract. The current maintenance 
contract improved TSA's ability to manage costs. Previously, DHS's 
Inspector General found TSA's initial electronic baggage screening 
maintenance contracts did not follow sound contracting practices. The 
Inspector General recommended TSA take action to control costs by 
modifying the type of contract used.[Footnote 26] TSA subsequently 
withdrew the initial maintenance contract and issued four new 
contracts, one of which we reviewed, for maintenance of detection 
machines at over 400 airports in the United States. 

Contracted security services at the San Francisco International Airport 
for the Screening Partnership Program also had well-defined 
requirements, and all measurable performance standards corresponded to 
contract requirements. This was an improvement from our prior reviews 
of TSA's Screening Partnership Program, which found that TSA had not 
finalized performance-based planning documentation, including 
developing measurable performance standards.[Footnote 27] The contract 
we reviewed contained well-defined requirements for gate, checkpoint, 
and baggage screening; basic preventative and corrective maintenance; 
initial, recurrent, and remedial training; and recruiting and 
supervising screeners. For example, the requirements for gate, 
checkpoint, and baggage screening services clearly stated the 
contractor should use technology and staff to prevent prohibited items 
from entering sterile areas of the airport and should work to minimize 
customer complaints while addressing in a timely manner any complaints 
received. The performance standards assessed how often screeners could 
successfully detect test images of prohibited items in checked baggage; 
the percentage of audited records and inspected equipment, property, 
and materials that were well-kept, operational, and recorded on 
maintenance logs; and whether all new hires received the required 
training before assuming their screening responsibilities. In terms of 
expected outcomes, the contractor achieved a 2.2 percent cost underrun 
during the first 5 months of the contract and exceeded most 
requirements. 

While the worker enrollment services contract for TSA's TWIC program 
for a tamper-resistant biometric credential for maritime workers 
included well-defined requirements and measurable performance 
standards, we were unable to review the contract's outcomes as key work 
under the contract was delayed.[Footnote 28] Prior GAO work found that 
during the TWIC program's prototype phase, the program contract costs 
doubled due to expanded requirements after contract award.[Footnote 29] 
The contract we reviewed contained well-defined requirements for 
developing enrollment centers; providing information technology 
maintenance services; and setting up help desk services for TSA 
employees, maritime workers, and other TWIC stakeholders. For example, 
the requirements for enrollment centers clearly stated the contractor 
was to provide maintenance services; infrastructure support such as 
work stations; and operations, management, and administrative support. 
Moreover, the contract's performance standards were measurable and 
linked to the requirements. For example, the program developed 
performance standards assessing the amount of time the contractor took 
to respond to and resolve each caller's issue and to enroll maritime 
workers in the program. By using measurable performance standards 
linked to the contract's requirements, TSA has improved its ability to 
identify and address performance issues and avoid negative outcomes. 

Quality Assurance Surveillance Identified Less Than Expected 
Performance: 

Agencies are required to ensure that contractors are providing timely 
and quality services and mitigate contractor performance problems. The 
FAR indicates that surveillance plans provide for contract quality 
assurance by specifying contractor work requiring surveillance and the 
method of surveillance.[Footnote 30] While not all of the contracts we 
reviewed had a quality assurance surveillance plan in place at contract 
award, in all cases DHS components monitored and assessed contractor 
performance against the contract performance standards. In the four 
cases where we found poor contract outcomes, program staff identified 
and documented less than expected performance and took action as shown 
in table 2. 

Table 2: Less Than Expected Performance Identified through Surveillance 
and Action Taken: 

Major investment by component: Customs and Border Protection: Automated 
Commercial Environment Task Order 23; 
Service: Developing trade systems software; 
Performance: Costs increased by 40 percent ($21.1 million) and 
contractor did not achieve project milestones according to schedule; 
Action taken: Contractor did not earn full incentive fees. 

Major investment by component: Customs and Border Protection: National 
Prime Integration; 
Service: Maintaining equipment used at border crossings, airports, and 
seaports; 
Performance: Longer than expected maintenance wait times; 
Action taken: Contractor did not earn full award fees. 

Major investment by component: Customs and Border Protection: SBInet 
Project 28; 
Service: Developing and fielding border surveillance systems; 
Performance: System failed the initial performance test and project was 
delayed 8 months with final acceptance in February 2008; 
Action taken: Contractor did not receive 20 percent of contract payment 
and DHS did not initially accept the system. 

Major investment by component: Transportation Security Administration: 
Secure Flight; 
Service: Maintaining database used to screen airline passenger data; 
Performance: Inadequacies identified in initial contractor planning 
reports provided to TSA; operational downtime; and other forms of less 
than expected performance; 
Action taken: Contractor did not earn full award fees and DHS worked 
with contractor to increase staffing levels. 

Source: GAO analysis. 

[End of table] 

In two of these cases--ACE and Secure Flight--program representatives 
identified less than expected performance through the use of a quality 
assurance surveillance plan. In contrast, the NPI program office told 
us they did not have a quality assurance surveillance plan for the 
first year of the contract, but instead relied on award fee plan 
criteria to assess contractor performance. The SBInet program office 
monitored Project 28's execution through other means, such as program 
reviews, technical interchange meetings, status and issue reporting, 
and observation of contractor testing. Our prior work has found that if 
performance-based acquisitions are not appropriately planned and 
structured, including surveillance planning, there is an increased risk 
that the government may receive products or services that are over 
budget, delivered late, and of unacceptable quality.[Footnote 31] 

DHS Has Faced Workforce and Oversight Challenges in Managing Its 
Service Acquisitions: 

Insufficient workforce and limited oversight of acquisition outcomes 
has presented significant challenges for DHS in implementing its 
service acquisitions, including those that use a performance-based 
approach. Having adequate staff who collaborate across functions, and 
reliable data are among the key success factors that we have identified 
for achieving intended outcomes for all types of service acquisitions. 
[Footnote 32] However, DHS continues to lack contracting and program 
staff with the expertise needed to adequately plan or monitor 
contractor execution of requirements, and some component acquisition 
representatives indicated a lack of collaboration between the program 
and contracting offices. In terms of oversight, inaccurate data has 
limited departmentwide visibility into DHS's service acquisitions, 
including those that are performance-based. 

Sufficient and Experienced Staff Are Needed to Ensure Successful 
Service Acquisitions: 

Our prior work has highlighted the importance of having the right 
people with the right skills to achieve successful acquisition 
outcomes.[Footnote 33] However, DHS has not fully defined the types of 
positions or numbers of staff for each position for its acquisition 
workforce. CPO representatives identified acquisition staff shortages 
as one of the primary obstacles to successful acquisitions, including 
those that are performance-based. CPO representatives have established 
departmentwide interim staffing goals addressing minimum and optimal 
staffing levels for contract specialists for each component. As of 
February 2008, CPO representatives reported that approximately 61 
percent of the minimum required staff, and 38 percent of the optimal 
level of contract specialists, were in place. 

Insufficient contracting and program office staff was a challenge for 
two of the investments we reviewed--SBInet and ACE. In these two cases, 
contracts were not on schedule or contractor performance was otherwise 
less than expected. In November 2006--the month following SBInet 
Project 28's contract award--the DHS Inspector General reported that 
DHS did not have the capacity needed to effectively plan, oversee, and 
execute the SBInet investment; administer its contracts; and control 
costs and schedule.[Footnote 34] For example, SBInet had a shortage of 
engineers, logistical personnel, contracting officers, and cost 
analysts. In January 2007, DHS's Investment Review Board confirmed 
there were critical staffing shortages in the SBInet investment, 
including Project 28. In February 2007, we reported that SBInet 
representatives expressed concern about finding an adequate number of 
program office staff with the right expertise. We subsequently reported 
in February 2008 that the SBInet program office staffing level had 
reached 305 positions out of a staffing goal of 470 by the end of 
fiscal year 2008.[Footnote 35] CBP representatives expressed concerns 
that staffing shortfalls could affect the agency's capacity to provide 
adequate contractor oversight. ACE program officials also indicated 
they were significantly understaffed, which prevented them from fully 
defining the requirements for ACE Task Order 23. Officials told us the 
investment had less than 10 percent of the staff they needed at the 
time the requirements were being defined. 

In its fiscal year 2007 departmentwide performance-based service 
acquisition management plan, DHS also emphasized the importance of 
filling program and contracting vacancies with staff experienced in 
developing and implementing the required elements of performance-based 
acquisitions. Contracting and program management staff for TSA's 
Screening Partnership and Electronic Baggage Screening programs stated 
that their previous experience helped them to develop well-defined 
requirements and measurable performance standards. In these cases, 
contracts were on budget and contract performance met or exceeded the 
government's requirements. Representatives from Coast Guard, CBP, and 
TSA noted success in supplementing expertise through acquisition 
support offices, also known as centers of excellence, to improve 
acquisition planning. Component representatives told us this support 
could help acquisition staff improve the quality of contract documents, 
including performance work statements. These support offices can serve 
as a force multiplier, allowing their high-demand skill-sets to be used 
across contracts and helping to improve contracting outcomes across the 
component. 

According to CPO representatives, hiring knowledgeable acquisition 
staff has been difficult due to a limited pool of qualified acquisition 
professionals and intense competition between the government and the 
private sector for those professionals.[Footnote 36] The CPO has the 
following ongoing departmentwide initiatives to recruit, train, and 
define its acquisition workforce that have the potential to better 
position DHS to address acquisition workforce challenges. These 
include: 

* centralize recruiting activities across the department; 

* institute an acquisition intern program; 

* employ direct hire authority[Footnote 37] and rehire 
annuitants;[Footnote 38] 

* reemphasize training opportunities and establishing a centralized 
acquisition workforce training fund; and: 

* conduct workforce planning. 

The need for improved collaboration among the acquisition workforce is 
also a recurrent theme in our work on acquisition management. We have 
noted that acquisition groups should have procedures in place that 
empower staff to collaborate when procuring goods and services and have 
controls and incentives to ensure collaboration occurs.[Footnote 39] In 
particular, leading organizations generally employ a multifunctional 
approach including contracting, finance, legal, and other participants 
as needed to help develop cost-effective acquisition approaches and 
help ensure financial accountability. When requirements are dynamic, as 
was the case for some of the investments we studied, there is a greater 
need for more sophisticated collaboration--a point echoed by CBP and 
TSA representatives who emphasize the importance of early collaboration 
for complex acquisitions. OFPP guidance also emphasizes that 
performance-based service acquisition is a collective responsibility 
involving representatives from budget, technical, contracting, 
logistics, legal, and program offices. 

For the eight major investments we reviewed, program and contracting 
offices jointly approved the acquisition plans, and representatives for 
several of the contracts we studied indicated they worked together 
effectively. However, at the component level, senior acquisition staff 
at TSA and CBP indicated that collaboration between contracting and 
program offices in general has been challenging. For example, they 
cited poor timing when bringing together key stakeholders in the 
contracting process, and in some cases, the program offices' preference 
for prescriptive requirements rather than a more outcome-oriented 
approach. Coast Guard and TSA acquisition representatives we spoke with 
indicated they have ongoing efforts to promote greater collaboration. 

Limited Data Inhibits DHS's Ability to Oversee Its Service 
Acquisitions: 

While DHS contracting and program representatives told us they use a 
performance-based approach to the maximum extent practicable, DHS does 
not have reliable data, either from FPDS-NG or at a departmentwide 
level, to systematically monitor or evaluate service acquisitions, 
including those that are performance-based acquisitions. Reliable data 
are essential to overseeing and assessing the implementation of 
contracting approaches, acquisition outcomes, and making informed 
management decisions. 

Our review of 138 contracts at the Coast Guard, CBP, ICE, and TSA coded 
in FPDS-NG as performance-based illustrates the difficulty in 
identifying such contracts.[Footnote 40] According to contracting 
representatives at the Coast Guard, CBP, ICE, and TSA, only 42, about 
30 percent, of these contracts could be confirmed as containing the 
required performance-based elements--a performance work statement, 
measurable performance standards, and a method of assessing contractor 
performance against performance standards. About 18 percent had some 
but not all of the required performance-based acquisition elements, and 
about 51 percent--totaling about $347.3 million--had none of the 
required elements (see table 3). This analysis indicates that the FPDS- 
NG data are not reliable for reporting on the performance target for 
eligible service obligations. For example, in fiscal year 2006, DHS 
reported awarding performance-based contracts for 21 percent of 
eligible service contract dollars. However, due to unreliable data, DHS 
is likely farther away from meeting the governmentwide target than 
previously reported. In addition, in July 2006 OFPP requested all 
agencies to report in their 5-year management plan on the use of 
performance-based contracts by service types--ranging from basic, such 
as janitorial and landscaping, to complex, such as information 
technology or systems development. The Acquisition Advisory Panel and 
DHS's CPO also have raised concerns regarding the accuracy of the 
performance-based designation in FPDS-NG. The Acquisition Advisory 
Panel reported from its review at 10 federal agencies that 42 percent 
of the performance-based contracts it reviewed had been incorrectly 
coded.[Footnote 41] 

Table 3: Review of Performance-Based Elements on Selected Contracts: 

Performance-based elements: All elements; 
Coast Guard: 18; 
Customs and Border Protection: 3; 
Immigration and Customs Enforcement: 0; 
Transportation Security Administration: 21; 
Total contracts: 42; 
Percent of total contracts: 30.4. 

Performance-based elements: Some elements; 
Coast Guard: 16; 
Customs and Border Protection: 0; 
Immigration and Customs Enforcement: 5; 
Transportation Security Administration: 4; 
Total contracts: 25; 
Percent of total contracts: 18.1. 

Performance-based elements: No elements; 
Coast Guard: 20; 
Customs and Border Protection: 5; 
Immigration and Customs Enforcement: 34; 
Transportation Security Administration: 12; 
Total contracts: 71; 
Percent of total contracts: 51.5. 

Performance-based elements: Total; 
Coast Guard: 54; 
Customs and Border Protection: 8; 
Immigration and Customs Enforcement: 39; 
Transportation Security Administration: 37; 
Total contracts: 138; 
Percent of total contracts: 100. 

Source: GAO analysis of DHS review of 138 contracts coded as 
performance-based in FPDS-NG. 

[End of table] 

According to DHS contracting representatives, contracts were miscoded 
as performance-based for several reasons. Contracting staff may have 
coded contracts as performance-based without the presence of the 
required elements. Contracting staff also may not have followed data 
validation procedures. Finally, there are technical FPDS-NG issues. For 
example, DHS contracting and program staff said FPDS-NG automatically 
applies the same designation to all orders issued under the master 
contract although some individual orders may or may not be performance- 
based.[Footnote 42] While DHS officials acknowledged this can occur, 
they stated that the data field in FPDS-NG could still be edited for 
orders issued under the master contract and that it was incumbent on 
DHS contracting representatives to review the designation for every 
contract action to ensure they are coded correctly. 

Inaccurate federal procurement data is a long-standing governmentwide 
concern. Our prior work and the work of the General Services 
Administration's Inspector General have noted issues with the accuracy 
and completeness of FPDS and FPDS-NG data.[Footnote 43] The Office of 
Management and Budget has stressed the importance of submitting timely 
and accurate procurement data to FPDS-NG and issued memos on this topic 
in August 2004 and March 2007. 

Accurate FPDS-NG data could facilitate the CPO's departmentwide 
oversight of service acquisitions, including those that are performance-
based. For example, the selection of a contract type is the principal 
means agencies have for allocating financial risk between the 
government and the contractor. For performance-based services, federal 
law establishes a preference for using firm fixed-price as opposed to 
cost reimbursable or time and materials contracts.[Footnote 44] In 
addition, the CPO has instructed components to avoid using time-and- 
material contracts for services. We reviewed the contract type for the 
42 contracts DHS confirmed were performance-based. Our analysis showed 
that DHS awarded about 70 percent of these 42 contracts as fixed-price; 
these actions represented about 42 percent of total obligations for 
these awards. About 37 percent of total obligations were awarded as 
cost-reimbursable, and 7 percent were awarded as time and materials 
contracts (see fig. 1). 

Figure 1: Percent of Contracts and Obligations by Contract Type for 42 
DHS Performance-Based Service Acquisitions (Fiscal Years 2005 and 
2006): 

[See PDF for image] 

This figure is a vertical bar graph depicting the following data: 

Contract type: Fixed price; 
Contracts: 69.0%; 
Obligations: 41.9%. 

Contract type: Cost reimbursable; 
Contracts: 9.5%; 
Obligations: 37.0%. 

Contract type: Time and materials/labor hours; 
Contracts: 11.9%; 
Obligations: 6.7%. 

Contract type: Other[A]; 
Contracts: 9.5%; 
Obligations: 14.4%. 

Source: GAO analysis of DHS confirmed performance-based service 
acquisitions. 

[A] "Other" includes contracts for which the contract type field in 
FDPS-NG was blank, designated as being none of the contract types 
listed within FPDS-NG, or coded with more than one contract type. 

[End of figure] 

At a departmentwide level, CPO representatives responsible for 
acquisition oversight indicated they have not conducted systematic 
assessments including costs, benefits, and other outcomes of a 
performance-based approach. To improve the implementation of 
performance-based acquisitions, CPO representatives established a work 
group in May 2006 to leverage knowledge among DHS components. They also 
noted that they are working with OFPP to develop a best practices guide 
on measurable performance standards and to gather good examples of 
performance-based contracts. In addition, the CPO has implemented a 
departmentwide acquisition oversight program that is intended to assess 
(1) compliance with federal acquisition guidance, (2) contract 
administration, and (3) business judgment. This program was designed 
with the flexibility to address specific procurement issues, such as 
performance-based service acquisitions, and is based on a series of 
component-level reviews.[Footnote 45] However, this oversight program 
has not yet included an evaluation of the outcomes of this acquisition 
method. 

Conclusions: 

As part of its expansive homeland security mission, DHS spends billions 
of dollars on service acquisitions for critical trade, transportation, 
and border security investments. Consistent with federal procurement 
policy, DHS has emphasized a performance-based approach to improve 
service acquisition outcomes. However, in keeping with our prior 
findings, DHS's designation of a service acquisition as performance- 
based was not as relevant as the underlying contract conditions. Sound 
acquisition practices, such as clearly defining requirements; 
establishing complementary measurable performance standards; and 
planning and conducting surveillance in order to take corrective 
actions are all hallmarks of successful service acquisitions. In the 
cases we reviewed, as well as in prior findings lacking these key 
elements, DHS did not always achieve successful acquisition outcomes. 
To improve DHS's acquisition management, the CPO is taking steps to 
address departmentwide workforce and oversight needs. However, improved 
data accuracy and systematic evaluation of contracting methods and 
outcomes are also key to successful oversight efforts. Continued 
emphasis on these areas is required to prevent less than expected 
acquisition outcomes for the department's critical mission. 

Recommendations for Executive Action: 

To increase DHS's ability to achieve improved outcomes for its service 
acquisitions, including those that are performance-based, we recommend 
that the Secretary of Homeland Security implement the following three 
actions: 

* routinely assess requirements for major, complex investments to 
ensure they are well-defined and develop consistently measurable 
standards linked to those requirements; 

* at a departmentwide level, systematically evaluate the outcomes of 
major investments and relevant contracting methods; and: 

* continuously improve the quality of FPDS-NG data to facilitate the 
ability to accurately identify and assess the use and outcomes of 
various contracting methods. 

Agency Comments and Our Evaluation: 

We provided a draft of this report to DHS and OMB for review and 
comment. In written comments, DHS generally concurred with our findings 
and recommendations and provided some information on efforts under way 
to improve acquisition management. The department's comments are 
reprinted in appendix II. OMB did not comment on the findings or 
conclusions of this report. DHS and OMB each provided technical 
comments, which we incorporated as appropriate and where supporting 
documentation was provided. 

In response to our recommendations that DHS (1) routinely assess 
requirements for major, complex investments to ensure they are well- 
defined, and develop consistently measurable standards linked to those 
requirements, and (2) systematically evaluate the outcomes of major 
investments and relevant contracting methods, DHS stated that the CPO 
is working to strengthen acquisition and procurement. For example, DHS 
noted that the CPO is revising the investment review process; reviewing 
major programs and investments; and building the capability to manage 
complex efforts by ensuring program offices are properly structured and 
staffed with the right people and skills. To assist in accomplishing 
these goals, DHS noted that the CPO has established a new division to 
provide oversight and support for acquisition programs. We have ongoing 
work at DHS reviewing the results of DHS's oversight of major 
acquisitions and plan to report on these initiatives in the final 
product for that engagement. Improving acquisition management has been 
an ongoing challenge since the department was established and requires 
sustained management attention. However, DHS's response did not address 
how the CPO's process and organizational changes will impact component- 
level management and assessment of complex acquisitions to improve 
outcomes. 

Concerning the third recommendation, that DHS continuously improve the 
quality of FPDS-NG data to facilitate the ability to accurately 
identify and assess the use and outcomes of various contracting 
methods, DHS stated that as part of the CPO's oversight reviews, the 
accuracy of the FPDS-NG data is validated for the review sample, 
including whether contracts have been properly coded as performance- 
based. DHS also added that the CPO is an active member of the OFPP 
group working to improve FPDS-NG and has established a governance board 
whereby the CPO reaches out to DHS components to improve data 
collection. While these initiatives may be steps in the right 
direction, DHS's response did not present the results of the oversight 
reviews, or state how these coordination efforts address the causes of 
miscoding or how they will improve the quality of FPDS-NG data in the 
future. 

DHS also noted other initiatives to improve the acquisition workforce, 
including working to obtain qualified acquisition professionals and 
conducting staffing studies to better define the department's needs. We 
also have ongoing work at DHS on the acquisition workforce and plan to 
report on the results of DHS's efforts in the final product for that 
engagement. 

As agreed with your offices, unless you publicly announce the contents 
of this report, we plan no further distribution for 30 days from the 
report date. At that time, we will send copies of this report to the 
Secretary of Homeland Security, the Director of the Office of 
Management and Budget, and other interested congressional committees. 
We will also make copies available to others upon request. In addition, 
this report will also be available at no charge on GAO's Web site at 
[hyperlink, http://www.gao.gov]. 

If you have questions about this report or need additional information, 
please contact me at (202) 512-4841 or huttonj@gao.gov. Contact points 
for our Offices of Congressional Relations and Public Affairs may be 
found on the last page of this report. 

Signed by: 

John P. Hutton, Director: 
Acquisition and Sourcing Management: 

[End of section] 

Appendix I: Scope and Methodology: 

[End of section] 

To evaluate selected Department of Homeland Security (DHS) components' 
implementation of a performance-based approach in the context of 
service acquisitions for major investments, we conducted a detailed 
file review of eight judgmentally selected performance-based contracts 
for major, complex investments with a total value greater than $100 
million.[Footnote 46] To identify the components with the highest 
obligations using a performance-based approach in fiscal years 2005 and 
2006, we used the Federal Procurement Data System-Next Generation (FPDS-
NG) and selected four components--Coast Guard, Customs and Border 
Protection (CBP), Immigration and Customs Enforcement (ICE), and the 
Transportation Security Administration (TSA).[Footnote 47] We excluded 
the Federal Emergency Management Agency (FEMA) from our review because 
of atypical fiscal year 2006 spending on Gulf Coast hurricane relief 
efforts. 

To identify contracts for major investments, we utilized several 
different approaches.[Footnote 48] Three issues required us to modify 
our plans--the lack of information in FPDS-NG, inaccurate DHS 
contracting data on major investments, and inaccurate information 
provided by one DHS component. Because the FPDS-NG major program field 
was typically blank, we were unable to use this field to identify 
contracts associated with DHS major investments. As a result, we asked 
DHS to provide us with a list of performance-based contracts associated 
with the four components' major investments. DHS provided a list 
including 76 contracts for 16 major investments. Due to data 
inaccuracies related to the contract numbers on DHS's list, we were 
only able to identify performance-based contracts associated with two 
major investments. We selected one contract from CBP's Automated 
Commercial Environment and one from TSA's Secure Flight that had the 
highest obligations. To obtain additional contracts to review, we met 
with CBP and TSA contracting representatives and they identified five 
other major investments with performance-based contracts, and we 
included one from each investment in our review. Coast Guard 
contracting representatives told us none of their major investments 
used performance-based contracts. However, we randomly selected and 
reviewed four contracts and seven orders at the Coast Guard from DHS's 
list of major investments, and found that five of them were performance-
based. Of these, we selected three performance-based orders from one 
contract for one major Coast Guard investment--the Response Boat-
Medium--to include in our review. Lastly, ICE contracting 
representatives also reported that none of their major investments used 
a performance-based approach and provided us with their best examples 
of a performance-based contract. We reviewed these examples and found 
that none of them were associated with a major investment. As a result, 
we excluded ICE from our contract review. 

For the selected contracts, with a combined estimated value of $1.53 
billion, we reviewed documentation, including acquisition plans, 
performance work statements and statements of objectives, quality 
assurance surveillance plans, and government or contractor performance 
reports where available. Performance-based service acquisitions were 
difficult to identify and were not clearly distinct from other service 
acquisitions because they did not always include all required elements. 
Therefore, we reviewed prior GAO and DHS Inspector General reports to 
provide a broader context for service acquisition and other DHS major 
investments. We reviewed public laws, federal and agency acquisition 
regulations, and the Office of Management and Budget's Office of 
Federal Procurement Policy (OFPP) memoranda and best practices. We also 
interviewed procurement and program representatives to determine the 
extent to which the performance-based planning process influenced 
expected outcomes. 

To identify management challenges that may affect DHS's successful 
implementation of service acquisitions for major investments, including 
those that use a performance-based approach, we analyzed management 
documents and plans; staffing data; and oversight mechanisms, including 
the Federal Procurement Data System-Next Generation (FPDS-NG), the 
governmentwide database for procurement spending. From FPDS-NG, we 
judgmentally selected 138 contracts for primarily basic services coded 
as performance-based and awarded in fiscal years 2005 or 2006 at Coast 
Guard, CBP, ICE, and TSA with obligations greater than $1 million 
dollars and asked the components to verify whether these contracts 
included the required performance-based elements outlined in FAR 
subpart 37.6. Finally, we interviewed contracting and program 
management staff and DHS Chief Procurement Office (CPO) representatives 
and met with representatives from the Office of Management and Budget's 
Office of Federal Procurement Policy (OFPP). To evaluate the extent to 
which DHS used different contract types for performance-based 
acquisitions, we reviewed the contract type on all contracts DHS 
verified as performance-based. 

We conducted this performance audit from January 2007 to April 2008 in 
accordance with generally accepted government auditing standards. Those 
standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. 

[End of section] 

Appendix II: Comments from the Department of Homeland Security: 

U.S.Department of Homeland Security: 
Washington, DC 20528: 

April 11,2008: 

Mr. John P. Hutton: 
Director: 
Acquisition and Sourcing Management: 
U.S. Government Accountability Office: 
441 G Street, NW: 
Washington, DC 20548: 

Dear Mr. Hutton: 

Thank you for the opportunity to review and comment on the Government 
Accountability Office's (GAO's) draft report GAO-08-263 entitled 
Department Of Homeland Security: Better Planning and Assessment Needed 
to Improve Outcomes for Complex Service Acquisitions. Technical 
comments have been provided under separate cover. 

The Department of Homeland Security (DHS) generally concurs with the 
findings and recommendations in the draft report and offers the 
following with respect to the report's three recommendations: 

GAO Recommendation #1: Routinely assess requirements for major, complex 
investments to ensure they are well-defined and develop consistently 
measurable standards linked to those requirements. 

GAO Recommendation #2: At a department-wide level, systematically 
evaluate the outcomes of major investments and relevant contracting 
methods. 

DHS Response to Recommendations #1 and #2: DHS is in the midst of many 
crucial acquisitions that are vital to its success. The Chief 
Procurement Officer (CPO) is working to strengthen acquisition and 
procurement by institutionalizing solid processes, including the 
following actions: 

A. Strengthening the requirements and investment review processes by 
improving the joint requirements council and Investment Review Board 
(IRB) process. We are preparing to initiate a new Department-wide 
requirements process and have reinvigorated our investment review 
process;
B. Reviewing the major programs and investments to ensure that the 
requirements are clear, cost estimates are valid, technology risks are 
properly assessed, schedules are realistic, contract vehicles are 
proper, and the efforts are well managed. We have held one formal 
Deputy Secretary IRB and are projecting one per month. We are also 
beginning the processes to conduct paper IRBs and Deputy UnderSecretary 
for Management IRBs, as well as establishing Acquisition Program 
Baselines and authorizing execution to the APB for all Level 1 and 2 
programs; 

C. Building the capability to manage complex efforts by ensuring that 
program offices are properly structured and staffed with the right 
people and skills to ensure efficient and effective program management 
and oversight; and to aggressively hire where we have known shortages; 
and; 

D. Examining best practice metrics in use by other departments with the 
intent to start implementation this year. 

The Acquisition Program Management Division (APMD) of the Office of the 
Chief Procurement Officer (OCPO) began operations in August 2007. The 
division was established to provide oversight and support for 
acquisition programs. To date, APMD has performed Quick Look 
assessments of 37 Level 1 programs and has overseen Deep Dive reviews 
of the SBInet and Advance Spectroscopic Portal (ASP) programs. APMD has 
provided advice and guidance to a number of programs, particularly in 
the area of cost benefit analysis. Currently the APMD team is focused 
on an aggressive Investment & Acquisition process re-engineering 
effort. The effort includes replacing DHS Management Directive 1400 
Investment Review Process, establishing revised investment and 
acquisition decision procedures, as well as processes for acquisition 
program baselining, periodic reporting, acquisition of services, and 
other initiatives as they are identified. 

CPO is also working to assure that DHS obtains qualified acquisition 
professionals at the right time with the right skill-set. Competition 
for these professionals is intense within the Washington, D.C. area. To 
resolve our personnel shortages, we are intensifying our human capital 
planning efforts to minimize skill and competency gaps as well as 
minimize our critical vacancies and reliance on contractors. For 
example, in response to the OMB 1102 Contracting Workforce Competency 
Gap Survey, we developed a training plan that spans the next three 
years. This training plan targets the contracting functional area 
within the DHS Acquisition Workforce, but it will also benefit other 
acquisition career fields including program management and Contracting 
Officer's Technical Representatives. 

CPO is also currently conducting staffing studies to better define our 
acquisition workforce needs. Currently our workforce includes program 
managers and contract specialists. As part of our human capital 
planning efforts, we will be identifying other required acquisition 
career fields such as test and evaluation, systems engineering, 
logistics, and cost estimating. We are aggressively working to ensure 
that each acquisition position, upon definition, is encumbered by an 
acquisition professional trained and certified at the appropriate 
level. To this end, we are continuously reviewing and updating our 
Acquisition Training Program, the underpinning of a good certification
program. We are utilizing the Defense Acquisition Workforce Improvement 
Act framework to develop DHS certification standards. We have also 
centralized a number of recruiting activities including issuing 
Department-wide vacancy announcements. Our centralized recruitment 
efforts to date have focused primarily on contracting professionals. 
Expansion to other acquisition career fields will occur as each series 
is defined and Department-wide needs are identified. This initiative 
supplements our Components' on-going recruitment efforts with a goal of 
recruiting the best candidates available. In 2005, the Department 
commenced the Acquisition Fellows Program. The goal of the Fellows 
Program was to attract new talent at the entry level into our 
acquisition positions, and retain and train them through a professional 
career development program. Building on the success of the Acquisition 
Fellows Program, CPO formalized and modeled it to further resemble the 
highly successful Department of Defense program. This year CPO received 
funding for the Acquisition Professional Career Program and 33 of the 
requested 66 positions were funded. In FY 2009, the plan provides for a 
total of 100 positions to be funded. Our inaugural class in the 
Acquisition Professional Career Program began in January 2008 and a 
second class will begin in June 2008. Our goal is to grow this program 
to 300 positions by FY 2011 to fill our critical acquisition positions. 

GAO Recommendation #3: Continuously improve the quality of Federal 
Procurement Data System-Next Generation (FPDS-NG) data to facilitate 
the ability to accurately identify and assess the use and outcomes of 
various contracting methods. 

DHS Response to Recommendation #3: As part of CPO's oversight reviews, 
the accuracy of the FPDS data is validated for the review sample, 
including whether the contract has been properly coded as a performance 
based contract. In addition, OCPO is an active member of the Office of 
Federal Procurement Policy group that is working to improve FPDS. OCPO 
has also established a Governance Board whereby CPO reaches out to the 
DHS Components to improve data collection. 

Thank you again for the opportunity to comment on this draft report and 
we look forward to working with you on future homeland security issues. 

Sincerely, 

Signed by: 

Penelope G. McCormack: 
Acting Director: 
Departmental Audit Liaison: 

[End of section] 

Appendix III: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

John Hutton, (202) 512-4841, or huttonj@gao.gov: 

Staff Acknowledgments: 

In addition to the individual named above key contributors to this 
report were Amelia Shachoy, Assistant Director; Don Springman; Jeffrey 
Hartnett; Sean Seales; Alex Winograd; Karen Sloan; Julia Kennon; Lynn 
Milan; Art James; Marie Ahearn; and Kenneth Patton. 

[End of section] 

Footnotes: 

[1] GAO, Defense Acquisitions: Tailored Approach Needed to Improve 
Service Acquisition Outcomes, [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-07-20] (Washington, D.C.: Nov. 9, 2006). 

[2] We excluded the Federal Emergency Management Agency (FEMA) because 
of atypical fiscal year 2006 spending on Gulf Coast hurricane relief 
efforts. 

[3] DHS has categorized major investments as Levels 1 and 2 and Level 3 
(information technology). For the purposes of our review, we focused on 
Level 1 investments. A Level 1 major investment is defined in DHS 
Management Directive 1400 as an investment with greater than $100 
million in total acquisition costs including planning, or for 
information technology investments, a life-cycle cost greater than $200 
million. 

[4] ICE officials told us that none of their major investments used a 
performance-based approach. 

[5] We reviewed either the contract or selected orders on the contract. 

[6] DHS's investment review process includes three phases: pre- 
acquisition, acquisition, and sustainment. The acquisition phase 
includes systems development, and the sustainment phase includes 
operations and maintenance. 

[7] Floyd D. Spence National Defense Authorization Act for Fiscal Year 
2001, Pub. L. No. 106-398 § 821(a) (2000) required that the Federal 
Acquisition Regulation (FAR) be revised to establish a preference for 
the use of a performance-based approach in the acquisition of services, 
which was done in FAR 37.102(a), providing that performance-based 
acquisition is the preferred method for acquiring services and 
generally is to be used to the maximum extent practicable. 

[8] FAR 37.601; FAR 37.602(b); FAR 37.604. A fourth element, 
performance incentives, is required where appropriate. 

[9] Floyd D. Spence National Defense Authorization Act for Fiscal Year 
2001, Pub. L. No. 106-398, § 821(a) (2000); FAR 37.102(a). 

[10] Office of Management and Budget, Office of Federal Procurement 
Policy, A Report on the Performance-Based Service Contracting Pilot 
Project, May 1998. 

[11] Office of Federal Procurement Policy Memorandum, "Increasing the 
Use of Performance-based Service Acquisitions," September 7, 2004. 

[12] Services Acquisition Reform Act of 2003, Title XIV, National 
Defense Authorization Act for Fiscal Year 2004, Pub. L. No. 108-136, § 
1423 (2003); National Defense Authorization Act for Fiscal Year 2006, 
Pub. L. No. 109-163, § 843 (2006) (extended the deadline for the 
Panel's report by 6 months). 

[13] Report of the Acquisition Advisory Panel to the Office of Federal 
Procurement Policy and the United States Congress, January 2007. 

[14] Responsibility for the acquisition function at DHS is shared 
between the CPO and each DHS component head. Eight DHS components have 
internal procurement offices with a Head of Contracting Activity (HCA) 
who reports directly to the component head and is accountable to the 
CPO. The eight components are: Coast Guard, CBP, FEMA, Federal Law 
Enforcement Training Center, ICE, Office of Procurement Operations, 
Secret Service, and TSA. The HCA for each component has overall 
responsibility for the day-to-day management of the component's 
acquisition function. See GAO, Department of Homeland Security: 
Progress and Challenges in Implementing the Department's Acquisition 
Oversight Plan, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-
900] (Washington, D.C.: June 2007). 

[15] In using the performance-based approach, sound contracting 
practices dictate that required contract outcomes or requirements be 
well-defined, providing clear descriptions of results to be achieved. 
FAR 2.101 specifically provides that a performance work statement for 
performance-based acquisitions describe the required results in clear, 
specific, and objective terms with measurable outcomes. 

[16] GAO, Coast Guard: Observations on the Fiscal Year 2008 Budget, 
Performance, Reorganization, and Related Challenges, [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-07-489T] (Washington D.C.: Apr. 
18, 2007). 

[17] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-20]; and GAO, 
Coast Guard: Status of Efforts to Improve Deepwater Program Management 
and Address Operational Challenges, [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-07-575T] (Washington D.C.: Mar. 8, 2007). 

[18] A letter contract authorizes a contractor to begin work and incur 
costs before reaching a final agreement on contract terms and 
conditions, including price. 

[19] GAO, Information Technology: Improvements for Acquisition of 
Customs Trade Processing System Continue, but Further Efforts Needed to 
Avoid More Cost and Schedule Shortfalls, [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-08-46] (Washington, D.C.: Oct. 
25, 2007). 

[20] The contractor testified in October 2007 before subcommittees of 
the House Homeland Security Committee, that it spent approximately 
double the original contract amount.Because this is a firm fixed price 
contract, absent changes or delays created by the government, CBP is 
not responsible for these additional costs. 

[21] GAO, Secure Border Initiative: Observations on the Importance of 
Applying Lessons Learned to Future Projects, [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-08-508T] (Washington D.C.: Feb. 
27, 2008). 

[22] GAO, Homeland Security: Departmentwide Integrated Financial 
Management Systems Remain a Challenge, [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-07-536] (Washington, D.C.: June 
21, 2007). 

[23] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-575T]. 

[24] Department of Homeland Security Inspector General, Major 
Management Challenges Facing the Department of Homeland Security, OIG- 
08-11 (January 2008). 

[25] Department of Homeland Security Inspector General, Transportation 
Security Administration's Information Technology Managed Services 
Contract, OIG-06-23 (February 2006). 

[26] Department of Homeland Security Inspector General, Evaluation of 
TSA's Contract for the Installation and Maintenance of Explosive 
Detection Equipment at United States Airports, OIG-04-44 (September 
2004). 

[27] GAO, Aviation Security: Preliminary Observations on TSA's Progress 
to Allow Airports to Use Private Passenger and Baggage Screening 
Services, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-126] 
(Washington, D.C.: Nov. 19, 2004); and GAO, Aviation Security: Progress 
Made to Set Up Program Using Private-Sector Airport Screeners, but More 
Work Remains, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-166] 
(Washington, D.C.: Mar. 31, 2006). 

[28] TSA awarded a contract in January 2007 to issue biometric 
credentials to maritime workers. TSA officials told us it took more 
time than originally planned to bring the TWIC prototype enrollment 
system into alignment with new government security standards. During 
the course of our review, the schedule to initiate work was delayed 
from March to October 2007. 

[29] GAO, Transportation Security: DHS Should Address Key Challenges 
before Implementing the Transportation Worker Identification Credential 
Program, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-982] 
(Washington, D.C.: Sept. 29, 2006). 

[30] FAR 37.604 and 46.401. 

[31] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-489T]. 

[32] GAO, Framework for Assessing the Acquisition Function at Federal 
Agencies, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-218G] 
(Washington, D.C.: Sept. 2005), and GAO-07-20. 

[33] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-20]. 

[34] Department of Homeland Security Inspector General, Risk Management 
Advisory for the SBInet Program Initiation, OIG-07-07 (November 2006). 

[35] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-508T]. 

[36] We have ongoing work on DHS's acquisition workforce and plan to 
report on these issues in the final product for that engagement. 

[37] Services Acquisition Reform Act of 2003, Div. A, Title XIV, 
National Defense Authorization Act for Fiscal Year 2004, Pub. L. No. 
108-136, § 1413 (2003) allowed federal agencies to exercise direct hire 
authority in accordance with regulations issued by the Office of 
Personnel Management. This authority was extended to September 30, 
2012, by the National Defense Authorization Act for Fiscal Year 2008, 
Pub. L. No. 110-181, Div. A, Title VIII, § 853 (2008). 

[38] General Services Administration Modernization Act, Pub. L. No. 109-
313, § 4 (2006) granted agency heads authority, in consultation with 
the OPM and OFPP, to approve reemployment of annuitants in fields 
related to acquisition management. 

[39] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-218G]. 

[40] The 138 contracts were awarded in fiscal years 2005 or 2006 with 
obligations greater than $1 million, totaling about $1.3 billion. 

[41] Report of the Acquisition Advisory Panel to the Office of Federal 
Procurement Policy and the United States Congress, January 2007. 

[42] Known as Indefinite Delivery Indefinite Quantity (IDIQ) contracts, 
these contracts provide for an indefinite quantity of supplies or 
services within stated limits during a fixed period of time. The 
government places orders for individual requirements. 

[43] GAO, Reliability of Federal Procurement Data, [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-04-295R] (Washington, D.C.: Dec. 
30, 2003); GAO, Improvements Needed to the Federal Procurement Data 
System-Next Generation, [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-05-960R] (Washington, D.C.: Sept. 27, 2005); and General 
Services Administration Inspector General, Review of the Federal 
Procurement Data System-Next Generation (FPDS-NG), Report Number 
A040127/O/T/F06016 (March 2006). 

[44] For fixed price contracts, a specified price is paid regardless of 
the contractor's costs, minimizing the financial risk to the 
government. For cost reimbursable and time and materials contacts, the 
government generally assumes the risk of cost overruns. For example, 
cost reimbursable contracts provide for the government to pay 
reasonable, allowable and allocable costs incurred by the contractor up 
to the contract's price ceiling. Time and materials contracts provide 
for acquiring services on the basis of fixed costs for labor hours and 
materials at the stated contract ceiling price. In time and materials 
contracts, if services delivered do not meet contract requirements and 
the government exercises its right to have the contractor correct the 
deficiencies, the government pays the additional labor and material 
costs to do so, excluding profit. 

[45] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-900]. 

[46] DHS has categorized major investments as Levels 1 and 2 and Level 
3 (information technology). For the purposes of our review, we focused 
on Level 1 investments. A Level 1 major investment is defined in DHS 
Management Directive 1400 as an investment with greater than $100 
million in total acquisition costs including planning, or for 
information technology investments, a life-cycle cost greater than $200 
million. 

[47] During our review TSA was exempt from the FAR and followed the 
Acquisition Management System, developed by the Federal Aviation 
Administration, which included acquisition policy providing that 
service contracts should incorporate performance-based contracting 
methods. 

[48] We selected eight major investments and reviewed either a 
performance-based contract or selected orders on a performance-based 
contract associated with each investment. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office: 
441 G Street NW, Room LM: 
Washington, D.C. 20548: 

To order by Phone: 
Voice: (202) 512-6000: 
TDD: (202) 512-2537: 
Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: