XBAT Contracting and Project Management
March 26, 2004
Evaluation Report No. 04-014
![FDIC](https://webarchive.library.unt.edu/eot2008/20090114034638im_/http://fdicoig.gov/images/fdicsmall3.gif) Federal Deposit Insurance Corporation
Office of Audits
Office of Inspector General
Washington, D.C. 20434
DATE: March 26, 2004
MEMORANDUM: Arthur Murton, Director, Division of Insurance
and Research and Michael E. Bartell, Chief Information Officer and Director,
Division of Information Resources Management
FROM: Russell A. Rau [Electronically produced version;
original signed by Russell Rau], Assistant Inspector General for Audits
and Michael MacDermott [Electronically produced version: original signed
by Michael MacDermott, Acting Director, Office of Internal Control Management
SUBJECT: XBAT Contracting and Project Management (Report Number 04-014)
The subject final report is provided for your information and use. Please refer to the Executive Summary section for the overall results of our review. Our evaluation of responses is incorporated into the body of the report, and the Division of Administration’s written response is included in its entirety as an appendix to the report.
If you have any questions concerning the report, please contact Stephen Beard, Deputy Assistant Inspector General for Audits, at (202) 416-4217, or Marshall Gentry, Director, Corporate Evaluations, at (202) 416-2919. We appreciate the courtesies extended to the evaluation staff.
Attachment
cc: Arleas Upton Kea, DOA
|
Executive Summary
At the request of Corporation senior managers, we evaluated the Extensible Business Reporting Language (XBRL) Business Analyst Tool (XBAT) contracting and development effort. The XBAT is an important component of the larger Call Report Processing Modernization (Call Mod) project, an FDIC Chairman initiative and interagency effort to develop a centralized data collection, validation, integration, and distribution system to replace the existing Call Report process. A key part of the Call Mod project is the implementation of a Central Data Repository (CDR) that will use emerging technologies to effectively collect and process Call Report data. FDIC tasked the Institution Data Management project team (IDM team), within the Division of Insurance and Research (DIR), with implementing the XBAT and CDR efforts. The FDIC engaged a contractor to develop the first version of XBAT to certain specifications and promised the delivery of XBAT under the larger CDR contract. However, following CDR contract award, the winning CDR bidder determined that XBAT did not meet all of the specifications the FDIC had promised to the CDR vendor.
EVALUATION OBJECTIVE
FDIC executives expressed concerns that the XBAT product delivered to the CDR contractor was not fully functional and asked us to identify lessons learned in contract administration and project management that could be applied to the larger CDR effort. Accordingly, our objective was to assess the adequacy of FDIC’s contract administration and project management for the development of XBAT. Appendix I discusses our objective, scope, and methodology.
WHAT WE FOUND
The FDIC received some value from the XBAT procurement. The XBAT contractor was to develop and deliver two products: (1) a framework of taxonomies for use by software vendors to develop Call Report software for financial institutions’ use and (2) a software product (XBAT) for creating, editing, and publishing taxonomies. Taxonomies contain common terms, definitions, and relationships for Call Report data items. DIR and the CDR contractor concluded the taxonomies were well developed and are being used by the software vendors to develop software. DIR considered it imperative to deliver the taxonomies to the software vendors early in the process to ensure timely implementation of the Call Mod project.
The development of the XBAT software was not successful. The FDIC determined that XBAT lacked the functionality to successfully manage Call Report taxonomies. As a result, FDIC issued a change order to the CDR contract for the development of a Metadata Management Tool (MMT) that will replace XBAT. According to the CDR contractor, development of the MMT should not impact the overall milestones for implementing the CDR.
We concluded that the XBAT procurement effort was impaired by a number of contracting and project management issues. Specifically:
- FDIC did not establish XBAT functional requirements early enough in the contracting process. As a result, neither the FDIC nor the XBAT contractor had a clear and consistent understanding of the functionality the XBAT should include.
- FDIC did not adequately plan or execute the XBAT procurement effort. To meet cost and time constraints, the IDM team waived important XBAT requirements intended to support the overall CDR effort. Consequently, the quality of the XBAT software suffered. Further, because the procurement did not consistently follow the FDIC Acquisition Policy Manual (APM), the FDIC was party to a contract that did not protect the Corporation’s best interests.
- The contracting officer (CO) and oversight manager (OM) did not completely fulfill their responsibilities and relied on the technical monitor (TM) for making changes to the functional requirements, accepting contract deliverables, and approving a key invoice payment. As a result, the procurement did not have adequate checks and balances to help ensure the project’s success.
- FDIC did not exercise adequate change management or project integration during the XBAT development effort and the CDR solicitation process. The CDR solicitation promised a fully functioning XBAT at the same time that the IDM team was waiving important functional requirements in order to meet cost constraints and product delivery milestones. As a result, FDIC promised a software product containing functionality that it could not deliver and had to reprocure the effort.
- Communication within and between the IDM team, senior level managers, and the acquisition team (CO, OM, and TM) was not effective. Accordingly, DIR executives and CDR bidders were not aware of the true status of the XBAT development effort.
- XBAT development did not fully comply with FDIC’s System Development Life Cycle (SDLC). Most importantly, FDIC paid the XBAT contractor before completely testing XBAT. Thus, FDIC had limited financial leverage to compel contractor performance.
These conditions occurred because the FDIC did not always follow established acquisition procedures, prudent project management practices, or the SDLC. As a result, the FDIC paid for software that did not meet all corporate needs or expectations and had to issue a change order to redevelop the software. Moreover, the conditions exposed the FDIC to risks for potential delays in the CDR project and to reputation risk in the eyes of the CDR contractor, other banking agencies, and FDIC-insured institutions (see Appendix II).
This report identifies a number of lessons learned. Table 1 presents the most important lessons to be applied to future system development contracting efforts.
Table 1: Key Lessons to be learned from the XBAT Development Effort
Contracting |
Project Management |
Functional requirements should be established early in the procurement
process. |
Closely coordinate projects that are interdependent, and promise only
what can be realistically delivered. |
Changes to system requirements should not be driven solely
by resource and schedule constraints - consideration must also be
given to impact on business needs. |
Continually evaluate communication channels to ensure
they are open and effective and provide balanced and complete information
about a project's
status and viability at key development stages. |
The CO, OM, and TM should work together as a team in meeting their assigned
responsibilities as part of establishing a control framework. |
Follow an SDLC methodology that is appropriate for the development effort
and perform adequate testing of deliverables before making payments. |
Source: OIG analysis
BACKGROUND
The FDIC is a member of the Federal Financial Institutions Examination Council (FFIEC), a formal interagency body empowered to prescribe uniform principles, standards, and report forms for the federal examination of financial institutions and to make recommendations to promote uniformity in the supervision of financial institutions. (Note: Other members of the FFIEC include the Board of Governors of the Federal Reserve System (FRB), the Office of the Comptroller of the Currency (OCC), the Office of Thrift Supervision, and the National Credit Union Administration.) The FFIEC has oversight responsibilities for the collection of Call Reports filed electronically by banks on a quarterly basis. The Call Report shows a bank’s condition and income and is used for multiple purposes including assessing the financial health and risk of the institution.
All commercial banks insured by the FDIC and all FDIC-supervised savings banks are required to submit quarterly Call Reports. Banks have 30 days following the quarter-end to submit their completed Call Reports electronically, and banks with foreign offices have 45 days. Banks have the option to use vendor-supplied software to compile and submit their Call Reports.
The FRB processes data for approximately 1,000 banks using a distributed process across 12 Federal Reserve District Banks. The FDIC processes data for approximately 7,700 remaining banks using a centralized process at its Headquarters location. Although each agency stores the data separately, ultimately the data are shared. The FDIC and FRB validate Call Report data for mathematical errors and logical relationships, and their staff addresses exceptions by contacting respondents and inputting explanations and corrections.
The FDIC, FRB, and OCC collaborated to improve the collection and management of financial data starting with the information gathered in the Call Reports. This interagency effort, called the Call Mod project, is intended to provide for collection, storage, and distribution of bank Call Report data in a central repository shared by the three agencies. The Call Mod project called for the FDIC to award a multi-million dollar contract to a private-sector contractor to design, build, and manage the CDR. The primary objectives of the Call Mod project are illustrated in Table 2.
Table 2: Call Mod Project Objectives
Project Objectives |
Decrease the time between the receipt of Call Report data by the
regulators and the release of data to agency and public users.
|
Increase the FFIEC Call Agencies' flexibility to adjust systems quickly
to accommodate changing information needs.
|
Increase consistency in Call Report data available for use within
the FFIEC Call Agencies by implementing a central repository with consolidated
collection and validation processes.
|
Decrease the overall cost of collecting, validating,
integrating, and distributing Call Report data.
|
Improve the transparency of FFIEC Call Agency data requirements
by facilitating the exchange of information through the use of standards
to describe both financial and technical reporting content.
|
Improve the efficiency of value-added calculations by
performing these calculations one time and using the results whenever
needed.
|
Source: CDR Solicitation Draft Statement of Work
The CDR was premised on using flexible technologies, one of which is a data exchange standard known as XBRL. XBRL is based on the eXtensible Mark-up Language, a set of Internet standards, and is geared to tagging each piece of data with enough information to speed its exchange among users. XBRL taxonomy is a description and classification system for the contents of financial statements and other business reporting documents. Taxonomies represent up to hundreds of individual business reporting concepts, mathematical relationships, and definitions.
The FDIC established the IDM team to identify and implement strategies to improve the collection, validation, integration, and distribution of financial institution data—the IDM team is the program office for the XBAT and CDR contracts. IDM is organizationally located within DIR. Appendix III presents a timeline of key events in the XBAT development process.
Benefits of and Challenges Impacting the XBAT Development Effort
The FDIC received some value from the XBAT engagement. DIR and the CDR vendor concluded the taxonomies were well developed and are being used by the software vendors to develop software. DIR considered it imperative to deliver these taxonomies to the software vendors early in the process to ensure timely implementation of the Call Mod project. The IDM team also noted that it learned a great deal about XBRL technology and functional requirements for a business analyst tool from the XBAT effort and that the CDR contractor was able to build on this knowledge in developing the MMT.
Our conclusions related to the adequacy of contract administration and project management of the XBAT procurement are presented in the context of recognizing that the IDM team faced a number of challenges during the time the XBAT services were being procured. XBRL technology was a new standard that had not been used in the federal government, and the IDM team had to gain acceptance from three agencies – FDIC, FRB, and OCC – that the XBAT would work for the overall CDR. Further, several IDM team members told us that the team had to contend with the pressure of delivering a product within tight time constraints and limited funding concurrent with satisfying the FDIC Chairman’s Call Mod initiative and the corporate performance objective of the FDIC making high-quality banking data available to the public on a more timely basis.
Functional Requirements
The FDIC did not establish XBAT functional requirements early enough in the contracting process. As a result, neither the FDIC nor the XBAT contractor had a clear and consistent understanding of what functionality the XBAT should include. |
What Should Have Happened
FDIC’s APM, section 4.A.4.a. specifies that the program office—in this case DIR, or more specifically, the IDM team—is responsible for identifying procurement requirements and providing a clear and specific description of the goods or services required. In doing so, the program office is responsible for preparing a Statement of Work (SOW) that clearly delineates what is to be procured. The APM states that, in developing the SOW, a thorough understanding of the required goods or services and expected results is critical. Key items to be considered and conveyed through the SOW include the nature of the services and minimum standards that must be met and the deliverables and scheduled milestones for service delivery.
FDIC Circular 1320.3, System Development Life Cycle, (SDLC) Version 3, outlines provisions that apply to all information technology project efforts, including new system development efforts as well as enhancing and converting existing systems. One of the eight phases of the SDLC is the Requirements Definition Phase, during which user performance needs are analyzed and user requirements are formally defined. Requirements for functionality, data, system performance, security and internal control, portability, and maintainability are defined to a level of detail sufficient for system design to proceed. Requirements are formally documented in the functional requirements document (FRD) and approved by the client and developer.
Text Box: The SOW for the XBAT project stipulated that the FDIC would
provide the contractor a draft FRD for the business analyst tool. The SOW
also stated that, in all cases, the principles, phases, and deliverables
of the FDIC SDLC as described in Circular 1320.3 must be applied. The tasks
outlined in the XBAT SOW included a statement that, during the SDLC Requirements
Definition Phase, the FDIC will analyze and define system requirements,
but the contractor may be requested to assist in defining these requirements.
The SOW also included a statement that the FDIC will work with the contractor
to gather requirements to develop an FRD to be used by the contractor to
design, develop, and test the system. |
What Actually Happened
XBAT functional requirements were not established early enough in the contracting process and evolved during the XBAT engagement. The IDM team and the XBAT contractor drafted the first FRD in late September 2002, followed by two additional draft FRDs dated October 2, 2002 and October 16, 2002, respectively. A subsequent FRD was drafted in June 2003, 10 months after the effective date of the contract. The October 2002 and June 2003 FRDs reduced the requirements specified in the September 2002 FRD. Because the functional requirements were evolving, neither the FDIC nor the XBAT contractor had a clear understanding of what functionality the XBAT should include.
XBAT contractor representatives indicated that the FDIC provided a draft FRD in August 2002, but noted the FRD was not detailed enough and that some requirements were unrealistic and reflected a lack of understanding about system capabilities and requirements. The XBAT contractor spent about 2 months adding detail to the FRD. The contractor estimated there were
5 to 10 drafts of the FRD, the last versions being drafted in October 2002. The contractor was not aware of the June 2003 FRD. Table 3 summarizes the various FRDs and their impact on the XBAT and CDR projects.
Table 3: XBAT Functional Requirements
Functional Requirements |
Description |
Impact on XBAT and/or CDR |
September 2002
FRD
(Draft)
|
Includes 5 categories of 57 requirements: functional,
hardware, software, performance, and security. Software requirements
are described as "To
Be Determined." Also includes SDLC reference and deliverables. |
Provided to CDR bidders on September 11, 2002. |
October 2, 2002
FRD
(Draft)
|
Includes the categories of requirements mentioned in
the September FRD, except for hardware requirements being described as "To
Be Determined."; Software requirements are described. Excludes SDLC
reference and deliverables, except for a Table of Contents reference. |
Provided to CDR bidders in the Request for Proposal,
Amendment 09, with an effective date of October 3, 2002. |
October 16, 2002
FRD
(Draft)
|
Includes the same categories of requirements as
the October 2, 2002 FRD. Hardware requirements are described as "To
Be Determined." Excludes SDLC reference and deliverables. Includes
two new functional requirements and one new performance requirement. |
XBAT Task Order 01, signed December 16, 2002, and
effective August 9, 2002, included a provision for the contractor to
develop XBAT to meet requirements described in the FRD with the final
date of October 16, 2002.
However, XBAT Task Order 01 also waived 18 requirements listed in the
October 16, 2002 FRD.
|
June 2003
FRD
(Draft)
|
Includes 5 categories of requirements identified
in the September 2002 FRD, but excludes 17 of the remaining specific
requirements listed in the September 2002 FRD. Some of the exclusions
are in the areas of application testing, system outputs, and reliability
requirements. The June 2003 FRD included three software requirements
and one system sizing and growth requirement that were not included in
the September 2002 FRD. The June 2003 FRD also included SDLC documentation
requirements. |
A fully functioning XBAT was not delivered to the
CDR contractor. The FDIC issued a $840,000 change order to the CDR contractor
for an MMT to replace XBAT. |
Source: XBAT FRDs, Task Order 01, and OIG AnalysisThe CDR contractor assessed the XBAT software delivered by the FDIC and determined that it did not meet the September 2002 FRD promised in the CDR solicitation. The CDR contractor concluded that the XBAT contractor had made significant progress in proving basic XBRL concepts and had also succeeded in capturing the underlying principles involved in creating XBRL taxonomies. However, the CDR contractor noted that its bid on the CDR contract was based on all XBAT FRD requirements being met, and the contractor expected to receive design documents and code as part of the XBAT that could be used in building CDR components.
Text Box: The CDR contractor’s assessment of three major requirement categories noted the following issues relative to the XBAT:
• The XBAT is able to produce taxonomies. However, there are some omissions from the requirements in the original FRD, such as validation, import/export, and print capabilities.
• The XBAT provides management of taxonomy source data. The remaining meta-data systems management requirements are CDR requirements.
• Virtually none of the meta-data testing has been implemented.
|
The FDIC agreed with the CDR contractor’s
assessment of XBAT and awarded a separate
contract to the CDR contractor to replace the
XBAT with a fully functioning MMT. The MMT
is a new tool for creating, editing, and publishing taxonomies.
Consequence
Because functional requirements were evolving during the XBAT development, FDIC and the XBAT contractor did not have a clear understanding of what functionality the XBAT should include.
Lessons Learned
Based on our discussions with DIR, the Division of Information and Resources Management, and Acquisition Services Branch (ASB) officials involved with the XBAT and CDR projects, we offer the following lessons learned:
- Functional requirements should be established early in the procurement process.
- When identifying procurement requirements, provide a clear and specific description of the goods or services required.
Procurement Planning and Execution
The FDIC did not adequately plan or execute the XBAT procurement effort. The IDM team waived important XBAT functions that were intended to support the overall CDR effort to meet cost and time constraints. Consequently, the quality of the XBAT software suffered. Further, because the procurement did not consistently follow the APM, the FDIC was party to a contract that did not protect the Corporation’s best interests. |
What Should Have Happened
The APM provisions for preparing a procurement requirements package include providing a clear and specific description of the goods or services required and a price estimate for the goods or services being requested, including the base period and all option periods. Preparing a cost estimate requires an identification of the cost components that apply to the proposed contract. Identifying these components requires a clear understanding of the goods or services being procured.
The APM provides for two basic types of pricing arrangements used in FDIC contracts, fixed price and level of effort. The APM suggests that the successful use of a firm fixed price arrangement requires a clear definition of requirements in the SOW and realistic estimates of work to be performed. A level of effort contract, such as a time and materials contract, is used when it is difficult to provide a detailed SOW or to estimate the price or duration of time required for performance.
The APM’s provisions for awarding a contract based on competitive range determination state that, following the initial evaluation of bids, if there is no one successful offeror and if there is a need to hold technical and/or cost discussions with offerors, the CO may establish a competitive range with those firms that have a reasonable chance of being selected for award. After discussions are completed, offerors are given the opportunity to improve their proposals through a best and final offer phase. The CO is responsible for ensuring that FDIC personnel do not: (1) help an offeror bring its proposal up to the level of other proposals through successive discussion opportunities; (2) disclose technical information pertaining to one proposal that results in improvement of a competing proposal; (3) indicate to an offeror a price that it must meet to obtain further consideration; or (4) furnish information about other offerors’ proposed prices.
The APM contains various provisions regarding basic ordering agreements (BOAs) and task orders, SDLC contracts and task assignments, contractual documents, official contract files, and legal review of contracting documents.
BOAs and Task Orders: The APM identifies a BOA as a written agreement between the FDIC and a contractor containing terms and conditions that will apply to FDIC-issued task orders during its term, a description of the services to be provided, and the method(s) for the pricing and issuing of orders under the agreement. A BOA is not a contract because it does not require that orders be placed and does not contain consideration. Any task order issued against a BOA, in accordance with the terms and conditions contained in that BOA, becomes a contractual instrument against which funds are obligated as consideration in exchange for the goods or services specified in the task order.
The APM procedures for awarding task orders include a requirement that the program office provide the CO with written requests for all task orders under a BOA. The request should be in the form of a memorandum or letter accompanied by a written statement of work specific to the assignment being requested. As shown in Table 4, The APM includes the following minimum requirements for requesting task orders and what a task order should include.
Table 4: APM Task Order Requirement
Request for Task Order Requirement |
Task Order Requirements |
Include the nature of services and minimum standards
that must be met, qualifications necessary to perform the work, deliverables
and scheduled milestones for their delivery, and standards by which the
contractor's performance will be measured. |
Incorporate terms and conditions of the BOA |
Detailed cost estimate |
Contain or incorporate a statement of work for the specific
task to be performed |
Technical evaluation criteria, if award is to be
based on other than price. |
Specify milestones with a schedule of deliverables. |
Weighting of technical and price proposals, if award
is to be based on other than price. |
State a period of performance. |
Procurement Requisition |
Pricing information, including ceiling prices for labor
and travel. |
SDLC Contracts and Task Assignments: DIRM generally uses contracts with task assignments for SDLC activities. These contracts allow the OM to issue individual task assignments under the terms of the contract. In this way, the OM can determine the timing and scope of deliverables to be provided under a task assignment without having the CO issue the task assignment. The contract and task assignment SOW should refer to the SDLC.
The program office is responsible for developing the task assignments, ensuring that (1) the total value of the various task assignments issued pursuant to a contract does not exceed the original expenditure amount of the contract and (2) that the work detailed in the task assignments does not go beyond the scope of work set forth in the contract. The OM signs the task assignments, and a copy is sent to the CO who has responsibility to retain the task assignment in the official contract file.
Contractual Documents and Official Contract Files: The APM states that the CO shall prepare and assemble the appropriate contractual documents. The CO is responsible for sending two originals of the contract to the contractor for signature, executing both copies of the contract on behalf of the FDIC, returning one of the originals to the contractor, and retaining the other original contract in the official contract file. The CO is also responsible for providing a copy of the contract to the designated OM.
The APM states that it is good business practice to have fully executed documents in place before a contractor commences work. However, on an as-needed basis, and only with the prior approval of the Assistant Director, Headquarters Acquisition Section, or head of the regional contracting function, the effective date of a contract may be prior to the execution date. This provision should be utilized when the CO intends to orally authorize a contractor to begin work prior to having in place a fully executed contractual document.
The APM states that the ASB is responsible for maintaining the official contract file associated with all phases of the contracting process. Each CO is responsible for establishing a contract file for each new contract and ensuring that all documents are filed prior to contract award according to the FDIC Formal Contracting File Checklist, attached as Exhibit XXIII to the APM. Documentation shall include that which is required under the APM relating to pre-solicitation, solicitation, evaluation, selection, pre-award reviews, and the award decision.
Legal Review of Contracting Documents: The APM identifies the Contracting Law Unit within the FDIC’s Legal Division as a member of the team supporting the FDIC’s contracting process. The Unit supports the development of contracting policy and procedures and provides advice and legal sufficiency reviews. The APM stipulates procurement responsibilities for the Legal Division, including requirements to (1) review solicitation packages for contracts of $100,000 or more; (2) review complex contracting requirements, as requested by the CO; (3) provide advice as required on issues involving contract scope; and (4) provide other assistance as requested by the CO. The APM does not specifically require that the Legal Division review contract documents unless requested by the CO.
What Actually Happened
ASB and the IDM team did not adequately plan the XBAT contract, and the IDM team underestimated the costs to develop XBAT. Although XBAT was a research and development effort employing new technology, ASB and the IDM team used a firm fixed price contract with short time frames and evolving requirements. Firm fixed price contracts are generally appropriate for low-risk engagements with well-defined requirements. Given the urgent need to develop the XBAT in time for delivery to the CDR vendor and the evolving functional requirements, the XBAT procurement would most likely not be categorized as a low-risk engagement suitable for firm fixed pricing. Appendix IV presents how XBAT procurement requirements changed over the course of the contract.
As shown in Figure 1, the costs for developing an XBRL business analyst tool increased from an early estimate of almost $500,000 to a cost of more than $1.5 million (considering XBAT and MMT development costs). We observed that XBAT development costs increased by more than 80 percent from the BOA to Task Order 01 at the same time that the IDM team was waiving important functional requirements.
[D]
XBAT contractor representatives informed us that they provided labor rates
and labor hour estimates in response to the initial XBAT request for proposal.
However, the XBAT contractor told us that during the best and final offer
period, FDIC reduced the number of option years and imposed a price ceiling
based on
FDIC budget authority. In response, the XBAT contractor presented a best
and final offer bid of $370,000 to develop XBAT—almost 26 percent lower than the FDIC’s
estimate. FDIC officials we interviewed did not recall providing a price
ceiling or FDIC budget authority information to the bidders during the best
and final
offer period.
Shortly after the contract had been awarded, the TM questioned the adequacy of contract funding for developing XBAT. Further, just 2 months following the start of work, the XBAT contractor indicated that the option year on the contract would need to be executed because the base year funding had been expended. The XBAT contractor also indicated at that time that work could be completed by the end of December 2002 for total funding of $1.1 to 1.3 million, nearly double the firm fixed price negotiated for the XBAT contract, unless requirements were reduced. The XBAT contractor acknowledged that, with the benefit of hindsight, its best and final offer bid was not sufficient to complete the XBAT contract requirements.
Moreover, the milestones established for development of the taxonomies and XBAT appear unrealistic. The TM said that, in hindsight, the 4 month timeframe to develop and deliver the XBAT was unrealistic, especially for new technology. The TM further stated that the goal of developing a fully functioning XBAT by December 2002 to accommodate the CDR was also unrealistic. The IDM project manager informed us that the FDIC awarded bonus points to the XBAT contractor during the best and final offer for early delivery of the taxonomies and software. An unsigned July 17, 2002 technical evaluation results memorandum providing conclusions regarding the technical merits of bidders’ oral presentations stated that the XBAT contractor provided a reasonable time schedule for performing the required tasks.
The XBAT contract award and execution also did not fully comply with APM provisions. ASB advertised the XBAT solicitation as a task-order-based contract, but ASB executed a BOA and issued task assignments that are typically used with an SDLC contract. The CO issued an advanced authorization letter on August 9, 2002, authorizing the XBAT contractor to proceed with work while contracting documents were being finalized. However, ASB and the contractor did not execute a task order for the BOA until December 16, 2002, 4 days before the XBAT products were scheduled to be completed for delivery to the FDIC. We saw no evidence that the ASB Assistant Director approved the effective date of the contract, as required by the APM. Further, as illustrated in Table 5, we noted several other contract award and execution activities that were departures from APM provisions.
Table 5: XBAT Contract Award and Execution Departures from APM Provisions
Description of Action in Question
|
The CO did not provide a copy of the signed contract for the XBAT to the designated OM. |
The task order contained a nearly 2-year
period of performance (August 9, 2002 through June 30, 2004) that differed
from the BOA’s 1-year period of performance, starting August 9, 2002
and ending August 9, 2003. |
The firm fixed price of the task order exceeded the BOA compensation ceiling by $30,000. |
Key documents associated with the XBAT
contract award and execution were missing from the official contract
file, including the unsuccessful bidders’ proposals, a signed selection
recommendation report, a SOW for the signed BOA, and a SOW for the task
assignments and task order. |
Changes and waivers of functional requirements
were made without the CO’s approval. The OM issued two task assignments – one
that waived a number of contracting requirements without authorization
from the CO. |
Source: OIG Analysis
Text Box: In October 2003, the FDIC awarded a change order to the CDR contract that specifically identifies a breakdown of the firm fixed price of the contract for each major deliverable. The contract specifies four project deliverables, including a master project plan and a design document, and identifies related payments and completion dates for each deliverable. In addition, the contract includes an acceptance clause that requires the contractor to obtain an acceptance signature by FDIC of formal deliverables. The acceptance procedures state that the FDIC will accept the deliverables as complying with the SOW or provide a written statement that identifies in reasonable detail significant deviations between the deliverable and the statement of work. FDIC did not include such an acceptance clause in the XBAT task order. |
In regard to the Legal Division’s involvement in XBAT procurement activities, we interviewed the Counsel assigned to the XBAT procurement effort. The Counsel indicated that his role was to review the contract and associated task orders for legal sufficiency. The Counsel reviewed the solicitation package, documents associated with the August 2002 advanced authorization letter, and the December 2002 Task Order 01. The Counsel noted that ASB and the IDM team were in a hurry and that there was a rush to issue the contract documents. We made the following observations about the Counsel’s review of XBAT procurement documents:
- The Counsel reviewed the solicitation documents as required by the APM. However, ASB issued the XBAT Request for Proposal 5 days prior to receiving the Counsel’s notice of review and acceptance.
- The CO indicated that he would not honor the TM’s request for an advanced authorization allowing the contractor to commence work prior to awarding a contract without the Legal Division’s review of the Intellectual Property (licensing) terms of the XBAT software. The Counsel provided contract language to the CO regarding XBAT licensing provisions prior to the issuance of the advanced authorization letter.
- We saw no evidence in the XBAT contract files of the Legal Division’s review of the BOA, although the APM does not specifically require a legal review of the contract unless such a review is requested by the CO.
- The Counsel informed us that he did not review the XBAT task assignments.
- In regard to the task order, the XBAT contracting and project management files indicated that the TM developed the task order and sent it electronically to the CO, OM, and IDM team for review on October 23, 2002. The XBAT contract files indicated that the CO did not request the Legal Division to review the revised XBAT task order until December 10, 2002, 6 days prior to the execution of the task order and 10 days prior to the scheduled date of XBAT delivery.
Consequences
Because the XBAT procurement was not adequately planned or executed, the quality of the XBAT software suffered. Further, because the procurement did not consistently follow the APM, the FDIC was party to a contract that did not protect the Corporation’s best interests.
Lessons Learned
- Changes to system requirements should not be driven solely by resource and schedule constraints—consideration must also be given to impact on business needs.
- Research and development efforts do not lend themselves to a firm fixed price contracting vehicle, given the uncertain and evolving nature of the services. The type of contract awarded should be based on the level of risk to the Corporation, with fixed price contracts generally used when risk has been reduced to a reasonable level.
- When procuring services for research and development, communicate to everyone involved that the project is a research and development effort, requiring closer monitoring.
- A time and materials level of effort contracting vehicle may be more suitable for a research and development engagement, given the difficulty in providing a detailed SOW or to estimate the price or time required for a research and development effort.
- Involve the Legal Division early in procurement planning, and subject key contracting documents such as the contract and SOW to legal review and concurrence prior to contract execution. Focus the legal review on both legal sufficiency and protecting the Corporation’s interests, particularly for contract provisions that pose the greatest risk for the Corporation.
Contract Administration
The CO and OM did not completely fulfill their responsibilities and relied on the TM for making changes to the functional requirements, accepting contract deliverables, and approving a key invoice payment. As a result, the procurement did not have adequate checks and balances to help ensure the project’s success. |
What Should Have Happened
The APM stipulates that contract administration encompasses oversight of all relationships between the FDIC and the contractor relating to contractor performance and provides the following responsibilities for the CO, OM, and TM.
Contracting Officer: Responsibility for ensuring compliance with the contract rests with the CO, who may delegate certain authorities to other FDIC personnel. The CO’s responsibilities include (1) monitoring contract performance for compliance with its terms and conditions; (2) enforcing contract provisions; (3) executing modifications to the contract; (4) assisting the OM or other program office representatives, as requested, in oversight of the technical business requirements of the contract; and (5) jointly with the OM, verifying costs incurred and invoiced to the FDIC under the contract and monitoring expenditures against the expenditure ceiling.
Oversight Manager: The OM is designated by the program office and has responsibility for (1) overseeing the performance requirements of the contract; (2) providing business and technical liaison between the FDIC and the contractor; (3) notifying the CO of any need to modify the contract; and (4) referring all questions regarding contract provisions to the CO. The OM is solely responsible for carrying out all duties and responsibilities set forth in the Letter of Oversight Manager Confirmation issued by the CO, with copies provided to the contractor and the OM.
Technical Monitor: It may be appropriate in very complex areas of performance to appoint one or more TMs for a contract. The duties of a TM are a subset of the duties of the OM, but the responsibility for contractor oversight remains with the OM. TMs are designated by the OM and appointed in a Letter of Technical Monitor Confirmation issued by the CO, with copies sent to the contractor and the OM.
The APM also requires that OMs and TMs be adequately trained to oversee contractor performance within the terms of the contract. The APM requires that OMs and TM attend the
FDIC Oversight Management Training course and the FDIC Contract Administration Training course and pass an examination for each course. Further, OMs and TMs must take a 1 day refresher course every 3 years.
The APM also requires a Contract Administration Plan (CAP) for contracts over $100,000. The objective of a CAP is to ensure that the CO and OM have a common understanding of both the contractor’s and the FDIC’s obligations under the contract. The CO prepares the CAP with the assistance of the OM immediately following contract award.
Text Box: The BOA executed for the XBAT engagement also included the following provisions regarding the OM and CO.
- FDIC Contracting Officer. The term “Contracting Officer” means a person designated in writing by the FDIC with FDIC delegated authority to enter into, modify, administer, and terminate contracts and orders. The Contracting Officer is the only person authorized by FDIC to issue any instructions or directions that affect any increase or decrease in the cost of this agreement, any change in delivery schedule or which change any other term of the agreement.
- FDIC Oversight Manager. The term “Oversight Manager” means the person designated in writing by the Contracting Officer to represent the FDIC for the purpose of monitoring technical performance under any task order awarded. The Oversight Manager is not authorized to issue any instructions or directions which effect any increase or decrease in the price of any task order awarded or which change the delivery date(s) or Period of Performance.
|
What Actually Happened
Contract administration was not adequate. The CO and the OM relied too heavily on the TM for approving deliverables, changing requirements, and approving invoice payments. Two DIRM OMs noted that the XBAT TM was very involved with the project and that both relied on the expertise and “hands-on” involvement of the TM for acceptance of some of the deliverables.
The TM sent an electronic message to the OM in early August 2002, requesting agreement on contract management authority and responsibilities. The electronic message was not copied to the CO. The TM acknowledged in the message that oversight management of the contract had been assigned to DIRM. However, the TM assumed responsibility for:
- technical management of the contract,
- the success of the project,
- developing and providing task assignments to the OM for review,
- managing day-to-day contract operations and informing the OM of contract
status, and
- directing the contractor’s work.
The first OM served in his position from August 2002 until late October 2002 at which time the second OM was appointed. The change in OMs was the result of corporate reorganization activities. The CO did not issue an APM-required Letter of Oversight Manager Confirmation to the first OM. In addition, the CO did not issue an APM-required Letter of Technical Monitor Confirmation to the TM. These letters were issued for the current OM and TM.
Moreover, the FDIC’s Training Management Server showed no record of the required
OM and contract administration training for the OMs or the TMs appointed for
the XBAT contract. We subsequently learned that the current OM completed OM training
and took an online refresher course.
Finally, the official contract files for the XBAT did not contain a CAP. The
OM and TM confirmation letters and a CAP would have been beneficial for the CO
and the OM to understand their roles in overseeing the contract, to ensure awareness
of specific deliverables and milestones, and to monitor changes to requirements,
timeframes, and funding.
Consequence
Because the CO and OM did not completely fulfill their responsibilities and relied too heavily on the TM for managing the contract, the procurement did not have adequate checks and balances to help ensure the project’s success.
Lessons Learned
- The CO, OM, and TM should work together as a team in meeting their assigned responsibilities as part of establishing a control framework.
- The OM should involve the CO in addressing contractor performance matters.
- OM and TM roles and responsibilities need to be clearly documented and communicated in the Letter of Oversight Manager (or Technical Monitor) Confirmation.
- A CAP is a good tool for helping ensure that the contractor performs in accordance with the terms and conditions of the contract and is required by the APM.
- A CAP allows both the OM and CO to identify specific deliverables and due dates and track future modifications and funding.
Change Management and Project Integration
The FDIC did not exercise adequate change management or project integration during the XBAT development effort and the CDR solicitation process. The CDR solicitation promised a fully functioning XBAT at the same time that the IDM team was waiving important functional requirements to meet cost constraints and product delivery milestones. As a result, the FDIC promised a software product containing functionality that it could not deliver and had to reprocure the effort. |
What Should Have Happened
The PMBOK® Guide (discussed below) and the APM include the following information relevant to change management and project integration.
Change Management: The Project Management Institute (PMI) has conducted extensive research and analysis in the field of project management and published a standards guide in 2000, entitled A Guide to the Project Management Body of Knowledge (PMBOK® Guide). The guide documents proven practices, tools, and techniques that have become generally accepted in the field of project management, including information systems development and implementation. The PMBOK® Guide is an approved standard of both the American National Standards Institute and the Institute of Electrical and Electronics Engineers. The guide identifies nine distinct knowledge areas associated with successful project management. The nine areas are integration, scope, time, cost, quality, human resources, communications, risk, and procurement management.
PMBOK® identifies project scope management as a subset of project management that includes the processes required to ensure that the project includes all the work required, and only the work required, to complete the project successfully. One of the processes involves controlling changes to project scope. Scope change control is concerned with influencing the factors that create scope changes to ensure that changes are agreed upon, determining that a scope change has occurred, and managing the actual changes when and if they occur. Scope change control must be integrated with the other control processes, such as scheduling, cost, quality, risk, and staffing.
The APM contains a chapter of provisions regarding changes and modifications to contracts and categorizes the types of changes as administrative or substantive. Substantive changes include a change in the amount of fees to be paid to the contractor, a change in the delivery schedule, and a change in the quantity and nature of deliverables. The CO is responsible for determining whether a proposed modification is within the scope of the contract. The OM is responsible for identifying the requirement for a modification, determining whether the cost to the FDIC caused by the modification will exceed the expenditure ceiling, and preparing a detailed, written explanation of the reason for and nature of the change or modification.
Project Integration: According to PMBOK®, the work of the project must be integrated with the ongoing operations of the performing organization. Project integration management includes the processes required to ensure that the various elements of the project are properly coordinated. It involves making tradeoffs among competing objectives and alternatives to meet or exceed stakeholder needs and expectations and includes project plan development, project plan execution, and integrated change control.
Integrated change control relates to coordinating changes across the entire project and is concerned with: influencing the factors that create changes to ensure that changes are agreed upon, determining that a change has occurred and managing the actual changes when and as they occur. One of the tools and techniques for integrated change control involves additional planning to allow for prospective changes that may require new or revised cost estimates, modified activity sequences, schedules, resource requirements, or other adjustments to the project plan. These changes must be communicated to appropriate stakeholders, as needed.
The CDR request for proposal included a copy of the FDIC’s General Provisions for contracts dated May 2002. The provisions contained the following language in a section addressing FDIC-furnished property:
The delivery or performance dates for this Contract are based upon the expectation
that FDIC-furnished property suitable for use (except for property furnished
as-is) will be delivered to Contractor at the times stated in the Statement
of Work, or, if not so stated, in sufficient time to enable Contractor to
meet the Contract’s delivery or performance dates.
What Actually Happened
Change Management: As discussed earlier, the FDIC issued Task Order 01 which waived certain functional requirements to meet the December 2002 deadline for delivery of the XBAT to the CDR winning bidder. Waived requirements included SDLC deliverables and the process to trace requirements and test all aspects of the system. However, the task order did not explicitly identify the SDLC and system testing waivers. The CO needed this information to determine whether the requested changes were within the scope of the original contract, as required by the APM for contract modifications.
Further, the OM and TM did not cross-reference the requirements waived in the task order to the individual requirements in the original FRD to clearly indicate those XBAT specifications that were no longer required. In October 2003, 10 months after the task order was signed, the IDM project manager prepared a schedule comparing the waived requirements to the October 2002 FRD. Nevertheless, the waived requirements should have been referenced at the time the task order was being negotiated especially because the FDIC had promised that the XBAT would be delivered to the CDR winning bidder in accordance with the September and October 2002 FRD specifications.
In September 2003, DIR appointed a new OM to oversee the remaining work on the XBAT contract. The OM prepared a list of requirements from the task order that had not been completed and discussed the requirements with the XBAT contractor. The XBAT contractor told the OM that the contractor believed that the FDIC agreed to waive these requirements, but the waivers were documented only in status reports and electronic messages rather than being handled through the formal contracting modification process. The OM estimated the cost of the unfinished work to be $40,000.
Text Box: In October 2003, the FDIC awarded a change order to the CDR contract to develop an MMT to replace XBAT. The MMT change order contained change control procedures as part of the contract requirements. The contract requirements state that, in the event that the FDIC or the contractor wishes to alter the statement of work, various procedures must be followed, including (1) documenting the description and reason for the proposed change; (2) coordinating change requests with the OM; and (3) investigating the impact of the change request on the price, timetable, statement of work, and specifications and relevant obligations under the agreement. These change control procedures would have benefited the XBAT engagement. |
Project Integration: XBAT, an important component of the overall CDR project, was being developed concurrent with the CDR solicitation process. The FDIC promised to deliver the taxonomies and software, meeting the September and October 2002 functional requirements, to the winning bidder for the CDR contract at the same time that the IDM team was waiving XBAT functional requirements. However, the IDM team did not assess the impact that the waivers and changes to the functional requirements might have on the CDR project.
As late as October 2, 2002, FDIC’s responses to questions posed by the CDR bidders indicated that XBAT would be delivered by December 31, 2002. In early November 2002, the XBAT contractor informed the TM and CO that more funding would be needed to complete all requirements of the original XBAT contract, but the “core functionality” could be delivered for the original contract amount. The changes to the FRD requirements were not factored into the CDR solicitation activities occurring during the period August 2002 through January 2003. Figure 2, on page 21, illustrates the timing of activities that parallel the XBAT development and the CDR solicitation efforts.
The XBAT was a relatively minor procurement effort but was also a critical path item to the much larger CDR effort. In September 2002, FDIC established the Capital Investment Review Committee (CIRC) to implement a systematic management review process for FDIC capital investments, defined as initiatives with total capital outlays in excess of $3 million. The CIRC currently monitors 14 initiatives, 1 of which is the CDR project. The XBAT is not a CIRC initiative because its development and maintenance costs are well below the $3 million threshold. However, the FDIC’s capital investment management review process should include CIRC initiatives as well as related or interdependent contracts and initiatives.
Further, the IDM team did not respond proactively to indications that XBAT development was experiencing problems and did not amend the CDR solicitation or notify CDR bidders that they would be receiving the XBAT software “as is” as opposed to receiving a fully functioning software product. The IDM project manager told us that he empowered the TM and placed too much reliance on his staff to ensure that the XBAT was progressing as planned and scheduled. In particular, the project manager relied too heavily on the TM to do the right thing and did not establish control points to independently verify that XBAT was on schedule, within cost estimates, and meeting performance objectives. The IDM project manager indicated that in hindsight, the FDIC should not have promised to deliver to CDR bidders an XBAT software meeting the September and October 2002 functional requirements. Instead, the FDIC should have offered XBAT “as is” without specific representations and warranties.
Consequence
The FDIC promised a software product containing functionality that it could not deliver and had to issue an $840,000 change order to the CDR contractor to replace XBAT.
Lessons Learned
-
Closely coordinate projects that are interdependent and promise only what can be realistically delivered.
- A change management process is needed to ensure that modifications are documented, communicated, and agreed upon.
- In situations where contract requirements are waived, the waived requirements should be clearly cross-referenced to the original contract requirements.
- In situations where FDIC determines that it is appropriate to split projects among multiple contracts, it is important that these projects are closely coordinated and that contracting documents specifically identify how deliverables from the separate contracts will be integrated.
- Bidders should be made aware of any modifications to solicitation documents, including requirements or milestone changes of related contracts or projects.
- Contracts should not generally be dependent on contract deliverables that have not been fully tested and accepted.
- When an initiative requires Capital Investment Review Committee review and monitoring, the Committee should monitor all contracts related to the initiative.
- In situations where FDIC property, such as software, is furnished to a contractor, strong consideration should be given to providing the property “as is,” rather than “suitable for use,” especially for property being produced through another contract or research and development effort.
![Project Integration Timeline. Select [D] link below image to view a text representation of the image.](fig2-04-014.gif)
[D]
Communications
Communication within and between the IDM team, senior-level managers, and the acquisition team was not effective. Accordingly, DIR executives and CDR bidders were not aware of the true status of the XBAT development effort. |
What Should Have Happened
The PMBOK® Guide identifies project communications management as a subset of project management that includes the processes required to ensure timely and appropriate generation, collection, dissemination, storage, and disposition of project information. It provides the critical links among people, ideas, and information that are necessary for success. Everyone involved in the project must be prepared to send and receive communications and must understand how the communications in which they are involved as individuals affect the process as a whole. Major processes of project communications management consist of:
- Communications Planning – determining the information and communications needs of the stakeholders, the individuals who need the information, when they will need it, and how it will be given to them.
- Information Distribution – making needed information available to project stakeholders in a timely manner.
- Performance Reporting – collecting and disseminating performance information, including status reporting, progress measurement, and forecasting.
- Administrative Closure – generating, gathering, and disseminating information to formalize phase or project completion.
The APM requires that the acquisition team maintain open communications at all times during contracting efforts.
What Actually Happened
Communications surrounding the XBAT engagement were not effective on several levels:
Within the IDM team. The XBAT TM waived contract requirements at the same time that the CDR Technical Evaluation Panel (TEP) Chairman was issuing solicitation documents and responding to bidder questions and answers about XBAT’s functionality. The TEP Chairman’s files included e-mail documentation addressed to the IDM project manager, CDR TEP Chairman, and XBAT TM from another IDM team member. The e-mail message discussed the need to notify CDR bidders of changes in XBAT functionality to avoid the potential of CDR contract default on the part of the FDIC. However, the IDM team did not act on this warning.
Further, it became necessary for the XBAT TM and the IDM project manager to recuse themselves from the CDR solicitation due to potential conflicts of interest. The TM’s recusal was necessary because he was working to secure post-retirement employment with a subcontractor to the CDR winning bidder. The TM coordinated his post-employment efforts with the FDIC’s Ethics Office. We verified that the TM did not violate post employment ethics restrictions. However, because he was arranging for employment with a subcontractor to the CDR winning bidder, the TM was precluded from discussing or being involved in the CDR solicitation and contract award effort.
The IDM project manager also recused himself from participating in the CDR solicitation because he owned stock in a company that was bidding for the CDR contract. The project manager stated this recusal impaired his ability to communicate with the CDR TEP Chairman. The project manager told us that he announced his recusal to the rest of the IDM team. However, the project manager did not discuss this issue with the FDIC Ethics Office or communicate his recusal to DIR executive managers or anyone outside of the IDM team.
Text Box: During a January 2003 CIRC meeting, the IDM project manager inappropriately announced the identity of the CDR winning bidder and subcontractor. The CIRC is to judge proposed capital investments on the merits of the project, not based on the reputation of the vendors implementing the project. Two CIRC members, the Deputy to the Chairman and the DIR Director immediately recused themselves from further participation in the CDR CIRC process because they owned stock in the CDR subcontractor—the same company from which the project manager had recused himself.
|
We verified the amount of stock that the project manager owned during the CDR solicitation and confirmed with the FDIC’s Ethics Office that the project manager was not required to (1) recuse himself from the CDR solicitation effort or (2) communicate his recusal to anyone. While the project manager was technically not required to announce his recusal, given his role as project manager for a $40 million dollar contracting effort, it would have been: (1) prudent for the project manager to discuss the stock ownership issue with the Ethics Office and (2) appropriate to communicate the recusal to executive DIR management in the spirit of full disclosure. Doing the latter may have resulted in DIR executive managers reorganizing the IDM team to achieve a more open communication environment.
The IDM Team and Senior DIR and FDIC Management. The XBAT and CDR status reports did not adequately communicate the timing and resource problems that the IDM team was encountering with XBAT or the potential impact that XBAT problems could have on the CDR project. For example, the October 2002 IDM status report included a discussion that the XBAT contract was being modified to a firm fixed price contract for delivery of the Call Report taxonomies and the XBAT tool by December 20, 2002. However, the status report did not communicate that the XBAT FRD had been changed and that the XBAT contractor would need additional funding to complete all functional requirements by December 2002. This information was critical at the time because the XBAT taxonomies and software had been promised as deliverables to the winning bidder on the CDR contract.
Senior DIR and FDIC management told us they relied on the executive and management-level acquisition team to oversee the less expensive XBAT contract. There was no evidence that these senior managers were actively involved in validating the information presented in status reports especially at key decision points such as CDR solicitation and contract award. An executive DIR manager told us that during status meetings with the IDM project manager, she asked questions about the status of the XBAT contract and any issues that could impact the CDR contract, but received information only that the XBAT project was running smoothly and on track. Further, we saw no e-mailed correspondence communicating questions or concerns about the XBAT contract above the IDM project manager level.
This level of executive oversight may have been reasonable, given the relatively low-dollar value of this contract and the high-level members of the contract management team assigned to administer the contract. Because it was an integral part of the larger CDR contract, the XBAT contract needed to have effective control mechanisms in place that would have given executive DIR management the true status of the XBAT contracting and development effort. The contract administration process, oversight management function, and the SDLC are critical control activities that are designed to provide executive managers a comfort level about a project’s success. While each of these control activities was in place, these processes or functions were ultimately not completely effective.
Members of the XBAT Acquisition Team. We saw numerous e-mails
evidencing inadequate communication among the CO, OM, and TM. For example,
on October 18,
2002, the OM informed DIRM management of problems being encountered on the
XBAT engagement, including the OM not receiving (1) a signed copy of the
contract, (2) a Letter of Oversight Manager Confirmation, or (3) the SOW. As
another
example,
the TM sent a message to DIRM, the OM, and the IDM project manager on August
9, 2002, communicating technical management and oversight management responsibilities.
This message was not copied to the CO or anyone in ASB.
Consequence
CDR bidders were not informed about the status of the XBAT product and the change in functional requirements. Potential ethics issues were not communicated to senior DIR managers. Communication within the IDM team and acquisition team and between the IDM team and senior DIR managers was negatively affected.
Lessons Learned
- Continually evaluate communication channels to ensure they are open and effective and provide balanced and complete information about a project’s status and viability at key development stages.
- For information technology engagements, there should be a clear line of responsibility and communication between DIRM and the requesting program office. For large contracts, it may be appropriate to document lines of responsibility and expectations for communication in writing.
- When members of the project or contracting team encounter personal or ethical conflicts that impact communication, the team should develop other means of effectively communicating project status and project issues or consider changes to the teams.
System Development Life Cycle
The XBAT development did not fully comply with the FDIC’s System
Development Life Cycle. Most importantly, the FDIC paid the XBAT contractor
before completely testing XBAT. Consequently, the FDIC had limited financial
leverage to compel performance by the XBAT contractor. |
What Should Have Happened
FDIC Circular 1320.3, System Development Life Cycle (SDLC) Version 3.0, dated July 17, 1997, defines a uniform process for developing, maintaining, and enhancing all FDIC automated information systems, and establishes the SDLC Version 3.0 as the standard methodology for system development at the FDIC. The objective of the SDLC is to employ a standard set of practices to ensure the delivery of quality systems that meet user needs in a timely, cost-effective manner.
The SDLC provides a structured framework and organizes roles, responsibilities, and development activities by clearly defined stages and phases. The phases of the SDLC are delineated by formal end products, quality reviews, and milestone concurrence and approval. Circular 1320.3 requires compliance with the SDLC process described in the Circular by all developers and corporate users involved in planning for and the development, acquisition, and maintenance of application systems, software, and hardware that support (1) a business function of the FDIC, (2) one or more offices within the FDIC, or (3) the exchange of data.
Text Box: The SDLC is divided into the following eight interdependent phases:
- Planning
- Requirements Definition
- External Design
- Internal Design
- Development
- Testing
- Implementation
- Maintenance
|
DIRM has overall responsibility for the management and development of corporate information technology. The program manager in information technology projects is the principal user representative from the FDIC division or office requesting or relying upon the automated information system. As the sponsor of the system, the program manager is the final user approval authority for the system. As such, the program manager is accountable and responsible for defining the required performance and ensuring that the stakeholders in the user community are involved in each phase of activities and that the developed application is addressing user needs.
What Actually Happened
The XBAT development did not fully comply with the FDIC’s SDLC provisions outlined in Circular 1320.3. For example, DIRM and the IDM team did not complete the SDLC test phase before paying the XBAT contractor. The test phase provides that the user and other designated testers validate that the functional requirements defined in the FRD are satisfied by the developed system and that there are no adverse effects on the overall process or other existing systems.
The SDLC requirements were included in the request for proposal for the XBAT
solicitation, the BOA executed for the XBAT contract, and the September 2002 FRD
containing functional requirements for the XBAT. These documents specifically
referenced FDIC Circular 1320.3. The XBAT SOW included the provision that, in all cases, the principles, phases, and deliverables of the FDIC SDLC as described in Circular 1320.3 must be applied to the XBAT application.
However, the SDLC provisions were removed from the revised FRDs prepared for XBAT in October 2002. Further, the SDLC provisions were not included in the task order awarded for the XBAT engagement.
Text Box: The September 2002 FRD included the following requirements:
Section 4.5, Testing and Traceability: A process must be in place to trace requirements and test all aspects of the system. Specifically,
- For testing of the developed system (or for future enhancements), test
data must be provided to enable a thorough test of the system to be completed.
- Modules
for each function must be separate so that testing of one process can be
made independent of another process.
- Changes and additions to the requirements must be added to the Traceability
Matrix for reference during future phases.
These requirements were eliminated
from the October 2002 and June 2003 XBAT FRDs.
|
In assessing the XBAT product delivered by the FDIC, the CDR contractor determined that XBAT met the SDLC requirements for the external design and internal design phases. However, the CDR contractor’s assessment indicated that XBAT did not meet the other SDLC requirements. One of the OMs told us that the TM, in consultation with ASB contracting representatives, agreed to a partial waiver of SDLC requirements, including the requirement for testing the system.
The XBAT contractor told us that XBAT user acceptance testing was the responsibility
of the FDIC. Under the SDLC, user acceptance testing is the final test of the
system being developed. The XBAT task order included the following provision
related to user acceptance testing:
The FDIC will develop test scripts for and sponsor the User Acceptance Testing
[UAT]. The contractor will assist in UAT which includes fixing bugs or system
errors uncovered during this process. This will be completed no later than
March 31, 2003.
The FDIC’s Systems Development Life Cycle Manual Version 3.0 requires information
technology development projects to include a test plan to ensure that all aspects
of the system are adequately tested and that the system can be successfully implemented.
The test plan is begun in the requirements definition phase and refined throughout
the design and development phases. Table 6 presents the various types of tests
that should be conducted during system development, the party responsible for
performing the test, and whether the test was performed under the XBAT development
effort.
Table 6: SDLC Testing Requirements
Type of Test
|
Description |
Performed By |
Performed under XBAT? |
Unit/Module Test |
Validates module's logic and adherence to FRD and technical
specifications. |
Individual developing the code. |
80 percent completed; requirement waived in Task Order
01. |
Integration Test |
Involves the subsystems that are made up of integrated
groupings of software units and modules. |
Developing organization |
Completed in June 2003. |
System Qualification Test |
Independent test that determines whether the system complies
with standards and whether it satisfies functional and technical requirements
when executed on target hardware using operational data files. Software
performance, response time, ability to operate under stressed conditions,
interfaces to other applications, security, and integrity control functions
are tested. |
DIRM |
90 percent completed in June 2003. |
User Acceptance Test |
Performed by the user in a nonproduction environment
which mirrors the environment in which the system will be used. System
interoperability, all documentation, system reliability, and the level
to which the system meets the user requirements will be evaluated. Performance,
recovery, and restart, and data quality testing may be performed. |
Client who will use the system. |
90 percent completed in June 2003. |
Source: FDIC SDLC Manual and IDM information.
The XBAT contractor submitted three invoices during 2002 under the XBAT contract, as shown in Table 7. The FDIC held each invoice for several months and ultimately paid the invoices in February and March 2003.
Table 7: XBAT Invoices and Payment Dates
Invoice |
Invoice Amount |
Invoice Date |
Check Date |
Purpose |
1 |
$75,690 |
9/26/02 |
2/10/03 |
Progress bill for professional services to support the
development of (1) XBRL taxonomies, (2) XBAT, and (3) XBRL demonstration
application. |
2 |
$299,592 |
11/04/02 |
2/05/03 |
Progress bill for work performed 9/1/02 - 10/31/02 in
support of the development of XBRL taxonomies and XBAT. Invoice stated
that 55 percent of the work related to Task Order 01 had been completed. |
3 |
$328,048 |
12/31/02 |
3/06/03 |
Progress bill for work performed 11/1/02 - 12/31/02 in
supporter of the development of XBRL taxonomies and XBAT. Invoice stated
that the December 20th delivery of work under Task Order 01 had been
completed. |
Source: Contracting Files
It appears that the FDIC paid the third invoice before the FDIC had completed user acceptance testing and determined that XBAT was fully functioning. For example, following FDIC payment of the third invoice, an XBAT status report, dated April of 2003, notes that 78 issues are outstanding, 72 of which pertain to user acceptance testing. The XBAT contractor noted that of the 78 issues remaining, 18 (23 percent) were critical (1 issue) or high priority (17 issues).
Consequence
Because the FDIC did not fully test XBAT before paying the XBAT contractor, FDIC had limited financial leverage to hold the XBAT contractor responsible for fulfilling contract requirements satisfactorily.
Lessons Learned
- System development contracting efforts should follow the FDIC’s System Development Life Cycle requirements contained in FDIC Circular 1320.3 or an equivalent.
- Contractors should be paid for their services only after deliverables are tested and accepted.
CORPORATION COMMENTS AND OIG EVALUATION
Our draft report contained no recommendations, and a written corporate response was not required. However, we solicited comments from DIR, DIRM, and the Division of Administration (DOA) on the lessons learned presented in the report and the discussion of the XBAT contracting and project management issues. DIR did not issue formal comments, but met with us to clarify certain aspects of the report. We subsequently made minor editorial changes to the report to address DIR’s comments. DIRM did not issue formal comments, but shared the report with DIRM managers responsible for contract administration and project management activities for their consideration on future projects. The Associate Director, ASB, provided DOA’s written response, dated March 18, 2004. DOA agreed that although the FDIC received value from the XBAT procurement, the project’s success was impaired by certain contracting and project management issues. DOA’s response summarizes some contracting efforts ASB has in process or expects to undertake in 2004 to enhance the overall controls governing the contracting process and to minimize the issues identified in our report for future system development efforts. DOA’s response is presented in its entirety in Appendix V to this report.
APPENDIX I: Objective, Scope, and Methodology
The objective of this evaluation was to assess the adequacy of the FDIC’s contract administration and project management for the development of the XBAT. We conducted this review in response to a request from FDIC senior management that the OIG and the Office of Internal Control Management evaluate the XBAT contracting and development efforts. The requestors expressed concerns that the XBAT product delivered for the CDR contractor was not fully functional and asked that we identify lessons learned in contract administration and project management of the XBAT procurement that could be applied to the larger CDR effort.
Scope
To accomplish our objective:
- We evaluated the effectiveness of the implementation of specific management controls related to the contract administration and project management for the development of the XBAT. These controls included the policies and procedures contained in the FDIC Acquisition Policy Manual as they apply to the XBAT procurement and oversight controls such as status reporting of the XBAT development project.
- We reviewed the XBAT procurement activities from fund authorization in May 2002 through contract award in December 2002. Because the XBAT contract is still in effect, we updated information through December 2003 for our report.
- We reviewed the FDIC Acquisition Policy Manual – Revision 1, effective March 31, 2000, for provisions that would apply to the XBAT procurement, which was awarded subsequent to the March 31, 2000 date and prior to the July 2003 subsequent update to the FDIC Acquisition Policy Manual.
- We conducted our evaluation work at the headquarters offices of DIRM, DIR, and DOA in Washington, D.C., during the period October through December 2003.
Methodology
We performed the following activities for our evaluation:
- We interviewed headquarters DIRM, DIR, and DOA officials who were responsible for contract administration and project management for the development of XBAT.
- We interviewed representatives of the XBAT contractor.
- We interviewed an official in FDIC’s Ethics Office.
- We reviewed key documents related to contract administration and project management of the XBAT development project, including:
o FDIC Acquisition Policy Manual.
o Oversight Manager Files.
o Technical Monitor Files.
o XBAT Pre-Award Files.
o XBAT Contract Files.
o IDM XBAT Files.
o IDM Monthly Project Status Reports on XBAT and CDR Projects.
o DIR Monthly Status Reports.
o DIRM Status Report to the FDIC’s Chief Operating Officer.
o XBAT Contractor Status Reports.
o Minutes of Call Modernization Project Steering Committee Meetings.
o Electronic Messages provided by two Technical Monitors, two Oversight Managers,
the CDR TEP Chairperson, IDM Team members, and the IDM Project Manager.
o Electronic Messages included in the XBAT Pre-Award and Contract Files.
o Bidders’ Questions and FDIC’s Answers for the XBAT Solicitation.
o Bidders’ Questions and FDIC’s Answers for the CDR Solicitation.
o XBAT Functional Requirements Documents, dated September 2002; October 2, 2002; October 16, 2002; and June 2003.
o Various Versions of the CDR Contractor’s Assessment of XBAT.
o CDR Change Order 01, executed October 25, 2003.
o FDIC Training Server Information for Oversight Manager Training.
We corroborated automated information used to support our evaluation results and lessons learned with other sources to ensure the information was sufficiently reliable. For example, we discussed information contained in project status reports and electronic messages with key personnel involved in the XBAT procurement activities.
Our work to address the Government Performance and Results Act included reviewing the FDIC’s 2003 Annual Performance Plan, in particular, the annual performance goal to maintain sufficient and reliable information on insured depository institutions. Embedded in the annual performance goal is a target to develop a more efficient approach to bank data collection and management. The CDR project is designed to modify and improve data management processes to more effectively collect, manage, and share information about insured institutions. The XBAT is an integral part of the CDR.
In addition, we reviewed the FDIC 2003 Agenda: Corporate Performance Objectives, Third Quarter Summary Report. In the Strategic Objective, Stability, the FDIC has established a performance objective to make high-quality banking data available to the public on a more timely basis. As part of the CDR project, the FDIC, FRB, and OCC are actively working with the financial institutions and trade groups on strategies to improve data quality and timeliness when the new CDR is implemented.
We did not develop specific evaluation procedures to detect abuse and illegal acts because we did not consider abuse and illegal acts to be material to the evaluation objective. However, throughout our evaluation, we were sensitive to the potential of illegal acts, including fraud, waste, abuse, and mismanagement.
We did not assess the FDIC’s compliance with applicable laws and regulations because we did not identify specific laws or regulations pertaining to the implementation of contract administration, or project management controls.
We conducted our evaluation in accordance with generally accepted government auditing standards.
Project Management Criteria
Although the FDIC is not required to comply with the Project Management Body of Knowledge PMBOK® Guide, we used it as the primary criteria for assessing FDIC’s project management controls for the XBAT procurement, especially in the areas of change control management and project integration. This guide contains generally accepted industry practices related to successful project management. Generally accepted means that the knowledge and practices described are applicable to most projects most of the time and that there is widespread consensus about their usefulness. Generally accepted does not mean that the knowledge and practices described are to be applied uniformly on all projects; the project management team is always responsible for determining what is appropriate for any given project.
APPENDIX II: XBAT Contracting and Project Management Challenges
What Actually Happened |
What Should Have Happened |
Consequences |
Lessons Learned |
FDIC did not establish XBAT functional requirements early enough in the
contracting process. |
In initiating procurement actions, requirements should be defined to
provide a clear and specific description of the goods or services required.
APM 4.A.4.a.(3) |
Because functional requirements were evolving during the XBAT development,
the FDIC and XBAT contractor did not have a clear and consistent understanding
of what functionality XBAT should include. |
1. Functional requirements should be established early in the procurement
process.
2. When identifying procurement requirements, provide a clear
and specific description of the goods or services required. |
FDIC did not adequately plan or execute the XBAT procurement effort.
The IDM team waived important XBAT functions that were intended to support
the overall CDR effort to meet cost and time constraints. Further, the
XBAT procurement did not consistently follow the APM. |
A procurement requirements package should include a schedule for delivery
of goods or performance of services and a price estimate, including the
base period and all option periods. (APM 4.A.4.a. (3) and (5))
A BOA is
a written agreement between the FDIC and a contractor, containing terms
and conditions that will apply to FDIC-issued task orders during
the term of the agreement. (APM 3.B.3.b)
A System Development Life Cycle contract with task assignments can
be used in DIRM contracts that incorporate a system development life
cycle documenting the
steps to be taken by the contractor. (APM 3.B.9.a.) |
Because the XBAT procurement was not adequately planned
or executed, the quality of the XBAT software suffered. Further, because
the procurement
did not consistently follow acquisition policies and procedures, the FDIC
was party to a contract that did not protect the Corporation’s best interests. |
1. Changes to system requirements should not be driven
solely by resource and schedule constraints – consideration must also
be given to impact on business needs.
2. Research and development efforts do not lend themselves to a firm fixed price
contracting vehicle, given the uncertain and evolving nature of the services.
The type of contract awarded should be based on the level of risk to the Corporation,
with fixed price contracts generally being used when risk has been reduced to
a reasonable level.
3. When procuring services for research and development, communicate to everyone
involved that the project is a research and development effort, requiring closer
monitoring.
4. A time and materials level of effort contracting vehicle may be more suitable
for a research and development effort, given the difficulty in providing a detailed
statement of work or to estimate the price or duration of time required for a
research and development effort.
5. Involve the FDIC Legal Division early in procurement planning and subject
key contracting documents such as the contract and statement of work to legal
review and concurrence prior to contract execution. Focus the legal review on
both legal sufficiency and protecting the Corporation’s interest, particularly
for contract provisions that pose the greatest risk for the Corporation. |
The CO and OM did not completely fulfill their responsibilities and relied
on the TM for making changes to the functional requirements, accepting
contract deliverables, and approving a key invoice payment. |
The OM is responsible for overseeing the performance requirements of
the contract, provides business and technical liaison between the FDIC
and the contractor, reviews and approves invoices for payments, verifies
satisfactory delivery of contract items, and notifies the CO of any need
to change the contract. (APM 7.B.1.h. and i.)
The CO is responsible for ensuring compliance with the contract. (APM 7.B.2.a.)
The duties of a TM are a subset of the duties of the OM, but the responsibility
for contractor oversight remains with the OM. (APM 7.B.1.d.) |
Because the CO and OM did not completely fulfill their
responsibilities and relied too heavily on the TM for managing the contract,
the XBAT procurement
did not have adequate checks and balances to help ensure the project’s
success. |
1. The CO, OM, and TM should work together as a team in meeting their
assigned responsibilities as part of establishing a control framework.
2. The OM should involve the CO in addressing contractor performance matters.
3. OM and TM roles and responsibilities need to be clearly documented and communicated
in the Letter of OM and TM Confirmation.
4. A CAP is a good tool for helping ensure that the contractor performs in accordance
with the terms and conditions of the contract and is already required by the
APM.
5. A CAP allows the OM and CO to identify specific deliverables and due dates
and track modifications and funding. |
FDIC did not exercise adequate change management or project integration
during the XBAT development effort and the CDR solicitation process. The
CDR solicitation promised a fully functioning XBAT at the same time that
the IDM team was waiving important functional requirements to meet cost
constraints and product delivery milestones. |
Scope change control involves influencing the factors
that create scope changes to ensure that changes are agreed upon, determining
that change
has occurred, and managing the changes when they occur. Scope change control
must be integrated with other control processes, such as scheduling, cost,
quality, risk, and staffing. (PMBOK® Guide 5.5 and 4.3) |
The FDIC promised a software product containing functionality that it
could not deliver and had to issue an $840,000 change order to the CDR
contractor to replace XBAT. |
1. Closely coordinate projects that are interdependent, and promise only
what can be realistically delivered.
2. A change management process is needed to ensure that changes are documented,
communicated, and agreed upon.
3. In situations where contract requirements are waived, the waived requirements
should be clearly cross-referenced to the original contract requirements.
4. In situations where FDIC determines that it is appropriate to split projects
among multiple contracts, it is important that these projects are closely coordinated
and that contracting documents specifically identify how deliverables from the
separate contracts will be integrated.
5. Bidders should be made aware of any modifications to the solicitation documents,
including requirements or milestone changes of related contracts or projects.
6. Contracts should not generally be dependent on contract deliverables that
have not been fully tested and accepted.
7. For major projects involving multiple contracts, the CIRC should consider
all related contracts.
8. In situations where FDIC property, such as software, is furnished to a contractor,
strong consideration should be given to providing the property “as is,” rather
than “suitable for use,” especially for property being produced through another
contract or research and development effort. |
Communication within and between the project team, senior-level managers,
and the acquisition team was not effective. |
Project communications management includes the processes
required to ensure timely and appropriate generation, collection, dissemination,
storage,
and disposition of project information. (PMBOK® Guide 10) |
CDR bidders were not informed about the status of XBAT and the change
in functional requirements. Potential ethics issues were not communicated
to senior DIR managers. Communication within the IDM team and acquisition
team and between the IDM team and DIR senior managers was negatively affected. |
1. Continually evaluate communication channels to ensure
they are open, effective, and provide balanced and complete information
about
a project’s status and viability at key development stages.
2. In information technology engagements, there should be a clear line of responsibility
and communication between DIRM and the requesting program office. For large contracts,
it may be appropriate to document lines of responsibility and expectations for
communication in writing.
3. In situations where members of the project or contracting team encounter personal
or ethical conflicts that impact communication, the team should develop other
means of effectively communicating project status and project issues or consider
changes to the teams. |
XBAT development did not fully comply with the FDIC’s
System Development Life Cycle. Most importantly, the FDIC paid the XBAT
contractor before
completely testing the XBAT. |
FDIC Circular 1320.3, System Development Life Cycle (SDLC) Version
3.0, included in the solicitation for XBAT and the Functional Requirements
Documents, defines a uniform process for developing, maintaining, and enhancing
all FDIC automated information systems, and establishes the FDIC System
Development Life Cycle Manual Version 3.0 as the standard methodology for
system development at the FDIC. |
Because FDIC did not fully test XBAT before paying the XBAT contractor,
FDIC had limited financial leverage to hold the XBAT contractor responsible
for fulfilling contract requirements. |
1. System development contracting efforts should follow
FDIC’s System
Development Life Cycle requirements contained in FDIC Circular 1320.3,
or an equivalent.
2. Contractors should be paid for their services only after deliverables are
tested and accepted. |
APPENDIX III: Key Events in the XBAT Project
Month |
Event |
December 2001 |
FDIC Chairman announces goal of providing accurate and
timely Call Report data to the public as soon as it is available. |
June 2002 |
XBAT request for proposal issued by FDIC |
July 2002 |
FDIC developing requirements for the XBAT tool. |
August 2002 |
FDIC issues advance authorization letter to XBAT contractor
while contracting documents are undergoing review. |
September 2002 |
FDIC and Contractor sign Basic Ordering Agreement for
XBAT. |
October 2002 |
FDIC OM and XBAT contractor sign Task Assignments 01 & 02 |
November 2002 |
FDIC continues working on a Task Order |
December 2002 |
FDIC and XBAT contractor sign Task Order 01 effective
August 9, 2002. Work completed on version 1 of XBAT. |
January 2003 |
Best and Final Offer for the CDR Contract. XBAT promised
as a deliverable to the winning bidder on the CDR contract. XBAT undergoing
quality assurance testing. |
February 2003 |
XBAT Basic Ordering Agreement is modified to increase
funding by $30,000. |
March 2003 |
Call Report analysts participate in quality assurance
testing for the validation criteria component of XBAT. |
April 2003 |
Version 1.3 of XBAT undergoing quality assurance testing. |
May 2003 |
FDIC awards CDR contract. |
June 2003 |
XBAT given to CDR contractor for analysis of XBAT architecture
and outputs. |
July 2003 |
CDR contractor completing a detailed analysis of XBAT,
focusing on business and technical requirements. |
August 2003 |
CDR contractor conducts an analysis of the status of
the XBAT tool against the Functional Requirements Document posted as
part of the CDR Request for Proposal. |
September 2003 |
XBAT contractor continues to correct nine issues/problems
identified in the XBAT application. |
October 2003 |
CDR Contract Change Order 01 awarded for design and development
of a Meta-data Management Tool to replace XBAT. |
APPENDIX IV: XBAT Procurement Requirements
Procurement Requirement |
Request for Proposal Issued June 7, 2002 |
BOA signed September 25, 2002 |
Task Assignment 01 Signed October 17, 2002 |
Task Assignment 02 Signed October 17, 2002 |
Task Order 01 signed December 20, 2002, effective
August 9, 2002 |
Scope of Work |
Provide consulting support for the development of a
set of XBRL taxonomies and a business analyst tool for FFIEC Call Reports |
No change |
Analyze and present an optimal architectural solution
for the development of an XBRL Business Analyst Tool and complete development
of XBRL Call Report frameworks for period March 2001 through December
2002. |
Develop XBAT to create and maintain XBRL Call Report
frameworks |
By 12/20/02, develop XBRL Call Report frameworks for the reporting periods of 3/2001 and 3/2002, and analyze, design, and develop an application called XBAT.
By 6/30/04, provide technical input to FDIC for developing frameworks for 6/2001, 9/2001, 12/2001, 6/2002, 9/2002, 12/2002, and 3/2003. |
Function Requirement |
The FDIC will provide draft requirements for a FFIEC
XBRL business analyst tool. |
No change |
Contractor is required to deliver a written functional
requirements document for XBAT by September 6, 2002. |
The contractor will develop the XBAT to meet requirements
as described in the FRD with the final date of September 30, 2002. |
The XBAT will be developed to meet requirements in the
FRD with the final date of Octo ber 16, 2002. |
SDLC |
In all cases, the principles, phases, and deliverables
of the FDIC SDLC as described in the FDIC Circular 1320.3 must be applied. |
No change |
SDLC requirements were not specifically mentioned in
Task Order 01. |
SDLC requirements were not specifically in Task Order
02, but were referenced in the September 2002 FRD. |
SDLC requirements not specifically mentioned in Task
Order 01 or referenced in the FRD. |
Funding Provisions |
5/08/02 procurement requisition for $1.19 million covering 6.5 years.
--Development--$498,400
--Maintenance--$693,456
|
September 2002 BOA for $682,330 covering 4 years.
--
Development--$395,200
--Maintenance--$287,130.
|
N/A |
N/A |
Task Order covering 23 months for $712,330, of which
$703,330 was related to development. |
Period of Performance |
June 3, 2002 through December 31, 2008. |
August 9, 2002 through August 9, 2003 with three 1-year
options. |
August 13, 2002 through March 31, 2003 |
September 18, 2002 through March 31, 2003 |
August 9, 2002 through June 30, 2004 |
APPENDIX V
CORPORATION COMMENTS
![FDIC Federal Deposit Insurance Corporation](https://webarchive.library.unt.edu/eot2008/20090114034638im_/http://fdicoig.gov/images/fdicsmall.gif) Federal Deposit Insurance
Corporation Washington, DC
20429
Office of the Associate Director Division of Administration
March 18, 2004
TO: Stephen M. Beard, Deputy Assistant Inspector General, Office of
Inspector General (OIG)
FROM: Ann Bridges Steely [Electronically produced version;
original signed by Ann Bridges Steely], Associate Director, Division of
Administration, Acquisition Services Branch
SUBJECT: Management Response to OIG Draft Report Entitled: XBat Contracting and Project Management
(Assignment Number 2003-071)
The Division of Administration (DOA) has completed its review of the subject Office of Inspector General (OIG) report. We agree with the OIG that the FDIC did receive value from the XBAT procurement. Although value was received, the report also concluded that the project success was impaired by certain contracting and project management issues. Overall, we agree with your assessment. Since the report contained no recommendations to DOA and no formal written comments were solicited or required, we would like to summarize some of the contracting efforts the DOA Acquisition Services Branch (ASB) has in process or expects to undertake this year. The intent of these efforts is to enhance the overall controls governing the contracting process and minimize those issues identified by the OIG on future systems development efforts.
ASB has begun a process to revise, update, or create policy to improve contracting processes and ensure a greater recognition of the value received on FDIC dollars spent. This will be an evolutionary process, but hopefully some immediate positive impacts will be recognized through our efforts. In general, the report touches upon areas ASB has deemed ripe for change. In particular, ASB has set out to revise the current APM to strengthen policy, streamline processes, and ensure a greater degree of usability for both the contracting and project management communities. Through various forms of training, coupled with greater emphasis on management oversight, we believe that increased proficiency and overall quality of the contracting actions and decisions will follow.
The key lessons learned throughout the OIG’s report provide a good basis for identifying and planning areas for emphasis to change or train. One of those areas will be in Project Management. ASB is leading an interdivisional effort to devise a Project Management (PM) framework that will provide for development of organizational policies and procedures, governance, training, and levels of proficiency to ensure adequate planning and management of FDIC projects occur. Coupled with this PM initiative is the development of an Acquisition Planning (AP) policy and procedure. This AP structure will provide the framework for building stronger acquisition teams to ensure that Contracting Officers and Program Managers work together to meet their assigned responsibilities through sound planning of the requirement from pre-award through contract closeout. AP will ensure, to the greatest extent possible, a better definition of program requirements early on in the procurement process. The definition of requirements will consider the risks associated with the different strategies for procuring the goods or services for the Corporation, and greater emphasis on certainty regarding program success.
Along with an AP policy and procedure, will be the implementation of a “business review process” to include deliberate checkpoints in the acquisition process to ensure sound strategies have been designed, as well as ensure the Corporation will receive the overall best value in contractor performance. This business review process will institute contract policy reviews to ensure proper selection of contract types are commensurate with the degree of risk and performance incentives needed to ensure successful performance. Additionally, greater collaboration will be sought with the FDIC contract legal staff both before issuance of a Request for Proposal, as well as prior to contract award, through mandated contract dollar review levels. These review levels will ensure that proper legal insight to our contracting actions minimize risk where possible and ensure more than just legal sufficiency, but a greater emphasis on sound business decisions.
Finally, ASB is exploring alternative methods of contracting for the FDIC goods and services through the development of statements of objectives and incentive type contract arrangements. We recognize there are contract type arrangements that could ensure adequate controls for monitoring performance, cost and schedule and we want to utilize those contract tools .
Should you have any questions regarding the response, my ASB point of contact is Michael Benavides, Chief of Contract Policy, (202)942-3557. The DOA audit liaison is Andrew Nickle, Audit Liaison for DOA. Mr. Nickle can be reached at (202) 942-3190.
cc: Arleas Upton Kea
Glen Bjorklund
Paul Sherman
Rodney Cartwright
Elroy Holden
|