B-400135; B-400135.2, Lockheed Martin MS2 Tactical Systems, August 8, 2008
DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.
Decision
Matter of: Lockheed Martin MS2 Tactical Systems
Marcia G. Madsen, Esq., Cameron S. Hamrick, Esq., David A. Dowd, Esq., Roger D. Waldron, Esq., Luke Levasseur, Esq., Melissa L. Baker, Esq., and Sarah A. Sulkowski, Esq., Mayer Brown; and Bucky P. Mansuy, Esq., Lockheed Martin MS2 Tactical Systems, for the protester.
John
W. Chierichella, Esq., Anne B. Perry, Esq., Jonathan S. Aronie, Esq., Keith R.
Szeliga, Esq., Jesse J. Williams, Esq., and Daniel J. Marcinak, Esq., Sheppard
Mullin, for Northrop Grumman, the intervenor.
Bridget
A. Jarvis, Esq., Christopher J. Biglin, Esq., Russell P. Spindler, Esq.,
Gregory Ircink, Esq., Stephanie Kroke, Esq., and Kristopher Fischer, Esq.,
Naval Air Systems Command, for the agency.
David A. Ashen, Esq., and John M. Melody, Esq., Office of the General Counsel, GAO, participated in the preparation of the decision.
DIGEST
Agency reasonably determined, in procurement for unmanned maritime surveillance aircraft, that awardee had significant advantage over protester with respect to past performance where: protester’s subcontractor, responsible for approximately 50 percent of contract effort, had recent past performance history of being unable to resolve staffing and resource issues, resulting in adverse cost and schedule performance on very relevant contracts for unmanned aircraft; record did not demonstrate that protester’s subcontractor had implemented systemic improvement that resulted in improved performance; while operating division of the awardee also had performance problems on very relevant contracts for unmanned aircraft, many had been addressed through systemic improvement; and overall performance of awardee’s team on most evaluated contract efforts was rated better than satisfactory, while the overall performance of protester’s team on 11 of 26 contract efforts was only marginal.
DECISION
Lockheed Martin MS2 Tactical Systems (LM) protests the Naval Air Systems Command’s (NAVAIR) award of a contract to Northrop Grumman (NG) under request for proposals (RFP) No. N00019-07-R-0001, for the Broad Area Maritime Surveillance (BAMS) Unmanned Aircraft System(UAS). LM challenges the evaluation of proposals and resulting source selection.
The Navy generally expects that the BAMS UAS will provide a persistent maritime Intelligence, Surveillance, and Reconnaissance (ISR) data collection and dissemination capability to the fleet, with BAMS on station 24 hours a day, 365 days a year for 20 years. Hearing Transcript (Tr.) at 47. The Statement of Objectives for BAMS (SOO) included in the RFP established the objective that each BAMS orbit (unit) provide (with no more than three aircraft aloft simultaneously) continuous surveillance capability for a minimum of 24 hours per day for 7 consecutive days, with an Effective Time on Station (ETOS) of no less than 80 percent, at a minimum mission range of 2,000 nautical miles (nm). BAMS SOO sections 1.0-3.1. Likewise, the Performance Based System Specification for BAMS (PBSS) included in the RFP required that the BAMS UAS “be capable of maintaining 80 percent (Threshold) and 95 percent (Objective) ETOS executed within a period of 168 continuous hours at a mission radius of 2000 Nautical Mile (nm) from its operating base.” PBSS sect. 3.1.1.
At Initial Operational Capability (IOC), defined as one base unit with sufficient assets to operationally support one persistent ISR orbit, the BAMS UAS missions will include maritime surveillance, collection of enemy order of battle information, battle damage assessment, port surveillance, communication relay, and support of maritime interdiction, surface warfare, battlespace management, and targeting for maritime and littoral strike missions. (At full operational capability, the BAMS UAS will provide for up to five simultaneous orbits worldwide.) While the objective is to achieve IOC in fiscal year (FY) 2113 or earlier, the minimum threshold requirement is for an IOC in FY 2014. SOO sections 1.0-2.0.
The solicitation provided for award of a
cost-plus-award-fee contract for system development and demonstration (SDD),
with cost-plus-award fee options for low rate initial production (LRIP 1), and
for furnishing the BAMS UAS to
Three offerors responded to the solicitation by the
closing time on
All three proposals were included in the competitive range. After conducting several rounds of written and oral discussions with the offerors, NAVAIR requested the submission of final proposal revisions (FPR). FPRs were evaluated as follows:
NG |
LM |
Boeing |
|
Technical (Overall) |
Highly Satisfactory/ Medium Risk |
Highly Satisfactory/ Medium Risk |
Highly Satisfactory/ Medium Risk |
Design Approach |
Outstanding/ Low Risk |
Outstanding/ Low Risk |
Outstanding/ Low Risk |
Program and Schedule |
Satisfactory/ Medium Risk |
Satisfactory/ Medium Risk |
Satisfactory/ Medium Risk |
Past Performance |
Moderate Risk |
High Risk |
Low Risk |
Experience |
Very Low Risk |
Very Low Risk |
Low Risk |
Cost |
|||
SDD/LRIP 1 |
$1.946 Billion |
$1.213 Billion |
$[REDACTED] |
O&S |
$12.382 Billion |
$12.495 Billion |
$[REDACTED] |
Production |
$6.603 Billion |
$2.064 Billion |
$[REDACTED] |
Australian Option |
$240 Million |
$112 Million |
$[REDACTED] |
Source Selection Advisory Council (SSAC) Proposal Analysis Report (PAR) at 3.
Although each proposal received the same adjectival ratings under the technical factor, NG’s (as well as Boeing’s) proposal was evaluated as having a strong advantage over LM’s. Specifically, NG’s proposal was evaluated by the SSAC as having a strong advantage over LM’s under the design approach subfactor of the technical factor. In this regard, the solicitation provided that evaluation under the design approach subfactor would encompass the extent to which the proposed BAMS UAS will be able to meet the mandatory as well as the offeror-specific tailored requirements of the BAMS PBSS, with emphasis on ETOS, open systems architecture, mission performance, due regard, and growth potential in the areas of unmanned aircraft (UA) Space, Weight, and Power (SWaP). NG’s proposed BAMS was evaluated as having a significant strength in the emphasis area of ETOS, offering a greater persistent ISR capability with an evaluated ETOS of 96.2 percent, significantly higher than LM’s evaluated ETOS of 84.6 percent. In addition, NG’s proposal was evaluated as having a significant strength in the emphasis area of SWaP, including a significantly higher weight margin (ability to add weight without breaching the ETOS threshold requirements) of approximately [REDACTED] pounds, nearly [REDACTED] times greater than LM’s margin of only approximately [REDACTED] pounds. Tr. at 122-49; Hearing Exhibit 102. NAVAIR noted that NG’s superior SWaP future growth capability could be used to incorporate future capability increments without breaching the ETOS threshold requirements, and that the resulting significantly greater design margin would reduce the risk of unknown design challenges requiring increased SWaP, particularly weight. In contrast, LM’s proposal was evaluated as having a significant weakness under the design approach subfactor for failure to provide a validated computer model of the performance of its proposed engine. The agency determined that NG’s strong advantage under the design approach subfactor offset a slight advantage for LM under the program schedule subfactor, such that NG’s proposal had a significant overall advantage over LM’s under the technical factor. SSAC PAR at 3-7.
In addition, the past performance of LM and GA-ASI (its principal subcontractor, which was proposed to perform approximately 50 percent of the work), was determined to represent a high risk, giving rise to substantial doubt that the LM/GA‑ASI team could perform the proposed contract effort. In this regard, the SSAC noted reports of poor past performance on very relevant work, including prior GA-ASI contracts for Predator-related work, with practically no systemic improvement demonstrated. According to the evaluators, the customer feedback for GA-ASI’s contracts was remarkably consistent across contracts, identifying difficulties in managing workload, problems with executing systems engineering and systems integration tasks, and problems with properly staffing a project. Further, the SSAC expressed concern that the additional resources required for the BAMS UAS program would further exacerbate the identified existing staffing and management shortcomings. Finally, the SSAC noted that LM itself had experienced difficulties in furnishing adequate staffing on its contract (the Po Sheng contract) to upgrade the command control system for Taiwanese F-16 fighter aircraft. Based on this record, the SSAC expressed concern that the LM team would encounter significant schedule delays and be required to make technical trade-offs in order to produce the Mariner. PAR at 7-10.
In contrast, the NG division serving as the prime
contractor/system integrator and performing approximately [REDACTED] percent of
the NG contract effort (NG Integrated Systems, Eastern Region in
The SSAC concluded that NG had a significant advantage over LM/GA-ASI with respect to past performance. According to the SSAC, while NG had implemented systemic improvements that were successful in improving performance for nearly all problems found on relevant contracts, nearly all LM/GA-ASI problems appeared to persist without implemented systemic improvement that resulted in improved performance. Moreover, the SSAC expressed particular concern that the poor past performance of the LM team was focused on GA-ASI, which was proposed to perform approximately 50 percent of the BAMS work, leaving a critical gap in LM’s capability to reliably perform the BAMS effort. The SSAC (with one dissenter) concluded that NG’s proposal’s strong advantage over LM’s proposal under the technical factor (the most important evaluation factor), and NG’s significant advantage under the past performance factor (the second most important factor), offset LM’s significant advantage with respect to cost. PAR at 7-11, 13-16.
The Source Selection Authority (SSA) concurred with the SSAC’s recommendation that NG’s proposal was most advantageous. According to the SSA,
[LM’s] past performance record identifies systemic problems at GA-ASI, which coupled with the lack of demonstrated systemic improvement, indicate that these problems will likely be repeated on the BAMS UAS program negatively impacting a significant portion of the work. Furthermore, while [NG’s] advantage in Past Performance in itself justifies the [cost] premium, the technical advantage of providing a large design margin further increases [NG’s] probability for success, mitigating some of the doubt associated with their Moderate Past Performance risk. . . . [NG’s] design approach has significant Space, Weight and Power (SWaP) growth capability, which may be used to incorporate future increments without breaching the Effective Time on Station (ETOS) threshold requirement. This significant strength can also be used as a design margin which acts to reduce the risk of unknown design challenges requiring increased SWaP, particularly weight. Finally, it is important to note that [LM’s] lower proposal evaluated costs do not include any costs associated with poor performance. Performance difficulties can result in a significant cost to the Government in terms of time (schedule slips), money (cost over‑runs and internal Government manpower and resources), and technical capability for the warfighter.
Source Selection Decision (SSD) at 2-3. Upon learning of the resulting award to NG, and after being debriefed, LM filed this protest with our Office.
PAST PERFORMANCE
Background
LM principally challenges NAVAIR’s evaluation of its own and NG’s past performance. In this regard, the RFP provided that the government would
evaluate the offeror’s, and (if applicable), its principal subcontractors’ and critical team members’ demonstrated past performance in delivering quality products and in meeting technical, cost and schedule requirements on similar programs for SDD, Production, and Operations and Support. The currency and relevance of the information, source of the information, context of the data, and general trends in contractor’s performance will be considered. Problems not addressed by the offeror will be considered to still exist. However, consideration for discounting problems may be given when those problems are addressed through demonstrated systemic improvement.
RFP sect. M.II.B.
In furtherance of the past performance evaluation,
offerors were required to identify contracts “whose performance is within five
years from the RFP release and contain efforts similar to those efforts, e.g.,
tasks, contract type, location, contract dollar value, etc., required by this
solicitation.” RFP sect. L.3.1. The information provided “should be related
to similar programs in the same division, or cost centers in which the Offeror
proposes to perform this effort,” and correspond to the descriptions of the
offeror’s experience under the experience factor.
For each relevant contract, offerors were required to describe performance in meeting technical and quality requirements, meeting schedule requirements, controlling cost, and managing the contracted effort (e.g., program management, cooperation with customer, subcontract management). RFP sect. L.3.4.2. In addition, and of particular importance here, offerors were required, “[f]or each past performance problem identified, [to] describe the status of the systemic improvement efforts and, where applicable, demonstrate the impact that the systemic improvement effort had on resolving the problem such that it would not reoccur.” RFP sect. L.3.1. Further, in addition to “[i]dentify[ing] those systemic improvement actions taken to resolve past problems,” offerors were required to “[p]rovide the records of such results and indicate where they are documented, preferably in Government record systems. Describe the techniques, elements, and tools used to correct problems on the contract and, if applicable, how these techniques, elements, and tools will be used during this program.” RFP sect. L.3.4.3. Finally, offerors were cautioned that “[t]he Government does not assume the duty to search for data to cure the problems it finds in the information provided by the Offeror. The burden of providing thorough and complete past performance information remains with the Offeror.” RFP sect. L.3.1.
The record indicates that, in evaluating past performance, NAVAIR undertook an in‑depth, detailed examination of the recent relevant past performance of each team. In this regard, the agency reviewed 30 contracts for the LM team, 19 of which (including 6 LM and 7 GA-ASI contracts) were determined to be either somewhat relevant or very relevant; received 23 past performance questionnaires for the team; conducted 7 interviews; and reviewed 34 CPARS. The agency reviewed 27 contracts for the NG team, 24 of which (including 17 NG contracts) were determined to be either somewhat relevant or very relevant; received 40 past performance questionnaires for the team; conducted 13 interviews; and reviewed 45 CPARS. SSEB at 40. Based on the totality of the information available from the above information sources, NAVAIR determined that the performance of the LM team was exceptional on 1 contract effort, very good on 6, satisfactory on 8, and marginal on 11. (Some contracts were divided into separate delivery/task orders for purposes of these ratings When considered on a contract-by-contract basis, without division into separate task order(s) contract efforts, the LM team’s performance was exceptional on one contract, very good on six, satisfactory on seven, and marginal on five. However, of particular significance to evaluation, of the five very relevant GA-ASI contracts for Predator-based aircraft, GA‑ASI’s performance was marginal on three contracts.) In contrast, NAVAIR determined that the overall performance of the NG team was exceptional on 3 contract efforts, very good on 10, satisfactory on 9, and marginal on only 3 contract efforts. Source Selection Evaluation Board Report (SSEB) at 31-40.
LM asserts that NAVAIR’s evaluation of LM’s, GA-ASI’s, and NG’s past performance was inconsistent with the solicitation and otherwise unreasonable in numerous respects.
Where a solicitation requires the evaluation of offerors’
past performance, we will examine an agency’s evaluation only to ensure that it
was reasonable and consistent with the solicitation’s evaluation criteria and
procurement statutes and regulations; determining the relative merits of
offerors’ past performance information is primarily a matter within the
contracting agency’s discretion. The
MIL Corp., B‑297508, B‑297508.2, Jan. 26, 2006, 2006 CPD para. 34
at 10; Hanley Indus., Inc., B‑295318,
Past Performance of LM Team
Although the SSEB noted that inadequate staffing and a shortfall in technical skills had adversely impacted LM’s ability to execute a somewhat relevant (the Po Sheng) SDD contract to upgrade the command control system for Taiwanese F-16 fighter aircraft, the panel generally acknowledged that LM had demonstrated “high quality technical performance” on five of six relevant contracts. SSEB at 52. In contrast, however, GA-ASI’s contract performance was a matter of great concern to the agency. Specifically, while recognizing that GA-ASI had demonstrated a willingness and ability to respond on short notice to evolving Global War on Terror (GWOT) warfighter requirements, the SSEB found that GA-ASI’s performance demonstrated: inadequate staffing, resulting in performance problems on SDD contracts for the MQ‑9 Reaper (a second-generation, Predator B model) and the MQ‑1C Extended Range/Multipurpose (ER/MP) UAS (a second-generation Predator model); unfavorable schedule performance on four of seven relevant GA-ASI contracts, including very relevant contracts for the MQ-9 Reaper, UAS ground control stations, MQ-1C ER/MP, I-GNAT Extended Range UAS (a version of the Predator with some differences for the Army), and MQ-1 baseline Predator; poor performance in meeting technical quality requirements on three of seven GA-ASI contracts, including contracts for the MQ-9 Reaper, MQ-1C ER/MP, and I‑GNAT Extended Range UAS; and workload exceeded the firm’s capacity on five of seven GA-ASI contracts, including contracts for the MQ-9 Reaper, UAS ground control stations, MQ-1C ER/MP, I-GNAT Extended Range UAS, and MQ‑1/MQ-9 maintenance support. In summary, the SSEB found the overall performance of GA-ASI on its very relevant contracts for the MQ-9 Reaper (most delivery orders), UAS ground control stations, MQ-1C ER/MP, and I-GNAT Extended Range UAS to be marginal. SSEB at 36-38, 52‑62.
Based upon the above past performance problems, the SSEB determined that there was substantial doubt that LM would successfully perform the required effort, and that an overall high risk rating therefore was warranted. According to the SSEB:
The Lockheed Martin team delivers a high quality, technical product and both Lockheed Martin and GA-ASI are motivated to meet the warfighter’s requirements. Lockheed Martin, as the prime contractor however, will be substantially challenged to ensure that GA-ASI will remain on schedule. The proposal includes a prominent role for GA‑ASI including aircraft design, UA manufacture, flight test, logistics, training support, communications subsystem and MCS aircraft control segment which represents approximately 50% of the proposed effort. There is, therefore, significant risk to the BAMS UAS program if GA‑ASI’s future performance trend reflects identified past performance difficulties in managing increasing workload, a possibility which the [Past Performance Evaluation Team] assesses as likely to occur.
Lockheed Martin and GA-ASI have recent past performance histories of being unable to resolve staffing issues resulting in adverse cost and schedule performance. Furthermore, there are documented concerns regarding the amount of work that GA‑ASI has taken on and the slow pace of implementing processes and process improvements that increased workloads and responsibilities require. Systemic improvement initiatives have been identified or are in work in several areas of concern; however, these efforts are not yet demonstrated to determine their effectiveness at lowering risk.
SSEB at 62.
LM disputes both the agency’s evaluation of its performance under several of the individual contracts and the determination that there was little demonstrated systemic improvement.
(1) MQ-9 Reaper/GCS
GA-ASI’s ongoing Air Force
contract No. F33657-02-G-4035 included very relevant (according to both LM and
NAVAIR) delivery orders for interim contract capability, SDD and production of
the MQ-9 Reaper (again, a second-generation Predator B model), and for the
pre-production and production of UAS ground control stations (GCS). NAVAIR received three Contractor Performance
Assessment Reports (CPAR) for these contract efforts, the most recent completed
on
LM challenges the overall marginal rating for GA-ASI’s performance under this contract on the basis that this overall rating was inconsistent with the category ratings in the latest 2006 Air Force CPAR of very good for technical, satisfactory for management, and marginal for schedule and cost control. However, while recognizing that GA-ASI “does an excellent job responding to quick reaction and rapidly evolving warfighter requirements in support of the Global War on Terror,” the 2006 CPAR nevertheless expressed significant reservations as to GA-ASI’s performance in several areas:
Systems engineering was rated satisfactory overall but remains an area of concern for the program. The company has not been able to develop a sufficient systems engineering staff to keep pace with the numerous other contracted efforts.
While satisfactory overall, [software engineering] is an area of concern for the program. Although the company continues to increase its software engineering staff, there continues to be limited software engineering resources to complete all contracted work. . . . The contractor needs to continue to increase their engineering staff in order to meet contracted commitments in parallel.
Several projects under this [Basic Ordering Agreement] have suffered from schedule delays, to include the MQ-9 ICC and MQ-9 SDD efforts. The schedule variances for these efforts are -20% and -44.9% respectively as of Dec. 06. While [GA-ASI] has committed to expanding the workforce, the contractor has insufficient resources to execute the contracted work on schedule in several key areas. The resulting schedule delays directly impact the fielding of combat capability.
CPAR, Contract No. 4035, 2006 Period. The 2004/2005 CPAR for contract No. 4035 included similar criticisms of GA-ASI’s performance, as well as marginal schedule and cost control ratings. Given the above continuing staffing and resources shortfalls, which resulted in “schedule delays directly impact[ing] the fielding of combat capability,” CPAR, Contract No. 4035, 2006 Period, and the repeated marginal schedule and cost control ratings in the most recent CPARs, we find no basis to question NAVAIR’s evaluation of GA-ASI’s overall performance under contract No. 4035.
LM further challenges NAVAIR’s assessment that systemic
improvement by GA-ASI on contract No. 4035 (as well as under other contracts)
had not been demonstrated. In this
regard, LM generally acknowledged in its December 4, 2007 discussions response
with respect to a number of GA-ASI contracts (including the MQ-9 Reaper, MQ-1C
ER/MP, and I-GNAT Extended Range UAS), that “the fundamental cause for GA-ASI
Past Performance issues was availability of trained staff to meet the demand
for our products and services”; that GA-ASI’s workload had exceeded its
capacity; that there had been “management task saturation”; and that there was
a “valid CPAR comment” regarding (overly) “[c]entralized management
structure.” LM Evaluation Notice (EN)
Response,
The agency’s evaluation in this area was reasonable. Although LM has suggested that the RFP did not require that there be documented results of any systemic improvements, and that merely hiring additional staff should be accepted as effective systemic improvement, as noted above, the RFP in fact required the offeror to “identify those systemic improvement actions taken to resolve past problems, . . . demonstrate the extent to which it will benefit the instant contract,” and “[p]rovide the records of such results and indicate where they are documented, preferably in Government record systems.” RFP L.3.4.3. Accordingly, in ascertaining whether there had been systemic improvement in correcting prior performance deficiencies and problems, the agency properly looked to see whether the record “demonstrate[d] the impact of the systemic improvement,” including whether there were any results of the claimed systemic improvement measures “in a record or documentation to show that action resulted in a tangible improvement to that program,” such that there was “independent verification [of] tangible improvement.” Tr. at 778-80.
Given the solicitation requirement that any improvements in
contract performance be documented, the agency reasonably determined that
overall systemic improvement by GA-ASI on contract No. 4035 had not been
shown. In this regard, as noted above,
notwithstanding the agency’s November 2007 request to the Air Force for updated
contract performance information, an updated CPAR or other updated past
information had not been furnished by the Air Force. Further, while LM furnished its own updated
Earned Value Management System (EVMS) data on contract No. 4035 in a
(2) MQ-1C ER/MP
Both NAVAIR and LM considered ongoing Army contract No. W58RGZ-05-C-0069, for the MQ-1C Extended Range/Multipurpose (ER/MP) UAS (a second-generation Predator model using the basic structure of the Predator aircraft with the Predator B avionics suite), to be very relevant to LM’s proposed BAMS Predator-based Mariner UAS. LM Past Performance Proposal at 3-9, 3-51. NAVAIR received for this contract: four past performance questionnaire (PPQ) responses, including December 10, 2007 and April 2007 responses from the Army Procuring Contracting Officer (PCO), an April 2007 response from the Army Product Manager, and a February 26, 2007 response from the Defense Contract Management Agency (DCMA) Administrative Contracting Officer (ACO); and a number of LM discussion responses that referred to the contract (as well as a number of other contracts).
LM challenges the overall marginal rating for GA-ASI’s performance under this contract, primarily on the basis that this rating was inconsistent with the input from the DCMA ACO and LM’s discussion responses.
We find that the agency reasonably rated GA-ASI’s
performance under contract No. 0069 only marginal. In this regard, the most recent detailed
information received by NAVAIR for this contract was the Army PCO’s
[GA-ASI has not met contracted . . . delivery schedules.
[GA-ASI] continues to struggle as the Systems Integrator.
[GA-ASI] has resisted hiring adequate engineering and technical staff to address all of the tasks they are currently contracted to perform.
The common theme within the delivery/schedule problems appears to relate back to the acceptance of contractual commitments which are physically beyond production capacity.
A major contributor is [that GA-ASI’s] senior management continues to obligate the company without fully reviewing and understanding the current workload and commitments.
Management task saturation coupled with [GA-ASI’s] highly centralized management structure both contribute towards the delays with the integration testing and coordination efforts . . . .
The engineering staff appears to be technically [competent], but in most cases are not empowered at the appropriate levels to make the necessary decisions to push the task forward in a timely manner to maintain schedule.
[GA-ASI] has made limited corrective actions and usually not without Government PMO insistence.
Army PCO PPQ Response, Contract No. 0069,
LM asserts that the overall marginal rating for GA-ASI’s performance on contract No. 0069, for the MQ-1C ER/MP, did not reasonably account for the February 26, 2007 PPQ response completed by the DCMA which reported that GA-ASI’s technical/quality and schedule performance was exceptional, its cost performance was very good, and its management performance was very good to exceptional.
We find LM’s position unpersuasive. As an initial matter, we agree with the agency that the DCMA ACO furnished little detail in support of his very favorable performance ratings, and that the detail that was furnished appears in some measure inconsistent with the high ratings. In this regard, for example, while the DCMA ACO rated GA-ASI’s cost performance as very good and its schedule performance as exceptional, the DCMA ACO reported cumulative, unfavorable EVMS ratings of 0.84 for CPI and 0.91 for SPI as of January 2007. Although the DCMA ACO stated that government-directed changes were the cause of schedule and cost issues, he also acknowledged that $18 million of a predicted $37 million cost overrun was believed to be the result of “cost growth within the contract scope,” as distinct from “scope growth,” and he referred to the fact that “[c]orrective actions are on-going,” including continued hiring by GA-ASI, thus seemingly implying that there was some contractor responsibility for performance problems. Tr. at 1784-93. In any case, the DCMA ACO’s response was furnished in February 2007, while the more detailed responses by the Army PCO and Army Product Manager represented more recent assessments based on the contract performance as of April and December 2007. We note in this regard that the PPQs completed in April 2007 by the Army PCO and Army Product Manager both rated GA-ASI’s performance as marginal to satisfactory for technical/quality and schedule performance, and satisfactory for cost and management performance, thus suggesting both that GA-ASI’s performance had deteriorated since the DCMA ACO’s assessment in February and continued to deteriorate up to the time of the marginal performance reported by the Army PCO in December 2007. Army PCO PPQ Response, Contract No. 0069, Apr. 2007; Army Product Manager PPQ Response, Contract No. 0069, Apr. 2007. Moreover, to the extent that the differences in assessment of GA-ASI’s performance represented a difference of opinion, as distinct from a mere change over time in the quality of performance, we consider it significant that it was the views of the Army PCO and Army Product Manager, rather than those of the DCMA ACO, that were broadly consistent with the reports in the Air Force CPARs on GA-ASI’s performance on the MQ-9 Reaper/GCS delivery orders, that is, the reports of continuing GA-ASI staffing and resources shortfalls resulting in schedule delays. In summary, based on the recent, detailed information concerning GA-ASI’s performance problems on contract No. 0069, which information was consistent with reports of staffing and resource shortfalls resulting in schedule delays on other contracts, NAVAIR reasonably rated GA-ASI’s performance on this contract as no better than marginal.
LM asserts that, in any case, NAVAIR failed to account for
systemic improvement by GA-ASI, such as increased staffing. As noted above, however, the RFP required a
showing of documented improvements in contract performance as a result of any
claimed systemic improvement measures. RFP sect. L.3.4.3.
While GA-ASI has apparently continued to increase its workforce, again,
an increase in workforce by itself, without a documented improvement in
contract performance, did not meet the solicitation standard for showing
systemic improvement. Here, not only did
the PPQ responses seem to suggest deteriorating performance on contract No.
0069 through December 2007, but, in addition, recent EVMS data furnished by LM
during discussions was not favorable. In
this regard, in its December 6, 2007 discussion response to EN LM‑PP-015,
LM indicated that the MQ-1C contract had been rebaselined after performance
resulted in increasingly unfavorable EVMS numbers at the beginning of 2007‑‑with
a decline in the CPI from 83.8 percent in January to 80.3 percent in April and
a similar SPI decline from 90.9 percent to 87.1 percent‑‑but
then, after the rebaselining was reflected in the EVMS data in September 2007
with fresh 100-percent CPI and SPI ratings, the numbers again began to decline,
falling to 98.4 percent and 98.2 percent, respectively, in
October 2007. LM Response to EN LM‑PP-015,
(3) I-GNAT Extended Range UAS
Both NAVAIR and LM considered Army contract No. DAAH01-03-C-0124, ending in December 2007, for the I-GNAT Extended Range (ER) UAS (an Army version of the Predator), to be very relevant to LM’s proposed BAMS Predator-based Mariner UAS. LM Past Performance Proposal at 3-9, 3-70. NAVAIR received three PPQ responses for this contract: a May 31, 2007 response from the Army Deputy Product Director, Unmanned Aerial Systems Program Management Office; an April 18, 2007 response from the Army PCO (who was also the Program Manager (PM) for this program (according to LM, LM Past Performance Proposal at 3-7), and the PCO for the MQ-1C ER/MP contract); and an April 2007 “coordinated” response from the DCMA ACO (who also was the ACO for the MQ-1C ER/MP contract) and the DCMA Program Integrator, which was subsequently updated by the DCMA ACO on May 24, 2007. In addition, NAVAIR received several LM discussion responses that referred to the contract (among a number of other contracts).
LM challenges the overall marginal rating for GA-ASI’s
performance under this contract, primarily on the basis that it fails to
account for the DCMA input. In this
regard, the record reflects what appears to be an irreconcilable difference
between the Army and DCMA evaluations of GA-ASI’s performance. On the one hand, the Army Deputy Product
Manager and the Army PCO/PM agreed on marginal ratings for technical/quality,
schedule, cost and management performance based on concerns that GA‑ASI
had “consistently failed to meet contractual delivery dates for the spares and
Ground Data Terminals and [was] beginning to show moderate slippage on delivery
dates for Air Vehicles and Ground Control Stations”; had demonstrated
resistance to hiring adequate personnel; had overly centralized management
structure that contributed to program delays; had difficulty in managing its
subcontractors; and ultimately was
“agreeing to contractual commitments which are beyond its production
capacity.” PPQ Response, Contract No.
0124, May 31, 2007, Army Deputy Product Director; PPQ Response, Contract No.
0124, Apr. 18, 2007, Army PCO/PM. On the
other hand, the coordinated response from the DCMA ACO and DCMA Program Integrator
offered the summary conclusion that there had been exceptional
technical/quality, schedule, cost and management performance, with “generally”
on-time performance or, as of May 2007, “on schedule” aircraft deliveries and
“[n]o major slippage on the production schedule.” PPQ Response, Contract No. 0124,
Apr. 2007, DCMA ACO; E-mail from NAVAIR to DCMA ACO,
We find that NAVAIR’s evaluation of GA-ASI’s performance on
contract No. 0124 was reasonable.
Confronted with materially differing ratings from the Army and DCMA
representatives as of May 2007, the agency unsuccessfully sought updates from
both entities on November 28, 2007, E-mail to Army Deputy Product Manager, Army
PCO/PM, and DCMA ACO, Nov. 28, 2007, and also raised the negative past
performance information from the Army with LM in a series of ENs in October
2007. Of particular significance in this
latter regard were LM’s October 12 responses to EN LM-PP-009 and EN LM-PP-11,
in which LM did not specifically refute the reports that GA-ASI had failed to
meet a number of contractual delivery dates, but essentially maintained that it
was simply “being responsive to the customer’s aggressive requests” and that
any performance difficulties were beyond its control. LM Response to EN LM-PP-009,
In summary, we find that LM’s challenges to the evaluation of its team’s past performance provide no basis for questioning the agency’s determination that the LM team‑‑in particular, GA-ASI‑‑had a recent past performance history of being unable to resolve staffing and resource issues, which resulted in adverse cost and schedule performance. We further find no basis for questioning the agency’s determination that, notwithstanding such systemic improvement measures as hiring additional staff, LM did not establish documented improvements in contract performance as a result of the systemic improvement measures; these efforts therefore did not furnish a basis for reducing the risk associated with the LM team’s unfavorable past performance.
Past Performance of NG
As noted above, NG’s performance was rated as marginal on only 3 of 20 contract efforts, very good on 10 and exceptional on 3. The SSEB specifically found that the NG division serving as the prime contractor/system integrator (NG Integrated Systems, Eastern Region) had satisfactory to very good past performance, demonstrating “high quality technical performance” on each of seven relevant contracts and favorable or excellent cost control on six of those contracts. SSEB at 63. Furthermore, the SSEB found that the overall NG team demonstrated excellent program management on most relevant contracts.
However, the performance of NG Integrated Systems, Western
Region,
Although NAVAIR recognized the potential for the NG
Western Region performance problems on the Global Hawk EMD contract to
translate into risk for NG’s Global Hawk-based BAMS effort, the agency
identified other considerations that mitigated this risk: NG’s overall generally very
good-to-exceptional technical/quality ratings; its flexible and responsive
management as indicated in most performance reports; the favorable performance
of NG Eastern Region, the proposed prime contractor/system integrator, in
controlling cost; and demonstrated systemic improvement with respect to most
prior performance problems. Regarding
the evaluated demonstrated systemic improvement, the SSEB determined that: of the five contracts on which there had
been unfavorable cost performance, systemic improvement had been demonstrated
on one, and some systemic improvement had been demonstrated on three (including
the Global Hawk EMD); of the three contracts on which there had been
underestimation of software development and integration resulting in schedule
delays and cost overruns, systemic improvement had been demonstrated on two; of
the five contracts on which there had been unfavorable schedule performance,
systemic improvement had been demonstrated on two (including the Global Hawk
EMD), and some systemic improvement had been demonstrated on two (including the
Global Hawk LRIP); and of the three contracts on which there had been
marginal management of subcontractors, systemic improvement had been
demonstrated on one and some systemic improvement had been demonstrated on
another. Based on this evidence, the
SSEB determined that only some doubt existed that NG would be able to
successfully perform the BAMS effort, thus warranting a moderate overall risk
rating (in contrast to the substantial doubt regarding successful performance
and resulting high risk rating for LM’s proposed contract effort).
LM asserts that NG’s moderate risk rating did not adequately account for NG’s performance problems, particularly on the Global Hawk EMD contract, and that the agency unreasonably credited NG with systemic improvements on a number of contracts, particularly the Global Hawk EMD and Global Hawk LRIP contracts. Based upon our review of the record, we find no basis for questioning the evaluation of NG’s past performance.
As indicated above, and as is otherwise amply demonstrated
in the record, NAVAIR fully recognized the problems encountered by NG Western
Region under the Global Hawk EMD contract, including those with respect to
unfavorable cost performance, underestimation of software development and
integration resulting in schedule delays and cost overruns, unfavorable
schedule performance, and marginal ability to manage subcontractors, all
resulting in an overall performance rating of marginal. SSEB at 67‑76; Contract Performance
Analysis Worksheet,
LM asserts that NAVAIR unreasonably credited NG with
systemic improvement with respect to most reported past performance
problems. Again, however, the record
supports the agency’s conclusion. For
example, the SSEB noted with respect to the prior unfavorable cost performance
under the Global Hawk EMD contract, that NG had implemented a number of cost
control measures, including a [REDACTED], designed to improve program oversight
and management and maintain cost and schedule within two percent of the
baseline. In this regard, the Air Force
program director advised NAVAIR on
the program has put in place a new Acquisition Program Baseline and is executing to it. . . . The contractor and government have made management changes and have put in place strong mechanisms to control cost and schedule and to predict growth in either. Overall, the program has made significant changes and I believe that it is now healthy and on a path to success.
E-mail From Program Director to NAVAIR,
In contrast, with respect to the prior underestimation of
software development and integration resulting in schedule delays and cost
overruns, the SSEB noted that as recently as September 7, 2007, the Air Force
had expressed concern to NG as to a 20‑month delay in the Block 10
software, which originally was scheduled for completion in January 2006, but
instead was rescheduled for operational release in late September 2007. The SSEB further noted that, while NG had
implemented a number of systemic improvement measures in this area, the
contractor had conceded that many of the initiatives were still in the early
stages of development. Based on this
information, the SSEB determined that systemic improvement had not been
demonstrated in this area. The SSEB
reached a similar conclusion with respect to the previous determination
regarding NG’s marginal ability to manage subcontractors. SSEB at 66-76; Declaration of Past
Performance/Experience Team Lead,
As is apparent from the above discussion, the SSEB undertook a detailed, reasoned approach to determining whether NG had demonstrated systemic improvement. LM has not shown the agency’s methodology or conclusions to be unreasonable. While LM suggests that there was no cost data to demonstrate systemic improvement, and further asserts that the agency failed to consider the impact of the software problems on the overall schedule, we think that, given the favorable EVMS data for 2007‑‑including a CPI of not less than .99 and an overall SPI of not less than .98 between January and October 2007‑‑and the various systemic improvement measures undertaken by NG (including its [REDACTED], the agency reasonably determined that some systemic improvement had been demonstrated in the cost area and that systemic improvement had been demonstrated in the overall schedule.
We conclude that the agency reasonably determined that only some doubt existed that NG would be able to successfully perform the required effort, thus warranting a moderate overall risk rating.
TECHNICAL
As noted above, NG’s proposal was evaluated as having a strong advantage over LM’s under the design approach subfactor of the technical factor based in part on the evaluation of NG’s proposed BAMS UAS as having significant strengths in the emphasis areas of: ETOS, with NG offering a greater persistent intelligence, surveillance, and reconnaissance capability with its proposed UAS orbit evaluated as having an ETOS of 96.2 percent, significantly higher than LM’s evaluated ETOS of 84.6 percent; and SWaP, including NG’s weight margin of [REDACTED] pounds, nearly [REDACTED] times greater than LM’s margin of approximately [REDACTED] pounds. NAVAIR noted that NG’s superior SWaP future growth capability could be used to incorporate future capability increments without breaching the ETOS threshold requirements, and that the resulting significantly greater design margin would reduce the risk of unknown design challenges that could require increased SWaP, particularly weight. NAVAIR concluded that NG’s strong advantage under the design approach subfactor offset a slight advantage in favor of LM’s proposal under the program schedule subfactor such that NG’s proposal had a significant overall advantage over LM’s under the technical factor. SSAC PAR at 3-7.
LM raises several challenges to the evaluation under the
technical factor. We find that none
furnishes a basis for questioning the reasonableness of the agency’s overall
determination of NG’s advantage under the factor. For example, LM asserts that the ETOS
calculation for its proposed Predator-based solution understated the time on
station because it was improperly based on an orbit of four aircraft,
ignoring the fact that LM had updated its original approach to add a fifth
aircraft in its December 4 proposal revisions.
However, the record supports the agency’s interpretation of LM’s revised
proposal as continuing to propose a 4‑aircraft orbit for purposes of the
ETOS calculation. Neither the origin of
the revisions nor the resulting revised proposal supports LM’s interpretation
of a 5‑aircraft orbit for purposes of the ETOS calculation. In this regard, LM advised NAVAIR in its
December 4 proposal revision that “[a]s a result of Government comments and
after rerunning the Government’s model received as part of EN LMGA-C-0040, we
have added an additional aircraft to our LRIP 1 cost and schedule. We will now produce 5 aircraft as part of
LRIP 1.” LM/GA Proposal Update, Response
to the 28 November BAMS Face-to-Face Discussions, at 2-3. However, EN LMGA-C-0040 was a cost EN, not a
technical EN, and the relevant section of the EN advised LM that the agency had
concluded that its proposal of 33 aircraft would not be sufficient to sustain
full operating capability for the required 20 years. Further, LM’s submissions from before and
after the December 4 revisions all referred to an orbit or input of
four aircraft, including: LM’s
Technical Proposal Figure 2.1.4-6 “ETOS Calculator Inputs,” responsive to
the RFP requirement that offerors “[i]dentify all assumptions used in
developing ETOS predictions” and complete an “ETOS Input Table,” RFP L.2.1.4;
the narrative in its Technical Proposal Section 2.1.4, “Effective Time on
Station”; and the proposal’s executive summary.
LM Executive Summary at 3‑4, 10, 22; LM Executive Summary Errata,
The protest is denied.
Gary L. Kepplinger
General Counsel
[1] While LM maintained that the EVMS data were invalid and not reliable, this only confirms the absence of any documentation of systemic improvement.
[2] LM
further asserts that, in evaluating the past performance of GA-ASI, NAVAIR
failed to account for information in DCMA’s Informal Preaward Survey on GA-ASI,
the results of which were reported to NAVAIR on or about
[3]
Consistent with the agency’s assumed 80 percent availability factor, and NG’s
proposal of three aircraft for the ETOS calculation, NG’s proposal was
evaluated on the basis of four aircraft per site. (LM’s proposal was evaluated on the basis of
five aircraft per site, consistent with its evaluated proposal of four aircraft
for the ETOS calculation.) Final
Evaluation SSAC Briefing,