INSPECTOR GENERAL
WASHINGTON, DC 20570
March 21, 2002
I hereby submit an Audit of Monitoring the Computer Maintenance Contract, Report No. OIG-AMR-34-02-01. This review was conducted to ascertain whether services specified in the computer maintenance contract were performed satisfactorily at the National Labor Relations Board (NLRB or Agency). We identified several areas where Help Desk operations could be improved. The other mandatory and optional services were performed satisfactorily.
The base cost for computer maintenance services for calendar year 2001 was approximately $2.2 million. Mandatory operational support included Help Desk services, monitoring networks, participating in evaluations of technology, providing technical assistance, and maintaining a reference guide.
Help Desk services included providing technical support for all hardware and software calls received. As discussed in the following paragraphs, customer feedback was minimal, the Agency overpaid for services, and some services were not timely.
The response rate to customer satisfaction surveys was 19 percent, which appears to be within a reasonable range experienced by other organizations. Nevertheless, improvements are attainable and may provide financial benefits to the Agency. The difference in performance bonus payments between a customer satisfaction level greater than 97 percent and one less than 90 percent is over $14,000 per month.
The Agency overpaid $3,899.88 in performance bonuses to the computer maintenance contractor for services between January and October 2001.
We reviewed 100 Help Desk calls to evaluate the timeliness of Help Desk resolution of service calls. Generally, the contractor provided Help Desk services required by the contract and resolved calls within 16 hours. The contractor, however, did not consistently escalate issues not resolved within 16 hours to NLRB officials as required by the contract. Nine calls were open longer than 16 hours, but five of them were not escalated to the appropriate Agency official as required by the contract.
The contractor did not complete hardware repairs or provide replacements or loaner equipment within the time periods stipulated in the contract. Of the 78 problems that we reviewed, 13 (17 percent) were not completed within the time period required by the contract - 8 hours at Headquarters and 12 hours at field offices. This was more pronounced with field offices. Thirty-nine of the calls reviewed were from field offices and 12 were not completed within 12 hours.
Our recommendations made to the Director of Administration include evaluating the survey methodology being used, reviewing the reasonableness of Help Desk response times, collecting overpayments, reviewing prior payments, and implementing a payment review process.
An exit conference was held on February 20, 2002 with the Director of Administration, Chief Information Officer, Customer Support Section Chief, and the Procurement and Facilities Branch Chief to discuss the findings and recommendations. A draft audit report was sent to the Director of Administration on February 21, 2002, for review and comment. She disagreed with the conclusion that the contracting officer was not properly appointed and the recommendation to evaluate the customer satisfaction survey methodology. She accepted the other four recommendations. The Director of Administration's comments are presented in their entirety as an appendix to this report.
Jane E. Altenhofen
Inspector General
The National Labor Relations Board (NLRB or Agency) administers the principal labor relations law of the United States, the National Labor Relations Act of 1935, as amended, which is generally applied to all enterprises engaged in interstate commerce, including the United States Postal Service, but excluding other governmental entities as well as the railroads and the airline industries. The Agency performs its mission by: (1) conducting secret ballot elections to determine if a group of employees wishes to be represented, for collective bargaining purposes, by a labor organization; (2) adjudicating representation issues if the parties cannot reach agreement; (3) investigating charges of unfair labor practices filed by the public with the Agency; (4) prosecuting, if the parties cannot settle and reach an agreement, those cases of unfair labor practices which the Agency determined to have merit; and (5) adjudicating those unfair labor practice cases which the Agency litigates. Approximately 30,000 charges of unfair labor practices and 6,000 representation petitions are filed with the NLRB each year.
In Fiscal Year (FY) 2001, the Agency employed about 1,992 people at Headquarters and in 32 Regional, 3 Sub-regional and 16 Resident Offices (field offices). The Agency's FY 2001 appropriation was $216 million.
In December 1999, the NLRB entered into a contract for computer maintenance services for calendar year 2000 with 4 optional years. Operational support services were procured using the firm fixed price methodology. The base cost for 2001 was approximately $2.2 million. Optional services for between $25,000 and approximately $1.7 million were available through indefinite-delivery indefinite-quantity type contracting in calendar year 2001.
Most Agency employees have personal computer (PC) workstations consisting of a central processing unit, monitor, and printer. The PCs are connected to local area and wide area networks that provide a communication linkage to Headquarters. The Help Desk received 4,624 service calls between January 1 and June 30, 2001.
Mandatory operational support includes Help Desk services; monitoring networks; participating in evaluations of proposed hardware, software, and networking technologies; providing technical assistance in integration and configuration, testing, implementation, and management of desktop solutions; and maintaining a reference guide for handling future calls.
The audit was conducted to ascertain whether services specified in the computer maintenance contract were performed satisfactorily. Our audit covered services provided between January 1 and June 30, 2001. We expanded our review of performance bonuses to cover the period through December 31, 2001 to follow up on errors identified between January 1 and June 30, 2001.
We evaluated monthly reports prepared by the contractor to determine whether they included information required by the contract and information necessary to evaluate contractor performance. We also identified and gained an understanding of the methodology used to compile information contained in the reports.
We selected a statistical sample of 78 items to determine whether Help Desk services were being performed by the contractor and to evaluate customer satisfaction survey response rates and user satisfaction. We selected another statistical sample of 78 items to determine whether hardware repairs were made within required timeframes. We selected a judgmental sample of 100 items and reviewed them to determine whether they were handled in a timely manner or escalated to NLRB management attention after 16 hours, as required by the contract.
We interviewed Agency and contractor employees and reviewed documents relating to non-help desk functions performed by the contractor including: monitoring networks; participating in evaluations of proposed hardware, software, and networking technologies; providing technical assistance in integration and configuration, testing, implementation, management of desktop solutions; and maintaining a reference guide for handling future calls. We reviewed delegations of authority to determine whether they were made in writing and were consistent with employees' positions and responsibilities.
We reviewed optional services purchased by the Agency to determine whether tasks were already included as mandatory contract requirements, costs were within estimates prepared by the Information Technology Branch (ITB), and inspection and acceptance were performed in accordance with the contract.
This audit was performed in accordance with generally accepted government auditing standards during the period August 2001 through March 2002.
DELEGATION OF AUTHORITY
The contracting officer (CO) was not properly appointed. Title 48, section 1.60 of the Code of Federal Regulations states that the CO shall be appointed in writing on a certificate of appointment, SF-1402. The appointment shall state any limitations on the scope and authority to be exercised by the CO and be readily available to the public and agency personnel. Without this appointment the CO operated without authority from at least May 10, 2000, when she began signing contract modifications.
The CO was appointed using a certificate of appointment on February 19, 2002.
MONTHLY REPORTS
Monthly reports prepared by the contractor contain information required by the contract and provide a comprehensive inventory of services provided to the Agency. The reports contain: Help Desk call activity; network monitoring results; status of trouble tickets, work orders, and long term projects; planned activities; and the results of user satisfaction surveys. In addition to monthly reports, the contractor prepared reports after each field office deployment.
The Agency does not conduct independent tests to determine the accuracy of information in the monthly reports, but the contracting officer's technical representative (COTR) and the ITB Infrastructure Section Chief are in frequent, usually daily, contact with the contractors. This provides them with timely information to judge the veracity of the information contained in the reports.
HELP DESK
User Satisfaction
In order to evaluate the response rate to user satisfaction surveys, we searched the Internet for appropriate benchmarks to compare against the Agency's response rate of 19 percent. We found a wide range of response rates with the surveys we reviewed, but the Agency's response rate appeared to be typical of that at other organizations. Even though we were unable to identify a standard, companies providing survey services provided tips to increase response rates, such as informing respondents, in the survey request, how the survey will be used, setting a deadline, and sending reminders. Improvements in the Agency's response rate are attainable and may provide financial benefits to the Agency.
The contractor is required to perform a customer survey of 100 percent of the service calls. The contract, however, does not provide a minimum response rate or require that the contractor follow up on surveys that did not receive a response. The contract provides a bonus equaling 3 percent of the monthly payment amount if the customer satisfaction level meets or exceeds 97 percent. If the customer satisfaction rate falls below 90 percent for 2 consecutive months, the Agency shall reduce the payment by 5 percent for the 2 months. Monthly reports showed that the contractor logged 4,624 calls during the period January 1 through June 30, 2001 and received 891 user responses to customer satisfaction surveys (19 percent).
Out of 78 Help Desk calls in our sample, that were from the period between January 1 and June 30, 2001, users responded to the survey 13 times (17 percent). In November 2001, we followed-up on the remaining 65 items to determine whether the user satisfaction for those not returning the survey was consistent with those who did return the survey. Our results were inconclusive due to the large number of respondents who could not remember whether they were satisfied with the contractor's performance. The results are shown below.
| Number | Percent | |
|---|---|---|
| User satisfied | 31 | 48 |
| User not satisfied | 4 | 6 |
| User could not remember | 21 | 32 |
| User not available to complete the survey because the employee left the Agency or is on extended leave | 7 | 11 |
| Survey not applicable because NLRB employee performed the task after call was escalated to them | 2 | 3 |
| Total | 65 | 100 |
The base price of the computer maintenance contract for calendar year 2001 was $2,160,008.44, or $180,000.70 per month. A 3 percent bonus on this monthly payment is $5,400.02 and a 5 percent penalty is $9,000.04 for a total difference in the monthly payment of $14,400.06
Performance Bonus
The Agency overpaid $3,899.88 in performance bonuses to the computer maintenance contractor for services between January and October 2001. The Agency had not paid for November and December 2001 services by the completion of our fieldwork. The overpayment resulted from incorrectly calculating the performance bonus to be $5,833.34 for months when performance goals were met as opposed to the correct amount of $5,400.02. This amounted to a difference of $433.32 for each of the 9 months in which a bonus was paid.
The monthly payments of $5,833.34 were based upon an obligation of $70,000 for the year. The monthly payments were based on this estimate rather than the actual contract costs. The invoices were not adequately reviewed by ITB or the Procurement and Facilities Branch for accuracy and compliance with contract provisions.
Contractor Provided Services
Generally, the contractor provided Help Desk services required by the contract. We selected a statistical sample of 78 Help Desk calls to determine whether the contractor was providing Help Desk services required by the contract, or whether those services were being inappropriately performed by Government employees. Four of the 78 Help Desk calls were resolved by NLRB employees. Three of these calls were for services not covered by the contract and were appropriately resolved by Agency employees. These consisted of establishing user access rights, troubleshooting the Agency mail system, and Internet issues dealing with the Government Printing Office. The fourth call dealt with a printer. The call was originally assigned to a contractor employee. After approximately 5 hours, the service call was forwarded to a NLRB employee for resolution.
Timeliness of Contractor Response
The contractor generally resolved Help Desk calls in a timely manner, but did not consistently escalate issues not resolved within 16 hours to NLRB officials as required. We reviewed 100 Help Desk calls to evaluate the timeliness of Help Desk resolution of service calls. Ninety-one of the service calls were closed within 16 hours, and 9 were open longer than 16 hours. The contractor is required to escalate all problem records remaining open for more than 16 hours to the COTR and/or Chief of Customer Support or a designee. We reviewed these nine service calls to determine whether they were escalated as required. Five out of nine services calls over 16 hours old were not escalated in accordance with the contract.
Hardware Repairs
The contractor did not complete hardware repairs or provide replacements or loaner equipment within the time periods stipulated in the contract. The contract requires the contractor to repair all Headquarters hardware problems within 8 work hours and all field office hardware problems within 12 work hours. If the contractor does not perform the repair within the above-stated timeframes, they are required to provide a loaner of equal or better quality.
From 1,317 hardware problems, we reviewed a statistical sample of 78 service calls: 39 field office calls and 39 Headquarters calls. Thirteen calls - 12 field office and 1 Headquarters - required the replacement of equipment and were not completed within 12 hours in field offices and 8 hours at Headquarters as required by the contract.
NON-HELP DESK MANDATORY SERVICES
Generally, the contractor provided non-Help Desk related mandatory services in conformance with the contract and to the satisfaction of Agency officials responsible for monitoring the contract.
OPTIONAL SERVICES
Between January 1 and June 30, 2001, the Agency issued four purchase orders and one task order to procure optional services through the computer maintenance contract. These services totaled $550,698.95 and included: technical writing of reference, operational, and procedural manuals; operational support for the System 80 during the absence of the primary operator; field office deployments; and field office equipment moves.
Our review showed that these items were not already included as mandatory operational support requirements in the computer maintenance contract, costs were within estimates prepared by ITB, and inspection and acceptance were performed in accordance with the contract.
We recommend that the Director of Administration:
1. Evaluate the customer satisfaction survey methodology being used and make changes to increase the response rate.
2. Evaluate the reasonableness of Help Desk response times related to computer hardware service calls and consider including consequences or penalties for not meeting stated response times in the next computer maintenance contract.
3. Review prior year bonus payments to determine whether they were made in accordance with the terms of the contract and calculated correctly.
4. Recover overpayments of monthly performance bonuses from the contractor.
5. Implement a process to verify that similar future payments are adequately supported and reviewed by ITB and the Procurement and Facilities Branch.
We concluded that the CO was not appointed on a certificate of appointment, SF-1402, and, therefore, operated without authority. The Director of Administration responded that the contract was modified to appoint the CO, and that the modification indicates that the CO was appointed and operating with authority. The contract modification designated, rather than appointed the CO. Additionally, the modification was executed by the CO, whereas the SF-1402 was signed by the Director of Administration.
The Director of Administration disagreed with our recommendation to evaluate the customer satisfaction survey methodology being used and make changes to increase the response rate because no or insignificant benefits would result. The Director noted in her response, "The OIG followed up on customers who did not complete the survey and found that there would not be a significant change from the results of the 19% return." Our report stated that our results were inconclusive due to the large number of respondents who could not remember whether they were satisfied with the contractor's performance. Nevertheless, we will accept the Director's response as sufficient evaluation of the survey methodology.
The Director of Administration's comments are presented in their entirety as an appendix to this report.
UNITED STATES GOVERNMENT
National Labor Relations Board
Memorandum
TO: Jane E. Altenhofen
Inspector General
FROM: Gloria J. Joseph
Director of Administration
DATE: March 18, 2002
SUBJECT: Draft Report "Audit of Monitoring the Computer Maintenance Contract"
Thank you for the opportunity to comment on the draft report. We do not concur with one of the OIG report findings and one recommendation. We do not agree that:
(1) "The contracting officer (CO) was not appointed in writing." and " Without this appointment the CO operated without authority".
On May 10, 2000, Section G.1, Contracting Officer's (CO) Authority, paragraph (c), of the Optimus contract was modified to appoint the CO (a copy will be provided). This modification provided official, written documentation of the CO's authority to administer the contract. Title 48, Chapter 1, Subpart 1.6 of the Code of Federal Regulations does state that the CO shall be appointed in writing on a Certificate of Appointment, SF 1402. In addition, the appointment shall state any limitation on the scope and authority to be exercised by the CO and readily available to the public and agency personnel. While the CO's appointment was not documented on the SF 1402 at the time the contract was awarded, the modification indicates that the CO was appointed and operating with authority.
We do not agree with the recommendation to:
"Evaluate the customer satisfaction survey methodology being used and make changes to increase the response rate."
The OIG report acknowledges that the 19% response rate is typical. There is no benefit to taking action to increase this rate. There would be a benefit only if the negative response was significant enough to alter the current distribution of responses. The OIG followed up on customers who did not complete the survey and found that there would not be a significant change from the results of the 19% return. Even if there was a slight shift, any benefit would be less than the cost of the efforts to increase the response rate by a more active campaign or by additional follow-up. Also, many users have already expressed a preference to not complete a survey after each service. They would prefer to take the initiative to make contact only if they were dissatisfied.
Actions to increasing the response rate would contribute nothing to the cost efficiency or customer satisfaction of the service provided, but would require additional cost and effort, and irritate our customers.