This is the accessible text file for GAO report number GAO-03-1067R entitled 'Payment Processing: Statistical Sampling Plan for Voucher Prepayment Examination' which was released on August 28, 2003. This text file was formatted by the U.S. General Accounting Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. On January 5, 2004, this document was revised to add various footnote references missing in the text of the body of the document. August 28, 2003: Ms. Martha Stearns: Deputy Director: Commercial Pay Services: Defense Finance and Accounting Service: P.O. Box 182317: Columbus, Ohio 43218-2317: Subject: Payment Processing: Statistical Sampling Plan for Voucher Prepayment: Examination: Dear Ms. Stearns: This report responds to a request made by your office that we approve the use of a statistical sampling plan for voucher prepayment examination of contract payments. While Title VII,[Footnote 1] "Fiscal Guidance," of the GAO Policy and Procedures Manual for Guidance of Federal Agencies requires prepayment examination of vouchers prior to their certification and payment, it permits the use of statistical sampling for vouchers up to $2,500. You requested a waiver of the $2,500 ceiling for statistical sampling and approval of an alternative voucher prepayment examination plan that would, among other things, use statistical sampling for certain vouchers up to a maximum of $500,000 for contract payments. In order to respond to your request, we reviewed your letter and the information provided in the attachments. We also contacted your staff to discuss your request in more detail. We analyzed the sections of the law pertaining to this request and reviewed the legislative history of 31 U.S.C. § 3521(b), upon which Title VII is based. Section 3521(b) of Title 31 provides for statistical sampling procedures. We did not test the Defense Finance and Accounting Service (DFAS) contract payment system; therefore, our response only addresses your request conceptually. At this time, we cannot approve your request for a waiver or your proposed alternative sampling plan. Your submission, requesting exception to the $2,500 maximum for statistical sampling in voucher prepayment examination, does not justify your projected savings of approximately $4.2 million per year. Further, it is unclear how the proposed plan will adequately protect the government's interest. In addition, absent GAO approval, DFAS certifying officers risk personal liability for improper payments should they rely upon the voucher prepayment examination process under the proposed alternative sampling plan. Background: As described in your letter and the information accompanying it and as explained by your staff during subsequent discussions, DFAS is evaluating its existing contract payment procedures with the intent of achieving cost-effective compliance with Title VII. We support such initiatives to create a government that works better and costs less. At the same time, agencies have the responsibility to protect the government's interest. The fiscal guidance in Title VII contains general accounting processing procedures that are required by law. The requirements are designed to complement agencies' existing federal accounting, internal control, and systems standards. All basic vouchers, voucher schedules, and invoices or bills used as vouchers must be certified for correctness and legality by an authorized certifying officer. A certifying officer is accountable for and required to personally reimburse the government for any illegal, improper, or incorrect payment made by a disbursing officer because of his or her certification.[Footnote 2] In addition, a disbursing official is accountable for ensuring that a voucher is in proper form, certified and approved, and computed correctly based on the facts certified.[Footnote 3] In appropriate circumstances, the Comptroller General may relieve a disbursing or certifying official from liability.[Footnote 4] Electronic techniques and systems are being used to generate, process, transmit, and store financial information in today's federal financial environment, requiring accountable officials to rely on the systems, controls, and personnel that process the transactions. However, the accountable officials must have reason to believe that the key processes and controls on which they rely are working.[Footnote 5] Further, even though all or some portion of the payment process may be automated, the certifying officer's basic accountability and legal liability for the propriety of the payment remains unaltered. As a viable alternative to the traditional practice of performing 100 percent prepayment examination of vouchers, section 7.4E of Title VII authorizes agencies to establish statistical sampling programs for the examination of vouchers in support of their certification and payment, provided the vouchers do not exceed $2,500. Within this maximum, agencies are required to establish their own dollar limitations based on cost-benefit analyses of their voucher examining operations.[Footnote 6] Any disbursing or certifying officer relying in good faith on the statistical sampling procedure adopted by the agency to disburse funds or certify a voucher for payment will not be liable for losses to the government resulting from payment or certification of a voucher not audited specifically because of the use of an authorized sampling procedure, provided that the agency has diligently carried out collection actions prescribed by the Comptroller General.[Footnote 7] Exceptions to the $2,500 maximum for vouchers that may be statistically sampled may be authorized by the Comptroller General, on a case-by-case basis, when it is clear that they are warranted economically and that the procedures and controls will adequately protect the government's interest. Prepayment Examination Process at DFAS: DFAS's voucher prepayment examination process is designed to ensure, among other requirements, that (1) payment is permitted by law, (2) appropriation amounts are available at the time and are being used for the intended purpose, (3) goods and services have been received and conform to the requirements of the order or agreement, (4) required administrative approvals have been obtained, and (5) quantities, prices, and calculations are accurate. According to the information submitted with your request, the DFAS Columbus Contract Pay organization entitles[Footnote 8] about 78,000 vouchers monthly at an average total monthly voucher value of $7.2 billion. Contractor invoices are processed and paid through a contract administration and payment system--Mechanization of Contract Administrative Services (MOCAS). Approximately 35,000 of the 78,000 monthly invoices are entitled through an automated payment of invoices (API) process. In the API process, the MOCAS system edit checks provide for the entitlement and automatic payment of these invoices. The remaining 43,000 payment vouchers are manually entitled by approximately 120 voucher examiners and supervisors. Vouchers are manually entitled if, for example, the contract governing the voucher payment has been modified, the contract has special payment instructions, or it fails the edits in the API process. The DFAS Front End Analysis (FEA) organization was established to detect contract pay voucher errors prior to disbursement. One primary function of the DFAS FEA organization is to perform independent voucher prepayment reviews. FEA staff judgmentally select vouchers at various points in the entitlement process and examine them for various attributes, such as input errors, payment according to contract, and availability of funds. According to your letter, FEA's voucher examination procedures are designed to assess MOCAS system errors or individual voucher examiner error rates by nonstatistical sampling from each day's manual entitlements. Of the approximately 78,000 vouchers processed each month, 55,000 are greater than $2,500 and therefore subject to 100 percent prepayment examination in accordance with Title VII. Currently, FEA staff nonstatistically select and examine an average of 1,500 vouchers above $2,500 per month. In a sample of manually entitled invoices from October 2000 through July 2001, FEA noted a primary error rate[Footnote 9] of 13.4 percent, with some of those errors attributable to overpayments totaling approximately $4.8 million for the 10-month period. DFAS's Proposed Alternative Sampling Plan for Prepayment Examination: Your alternative voucher prepayment examination plan for contract payments would shift the role of the FEA independent voucher team, made up of 6.5 full-time equivalents, from detecting input errors and detecting individual voucher examiner errors to playing a key role in the prepayment certification process for accountable officers. Specifically, FEA staff would perform a review of (1) 100 percent of all manually entitled payment vouchers equaling or exceeding $500,000 and (2) a statistical sample of manually entitled vouchers less than $500,000 combined with all API vouchers regardless of amount, thereby excluding API vouchers from the 100 percent voucher examination threshold. All vouchers selected as part of this sampling plan would be reviewed to ensure that they were legal, proper, and correct for payment using standardized review criteria. Any discrepancies noted would be reported, tracked, monitored, and resolved. Your analysis noted that this more systematic examination of vouchers would ensure that FEA staff review over 53 percent of total monthly contract payment dollars, as compared to the 6.5 percent of the total dollars that are currently examined through the judgmental sampling of manually entitled vouchers only. In turn, DFAS expects that this heightened review will detect and correct more potential overpayments and result in savings to the government. The analysis prepared by a contractor for DFAS concludes that the 6.5 FEA full-time equivalents can perform 1,000 manually entitled voucher examinations per month if the maximum threshold for statistically sampled vouchers was raised from $2,500 to $500,000. Thus, it appears to us that the resources available are driving the proposed threshold and sample size rather than the consideration of other key factors, such as an appropriate risk level and acceptable error rate that would provide reasonable assurance that the government's interest is protected. Our Assessment of the Proposed Alternative Sampling Plan: Your proposed alternative sampling plan, which includes waiving the $2,500 maximum for statistical sampling of voucher prepayment examination for contract payments, does not justify the proposed savings or provide assurance that the government's interest will be adequately protected. Further, DFAS certifying officers risk personal liability for improper payments should they rely upon the prepayment examination process under the proposed alternative sampling plan. GAO's Policy and Procedures Manual sets out a relatively straightforward formula for agencies to use in deciding whether to adopt sampling in the certification of payment vouchers under 31 U.S.C. § 3521(b). The cost of examining all vouchers should be compared with the total costs of sampling (i.e., the cost of examining the sample plus the amount of projected losses from undetected errors in vouchers not examined).[Footnote 10] Where the costs of reviewing all vouchers exceed the costs of sampling plus estimated losses, sampling offers savings. Savings are achieved when the combined costs of (1) examining the sample and (2) projected losses due to undetected errors on invoices not examined are less than the cost of examining all invoices. Through analysis, the sampling plan must identify a tolerable error rate,[Footnote 11] the number of invoices to select for examination, and the selection method. Your request describes approximately $4.2 million in projected savings per year as a result of the alternative sampling plan. Based on our review of your proposed sampling plan, we question this projection. Specifically, adequate evaluation of sample results involves defensible projections to the entire population using statistically valid samples. The sample results in your plan cannot be projected to the population because it is not clear how the monthly sample sizes were determined or how the samples were picked from the monthly population. The DFAS analysis is centered on 10 monthly selected samples of manually entitled vouchers, but does not document how the sample sizes, ranging from 985 vouchers selected in July 2001 to 1,894 vouchers selected in April 2001, were determined relative to the total monthly population of manually entitled vouchers. In addition, the methodology used to record errors in the sample is unclear in the analysis. The analysis indicates that the error rate of 13.4 percent is a result of 2,019 "primary" errors noted in the 15,021 vouchers reviewed. A primary error is defined in the analysis as "the first error on a reviewed invoice." However, an invoice may have more than one error. Because total errors are not known or identified, the results cannot be viewed as representative and used to project potential losses to the remaining vouchers in the population. Hence the measured savings calculated as a result of FEA's prospective ability to detect and correct potential contract payment errors under the proposed plan are unjustified. We have additional concerns about whether this plan adequately protects the government's interest, based on the Department of Defense's (DOD) long-standing problems with accurately accounting for and reporting on disbursements. We have previously reported[Footnote 12] on long- standing problems in MOCAS contract payments. For example, recent DFAS data showed that almost $1 of every $4 in contract payment transactions was for adjustments to previously recorded payments--$49 billion of adjustments out of $198 billion in transactions. DOD had been developing a new system, the Defense Payment Procurement System (DPPS), which was designed in part to resolve DOD's long-standing disbursements problems and streamline contract payment processes. However, as we previously reported,[Footnote 13] DOD terminated DPPS in December 2002 after 7 years of development due to poor program performance and increasing life cycle costs. Consequently, DFAS will be relying on MOCAS, which was implemented in 1968, and other legacy systems to meet its requirements for the foreseeable future.[Footnote 14] In addition, DOD contracts processed by DFAS contain numerous fund citations and complex payment allocation terms. These factors, along with contract modifications, changes in payment allocation terms, and nonstandardized contracts and payment terms contribute to the fact that a large percentage of payment vouchers will need to be manually entitled for the foreseeable future. Based on our prior findings, this area has been very susceptible to errors. One of the requirements imposed on the certifying officer is to ensure (or have in place a system that can offer a reasonable degree of assurance of) the "legality of a proposed payment under the appropriation or fund involved."[Footnote 15] A certifying officer's legal liability is strict and arises automatically at the time of an illegal or improper payment.[Footnote 16] The Comptroller General has authority to relieve a certifying officer from liability when we find that the certification was based on official records and the official did not know, and by reasonable diligence and inquiry could not have discovered, the correct information.[Footnote 17] The Federal Managers' Financial Integrity Act of 1982 (FMFIA) requires agency heads to periodically evaluate their systems of internal control using the guidance issued by the Office of Management and Budget (OMB).[Footnote 18] Title VII also provides that FMFIA reviews should be designed, carried out, and reported in ways that show whether the voucher processing and disbursing system and controls are reliable. To the extent practicable, the needs of certifying and approving officials should be considered in advance of an FMFIA review, and the review results should be made available or communicated to them in reports tailored to their special interests. The FMFIA review is a good starting place for assessing risk in the contract pay certification and disbursing process, which, used in conjunction with an objective and scientifically developed approach encompassing the criteria[Footnote 19] referred to above, can provide the guidance for alternative plans that we could evaluate in the future. Conclusion: Title VII allows for a deviation from the $2,500 limitation for using statistical sampling to certify vouchers to the extent that there are resultant savings while adequately protecting the government's interest. Title VII does not distinguish between certification of automated or manual entitlements, but provides for statistical sampling of vouchers less than $2,500 as a technique in large-volume, highly automated voucher processing operations. Any alternatives to 100 percent prepayment examination must be part of a total system that includes well-defined responsibilities, organizational structures, and an adequate set of internal controls. Until a sound statistical sampling plan with justifiable savings is presented, we cannot grant a waiver of the $2,500 maximum for statistical sampling of contract payments or approve the alternative voucher examination plan submitted. The proposed alternative sampling plan does not justify savings or adequately protect the government's interest. In addition, good faith reliance on this particular plan would not relieve a disbursing or certifying official of liability for an improper payment or certification of a voucher not audited. The contents of this report were discussed with Bill Felice of your staff. We hope our comments are helpful. If you have any questions, or would like to discuss these matters further, please contact me at (202) 512-6906 or Mary Arnold Mohiyuddin, Assistant Director, at (202) 512- 3087. This report will be available at no charge on GAO's Web site at http://www.gao.gov. Sincerely yours, McCoy Williams: Director: Financial Management and Assurance: Signed by McCoy Williams: (190060): FOOTNOTES [1] Title VII of GAO's Policy and Procedures Manual for Guidance of Federal Agencies. The manual is the official medium through which the Comptroller General promulgates (1) accounting principles, standards, and related requirements and material for the guidance of federal agencies in the development of their accounting systems and internal auditing programs, (2) uniform procedures for use by federal agencies, and (3) regulations governing GAO's relationships with other federal agencies and with private concerns doing business with the government. 31 U.S.C. § 3521(b) states that heads of agencies may prescribe statistical sampling procedures to audit vouchers when they decide that economies will result. The term audit as used in 31 U.S.C. § 3521 (b) refers to the examination of disbursement vouchers prior to their certification for payment. [2] 31 U.S.C. § 3528 (a)(4). [3] 31 U.S.C. § 3325 (a)(2). [4] 31 U.S.C. § 3528 (b). [5] The most significant factors on which confidence in the system and controls should be based are (1) a well-defined organization structure and flow of work, with appropriate separation of responsibilities and clearly written policies and procedures governing the examination, approval, and certification of disbursement vouchers, (2) effective application of available technology and concepts to achieve efficient and effective voucher processing, and (3) review of the voucher processing procedures and controls in sufficient scope, depth, and frequency to provide reasonable assurance that key processing procedures and controls are working and reliable. [6] GAO's Policy and Procedures Manual for Guidance of Federal Agencies, Title VII, "Fiscal Procedures," ch. 7, sec. 7.4E. [7] 31 U.S.C. § 3521 (c). [8] The entitlement process, according to DFAS, refers to the preparation of the payment package, including the matching of an invoice, receiving report, and contract. Entitlement precedes prepayment review procedures. [9] A primary error is defined in the business case analysis as "the first error on an invoice that has been reviewed." An invoice may have more than one error. [10] GAO's Policy and Procedures Manual for Guidance of Federal Agencies, Title VII, app. III. [11] Tolerable error rate is the point at or below which savings occur. [12] U.S. General Accounting Office, Canceled DOD Appropriations: $615 Million of Illegal or Otherwise Improper Adjustments, GAO-01-697 (Washington, D.C.: July 26, 2001). [13] U.S. General Accounting Office, DOD Business Systems Modernization: Continued Investment in Key Accounting Systems Needs to Be Justified, GAO-03-465 (Washington, D.C.: Mar. 28, 2003). [14] U.S. General Accounting Office, DOD Financial Management: Integrated Approach, Accountability, Transparency, and Incentives Are Keys to Effective Reform, GAO-02-573T (Washington, D.C.: Mar. 20, 2002). [15] 31 U.S.C. § 3528(a)(3). [16] 54 Comp. Gen. 112, 114 (1974). [17] 31 U.S.C. § 3528 (b)(1). [18] 31 U.S.C. § 3512 (d). [19] The criteria include (1) defining the universe of vouchers, (2) determining the size of the sample needed, and (2) analyzing the results of the sample using the appropriate statistical procedure.