Cost-Benefit Analysis of Reducing UI Payment Errors

As a part of the President's Management Agenda for DOL, ETA is responsible for preparing a cost-benefit Analysis of reducing UI benefit overpayment errors. This paper begins to respond to that requirement and establishes a methodological framework for further analyses. The text boxes contain notes on analyses to be performed and analytical approaches.

The analysis relies heavily on Benefit Accuracy Measurement (BAM) data for estimates of the overall extent of UI overpayment errors, their causes (i.e., where in the benefit payment process they occur), and why they occur.

Where the Errors Occur

Table 1 shows where UI overpayments were found during FY 2001 (this pattern does not change much from one year to the next). According to BAM, approximately $2.3 billion were overpaid in the year altogether. Seventy percent of errors occur after the initial eligibility determination stage, a fifth at the point of determining separation eligibility, and only 8 percent for monetary reasons. In actuality, the percentage due to weekly eligibility violations is even greater than the 70% shown, because BAM is known to underestimate Benefit Year Earnings violations (underreported earnings during the week, usually claimants who have gone back to work but continue to claim benefits.) Monetary determination errors rarely involve total eligibility; many UI cases involve small monetary mistakes, and the number of positive and negative errors is roughly equal.

	% BenPd	Estimated $ Amount
Table 1
Estimated UI Overpayments by Cause, FY 2001
Monetary Determinations	0.7%	$184,045,458

Separation Determinations
Discharge	0.6%	$159,258,746
Voluntary Quit	1.2%	$325,118,844
Other Separation Issues	0.0%	$11,978,258
All Separation Errors	1.9%	$496,355,848

Weekly Eligibility Errors
Benefit Year Earnings	2.4%	$622,943,527
Work Search Issues	1.3%	$349,807,344
ES Registration	0.7%	$190,510,525
Able+Available	0.6%	$165,050,121
Other Eligibility	0.3%	$85,819,302
Other Causes	0.3%	$82,973,031
Social Security	0.1%	$32,837,431
Illegal Alien	0.1%	$28,162,280
Severance/Vacation	0.1%	$24,285,328
Dependents Issues	0.1%	$22,738,748
All Weekly Eligibility Errors	6.1%	$1,605,127,637
Total—All Causes	8.7%	$2,285,528,943

Calculate average $ amount per error by cause from BAM

Nonmonetary Errors in Context

This section relies on analysis done of 1998 UI activity, combining UI required reports data on UI separation and nonseparation adjudications made and claims denied, weeks paid, separation determinations made, and BAM data on case errors involving total denial of benefits, and why errors occurred (whether the agency knew or could have known about the error.) Its purpose is to show separation and weekly eligibility errors in context of overall claims activity at that level of determination. In part, this analysis is driven by the belief that a critical distinction involves the degree of agency knowledge of errors before corrective action is considered. It makes a major difference whether the agency had information sufficient to detect a potential issue or to make a correct determination and failed to use it or used it incorrectly, or whether the agency's procedures simply could not detect the issue.

The Pattern of Separation Determinations and Errors.

The first chart shows the pattern of UI separation determinations. In 1998, the UI system made about 15.7 million separation determinations (this is calculated as the sum of new plus additional initial claims from the 5159 report, plus separation denials from the 207 report). Issues were detected on about 3.4 million (22%), resulting in 1.9 million denials. State staff raised no issue about 12.3 million, or 78%, in effect automatically allowing the claim. Inferring from BAM results on percentages of dollars overpaid because of separation issues, 12.0 million of these were decided correctly, but about 250,000 decisions involved missed issues. This analysis suggests that agencies are quite effective at detecting and denying potentially deniable initial claims where separation issues are concerned. Out of 2.1 million claims that should have been denied for separation reasons, they actually denied nearly 1.9 million, or 87%.

The second chart illustrates just the errors. Using BAM data in which errors have been coded for "Prior Agency Action", three quarters of the errors occurred because the agency, using existing procedures, could not detect the issue. In twenty percent of the cases, the agency had sufficient information to detect the issue but failed to follow through to identify it, or failed to follow procedures that would have identified it. BAM estimates that 5% of errors occurred because the agency made the wrong determination; these are assumed to have occurred on adjudicated issues. (These figures ignore less than 5% of separation errors that the agency was in process of resolving when BAM examined the case).

The Pattern of Non-Separation Determinations and Errors.

Nonseparation decisions were analyzed in the same way as separation issues. Total nonsep decisions were computed as the sum of weeks paid and nonsep denials. In 1998, States made a total of 103.7 million nonsep decisions. The great preponderance was correct decisions to pay individual weeks claimed, made with no issues raised. About 95.7 million, or 92%, were correct with no issue. Agency staff raised and adjudicated about 4.3 million issues (4% of all decisions, versus 22% for separation decisions), resulting in 2.4 million denials. They also paid about 3.7 million weeks that they should have denied.

Although there were about 15 times as many nonsep errors as separation errors, the patterns of both are almost identical. Again, ignoring errors the agencies were in the process of resolving when BAM reviewed the cases, 79% of the dollars overpaid occurred because agencies were unable to detect the issue, 16% because they failed to detect a detectable issue, and 5% because they drew the wrong conclusion from sufficient data.

Further work: replicate this analysis for FY 2000 and FY 2001 (a 2-year average might be more reliable than one year) and also extend it to monetary determinations. This will require a run of case error rates by monetaries, seps, and nonseps for every State + US totals. In 1998, the analysis only included the counts for cases in which the full WBA was disallowed. For FY 2000 and 2001, the analysis should also include partial overpayments.

Taking Steps to Correct Errors: Which Causes First?

BAM data from Table 1 indicate that three times as many dollars are overpaid due to errors at the weekly eligibility level of benefit payment administration as occur at the level of judging separation reasons, $1.6 billion vs. $0.5 billion, and nearly nine times as much as at the monetary level ($1.6 billion vs. $0.2 billion). More critical in making decisions about corrective action, however, is the average error per case:

	Average $ Per Error
Monetary errors	$33
Separation Errors	$201
Nonseparation/weekly Eligibility Errors	$135

Analysis: Run averages for full WBA cases and partial. It would probably be best to do these for the causes shown in Table 1, and FY 2000 and 2001

This suggests that states will obtain a larger "bang for the buck", other things being equal, by addressing first separation errors, then weekly eligibility errors, and lastly monetary errors. However, operational realities may dictate a different conceptual approach-initial claims vs. continued claims, for example.

Prevention, Recovery and Detection

The charts above indicate that the bulk of nonmonetary errors occur because state procedures cannot detect the eligibility issue (the same is true for monetary determination errors: in FY 2001, 94% were judged undetectable). Thus, in FY 2001 BAM estimated that agencies' inability to detect that an issue existed was at the root of 73% of all dollars overpaid.

Whether or not a State UI agency can detect potential eligibility violations is critical in determining the prevention strategy. It is, naturally, critical also in determining whether actual overpayments can be recovered.

Preventing Overpayments From Occurring.

Using Existing Information Properly. In any given year, BAM concludes that over 20% of overpayment errors occur despite the fact that agencies have detected, or should have detected, the potential error. That is, they fail to follow through to conclude eligibility issue exists, fail to follow procedures that would have detected the error, or make the correct eligibility decision from information they have. In FY 2001, such errors amounted to approximately $548 million, 24% of all dollars overpaid.

Analysis: Run FY 2000 and 2001 $ overpaid by Prior Agency Action and cause, as done in previous years, and note any changes. Depending on year-to-year variation, an average may be appropriate.
- Reducing errors of this type involves proper training and supervision to ensure that agency staff follow procedures and use all available information at hand to detect potential issues and draw the proper conclusions about claimant eligibility.
Detecting Potential Eligibility Issues. BAM indicates that about three quarters of dollars overpaid in any given year occur because agency payment procedures cannot pick up the issue. Generally speaking, claimants have a monopoly on the information critical to determining whether they are eligible. If they do not choose to reveal to the agency whether they made a work search contact, were able and available for work in a given week, or were working the week for which a claim was filed, etc., it is prohibitively expensive for the agency to find out.

Analysis: It appears we can estimate the likely benefits of contacting claimants, e.g., through Eligibility Reviews, and following up with employers, as follows. First, we want to look only at overpayments (OPs) ($ and cases) that were undetectable, run by the 12 or 14 major BAM cause codes. Cross-tab these by Prior Employer Action, to determine (esp. in case of sep OPs) where employers failed to respond fully or timely. Examine the nondetectable OPs by Point of Detection to determine what could be gained by following up with Employers (probably most useful in the case of seps and maybe monetaries, depending on the costs of doing so), and also calling in claimants through the ERP or similar program. Once the selection is programmed, run it for FY 2000 and 2001, and average the results if warranted. The BAM 1990 phone pilot costs are available as a guide to cost.

Preventing errors of this kind involves adopting new procedures for error detection
Unfortunately, State processes seem to be moving in the direction of detecting fewer, not more, errors. A comparison of undetectable errors in 2000 with 1989 suggests that the shift to remote claims taking has increased the proportion of errors considered undetectable. For all causes together, BAM judged that 68% of errors were undetectable in 1989 versus 72% in 2000. (When Work Search errors, which declined as a share of all errors over the period, are excluded, the proportion of undetectable errors rises from 61% to 70%). The share went up for 11 of the 12 usual cause codes; only Base Period Earnings errors were more detectable in 2000 than 1989.)
If BAM costs are used as the standard for detection costs, here are the average times for obtaining BAM information by telephone from the 1991 BAM Telephone Pilot Study:

Claimant interview, 35 minutes;
Base Period wages, 24 minutes;
Separation investigation with base period employers, 5 minutes;
Other investigations with BP employers, 10 minutes;
Work Search, 10 minutes;
Third party contacts, 5 minutes;
Benefit Year Earnings investigation, less than 5 minutes;
Locating respondents, 35 minutes.

Analysis: Examine State performance to determine which States have higher detection ratios; consider using this information for a benchmarking study. Do a regression study with such factors as frequency of ERPs. What does RJM show about state time input to pay various claims or make determinations at the monetary, sep, nonsep levels?

Detection and Recovery of Overpayments

BAM considers about three quarters of dollars overpaid to be undetectable by the agencies during their usual payment administration processes, and thus impossible for the agency to prevent without changes in those processes. Certain errors that are rarely detectable in time to prevent their occurrence, however, can be readily detected after the fact, and can potentially be recovered. Perhaps the best example is Benefit Year Earnings violations. In 2000, BAM estimated that about 94% of BYE overpayments were undetectable by continued claims payment processes; however, with various cross-matches, primarily the benefit-wage crossmatch, agencies should be able to detect nearly all these errors 5 to 6 months after occurrence. All agencies maintain Benefit Payment Control (BPC) operations to detect, establish, and recover overpayments. In many cases, it may be more cost-effective to attempt to recover certain overpayments than to attempt to prevent their occurrence. At least, that's what States seem to assume.

An examination of BAM and Benefit Payment reveals the limitations on this process:

Legal Ability to Recover. For various reasons, not all overpayments are recoverable. In 2001, BAM estimated that approximately $480 million out of $2.3 billion, or 21%, of the overpayments could not be recovered due to finality, potential hardship, and other causes. It estimates that $1.8 billion in overpayments were potentially recoverable (fraud plus non-fraud).
Ability to Detect. Many overpayments are difficult to detect even after the fact. BPC operations are most effective at detecting certain categories of overpayments, especially earnings while claiming and separation violations. It appears that in FY 2001 they could most readily detect about $1.2 billion, or 67%, of the $1.8 billion, in potentially recoverable overpayments.
Ability to Establish. In FY 2001, States reported establishing $636 million, or 51% of the "most detectable" categories of overpayments, for recovery. (The true ratio is somewhat higher because most overpayments established in FY 2001 actually occurred at least 6 months earlier.) A State's ability to establish an overpayment requires fulfilling two of the conditions it must fulfill to deny a claim: detecting the potential violation and conducting an investigation to determine that the overpayment is real. There is a third condition: cost-effectiveness. Establishing and recovering overpayments are expensive activities, and thus agencies are unwilling to establish many small overpayments they deem too small to be worth attempting to recover.
Ability to Collect. Establishing an overpayment is one thing; recovering it is another. On average, States usually manage to recover slightly over half of the overpayments they establish.

Analysis: RJM contans $ spent administering payments; $ spent on BPC. Relate to State differences in establishment ratios (using the new "most detectable recoverable OP" estimates for denominator) and recovery ratios (recoveries/establishments). Relate to methods used, e.g., as previously done in analyzing 227 data. Examine possible benefits of expanding BPC activities by looking at establishments, and the ratio of establishments to our "most detectable & recoverable" measure, and the recovery/establishment ratio. The first ratio will tell who's doing the best job, and some average, e.g, top 10, would give us an attainable ratio. Use RJM data to get a sense of how much effort goes into those high ratios, and how much is due to a higher ratio of establishments/BPC SY.