The first step in the CLR calculation is to calculate expected failed assets. This calculation begins with a list of institutions that are at heightened risk of failure, Confidential which the FRC measures using the institutions’ composite CAMELS ratings.⁴ For institutions with CAMELS ratings 4-5 (the lowest ratings), the FRC calculates 2-year historical failure rates for institutions with similar ratings and capital adequacy (described below). For institutions whose failure is imminent,⁵ called the“100-percent failure” group, the formula uses a 100-percent failure probability. The FRC multiplies these failure probabilities by the institutions’ assets to yield expected failed assets.

The second step in the CLR calculation is to translate expected failed assets into the FDIC’s expected losses. The FRC multiplies the expected failed assets of each institution by an estimated loss rate on those assets, where loss rates are derived from historical experience, 1987 through the present, for institutions of similar size.⁶

The final step in the CLR calculations is to alter the result of the formula according to the judgment of the FRC. The FRC may adjust the failure probabilities used in Step 1, the loss rates used in Step 2, or the individual reserve for any given institution. In determining whether to deviate from the CLR formula, the FRC generally considers the current economic climate, input from the DSC on likely failure probabilities, an alternative loss-rate model, and loss estimates prepared by the DRR based on asset valuation reviews or cost-test analyses.

Performance of the CLR

The ideal CLR would be accurate, robust in reflecting changes in economic and financial conditions, and based on a methodology that was transparent to all stakeholders. Accuracy is an objective criterion that can be measured arithmetically; e.g., by calculating the mean squared error between the CLR and the FDIC’s actual losses over time.⁷ Robustness refers to the ability of the methodology to respond to changing future conditions, rather than merely explaining what has happened before. Transparency is a function of both the method itself and how that method is applied. Simple methods applied without exception will generally be more transparent than complex methods with wide latitude for exceptions. The CLR can be improved on each dimension: accuracy, robustness, and transparency.

1. Each of the three steps in the CLR methodology (Exhibit 1-1) introduces errors that decrease the accuracy of the CLR.

In Step 1, the FDIC does not reserve for “unanticipated” failures, i.e., for failures of institutions with CAMELS ratings of 1-3, so any failure by such an institution leads to inaccuracy in the CLR. In 1990-2000, 18 percent of the FDIC’s losses from failed institutions were unanticipated, and the average annual loss from such failures was $238 million. Reserving nothing for these failures tends to make the CLR underestimate actual losses in most years, although other types of errors described below may have offsetting effects.⁸

There are also errors in estimated failure probabilities for those institutions that are included in the calculations. The FRC forms estimates of failure probabilities for each of four risk groups based on CAMELS ratings and capital adequacy. The institutions are first divided into those with CAMELS ratings 4 or 5. These two groups are further divided into those projected to have capital above or below 2 percent of assets in 12 months’ time.⁹

The FRC’s estimated failure probability for each of these four groups is the average annual failure rate for institutions in that group over the prior two years, weighted by each institution’s assets. This methodology is subject to two types of inaccuracy: differences in failure probabilities between institutions within each risk group and changes in failure probabilities over time.

Institutions within each risk group may have different failure probabilities. For example, in the 1990-2000 interval, the annual failure rate for CAMELS 4 institutions with less than 2 percent capital (the “4-minus” group) was 0.051 percent, but within that group, the failure rate for the largest half of institutions, by assets, was 0.090 percent, while that for the smallest half was 0.046 percent – a statistically significant difference. If these historical failure rates were accurate, rather than being random and anomalous, then the CLR method introduced inaccuracies by assuming the average failure rate for both halves of the 4-minus group. Specifically, 0.051 percent was too small a failure probability for the largest institutions and too large a failure probability for the smallest institutions. While such within-group differences no doubt exist and contribute to errors in the current CLR methodology, the magnitudes of such errors are difficult to quantify.

The second potential source of inaccuracy in the FRC’s historical failure-probability estimates stems from changes in failure rates over time. Such changes can result from pure randomness (e.g., failure from fraud or poor corporate governance) or from changes in the characteristics of institutions within each risk group (e.g., capital degradation of every institution in the 4-minus group due to a change in the economic climate). These two effects combine to cause failure rates to differ substantially from year to year. For example, the failure rate of the 4-minus group was 0.052 percent in 1990-1994, but it fell to 0.045 percent in 1995-2000. Accordingly, using historical failure rates in 1995 would likely have led to overestimates of failures.

In general, any historical moving average, or “look-back period,” for failure rates will underestimate failure probabilities during a period of unusually high failure rates, since the look-back will estimate failure probabilities based on earlier experience, and similarly overestimate failure probabilities after the temporarily high failure rates have subsided, since the look-back will incorporate high failure rates that will have since passed.

There are additional costs to accuracy in Step 2 of the CLR calculation, when converting estimated failed assets into estimated losses. As described above, FRC uses losses from 1987 onwards to form estimates of loss rates for institutions based on their size, by assets, for five different size ranges. As with failure rates, there are two primary sources of error in these historical estimates of loss rates: differences in loss rates within any given size group and changes in loss rates over time.

Obviously, not all institutions of a given size will cause the same loss if they fail. For example, the asset composition of an institution will influence the FDIC’s loss rates, with almost no losses realized on liquid assets like securities and substantial losses on less-liquid assets like commercial loans. This means that institutions of the same size often have predictably different loss rates, whereas the current CLR methodology assumes that those loss rates are the same.

The other source of error in loss rates stems from changes in loss rates over time. Loss rates depend on a number of factors that are likely to change from one year to the next, such as prevailing market conditions, the prevailing cause of failures, and the credit quality of the assets being liquated. For example, the average loss rate for institutions with assets $100 million to $500 million was 17 percent for 1990-1994, but it was 27 percent for 1995-2000. Since the CLR uses average loss rates from 1987 to the present (23 percent in this size band), the CLR’s loss estimates in the late 1990s incorporated the lower loss rates from the early 1990s, leading to reserves that were erroneously low for institutions in this size group.

The desire to have accurate and up-to-date loss rates must be traded off against the statistical challenges of estimating loss rates based on very few failures. In the example above, if the loss rate had been estimated with more-recent failures rather than those in the early 1990s, then the loss-rate estimate would have been based on just a handful of failures, so it would have been vulnerable to unusual failures that did not reflect prevailing conditions.

The final source of inaccuracy in the CLR comes in Step 3, when the FRC exercises its judgment in determining whether to deviate from the historical failure- and lossrate estimates. In practice, this has led to wide departures from the historical estimates for failure rates (Exhibit 1-2). Loss rates are rarely overridden by the FRC.

D
* Average failure rate over the 5 .buckets. (e.g., CAMELS 4 with more than 2% capital), weighted by the number of institutions in each bucket in the given quarter

In all but two quarters from 1997 to 2001, the 2-year historical failure rate was closer to the actual failure rate than was the failure probability used by the FRC. As such, the subjectivity exercised by the FRC in setting failure probabilities has in general decreased the accuracy of the CLR.

2. Each of the three steps in the CLR methodology (Exhibit 1-1) affects the robustness of the CLR.

In Step 1 of the CLR methodology, the groupings for failure rates are based on CAMELS ratings and capital forecasts. Both of these criteria are fairly robust to changes in conditions. CAMELS ratings represent the best and most current knowledge of supervisors and are flexible enough to reflect new and emerging concerns. For example, if a new and risky lending practice emerged in the banking industry, then supervisors can incorporate the impact of that practice in the CAMELS ratings, especially as it affects asset quality and capital.

Likewise, capital is likely to remain a fundamental component of an institution’s soundness, irrespective of innovations in the industry, so it is a robust measure of probability of failure, even on a lagged accounting basis. In this area, the only lack of robustness may come from how capital is forecast, since the dynamics of balance sheets can change with industry practices, and the FDIC’s Pro Forma Model may miss such changes.

In Step 2, the FRC’s current loss-rate methodology is not robust, because the typical asset profile of the banking industry varies over time. For example, the 1990s saw rapid increases in the prevalence of syndicated loans, subprime lending, and credit derivatives. Since different assets have different loss rates, a robust CLR would adapt to introductions of new asset classes. The current CLR methodology does not, since it assumes that loss rates vary based only on the size of an institution, not on its asset composition.

Finally, in Step 3, the FRC may use its judgment to override the historical failure and loss rates used in the CLR calculations. Failure rates are overridden routinely, loss rates only infrequently. In theory this flexibility enhances the robustness of the CLR, since the FRC is composed of representatives from four divisions of the FDIC, with extensive experience and access to the most accurate and current information about possible losses from failures. As such, the FRC participants as a group are in a better position than anyone else to know whether the historical failure and loss rates may be inappropriate for the coming year. However, as described above, the robustness derived from this discretion comes at the expense of accuracy, since subjective deviations from historical failure and loss rates have usually enlarged CLR errors.

3. The formulaic part of the CLR methodology (Steps 1 and 2) is transparent, but subjective deviations from the formula (Step 3) are not.

The most transparent methodology would be simple and applied uniformly. The FDIC’s current methodology is quite simple: it reserves for failing institutions and those with CAMELS ratings 4 and 5, based on the failure rates for the past 2 years and average loss rates for similar-sized institutions since 1987, unless the FRC judges that such historical estimates would be inappropriate going forward.

On the other hand, the discretion that FRC exercises to override historical failure and loss rates detracts from the transparency of the CLR. The FRC can deviate from the historical estimates for a variety of reasons.¹⁰ This latitude leaves room for outsiders to speculate about whether the CLR has been modified or adjusted for other than purely reporting purposes.

Specifics of Recommendation 1.1 (Refining the CLR methodology)

The FRC can improve the accuracy, robustness, and transparency of its CLR methodology by adopting the following eight recommendations. The first three relate to bounding the FRC’s subjectivity in estimating failure probabilities, while the remaining four explain how the FRC should use more information from institutions’ balance sheets in estimating loss rates. The FRC and DIR should implement all of the recommendations during the next 3 months.

1.1.a. The FRC should develop explicit guidelines about when it will deviate from historical failure rates. The current CLR methodology uses a 2-year look-back for estimating failure probabilities. This period was chosen to minimize the meansquared error of the CLR relative to other possible look-back windows, enhancing the overall accuracy of the CLR. Nevertheless, the FRC has often used its broad discretion to depart from the 2-year failure rates and substitute its own estimates of expected failure rates not strictly grounded in historical experience. Although wellintended, these deviations typically have increased the error in the CLR. Developing explicit guidelines governing when to deviate from the 2-year lookback will help the FRC adhere more often to historical failure rates and guide DIR’s ongoing research into anticipating future failure rates. Preliminary analysis suggests that historical failure rates may be too low when high-yield bond default rates have recently increased, vacancy rates in commercial real estate are increasing, or subprime lending makes up a larger-than-average fraction of assets for institutions on the reserve list. Making this research and FRC’s deviation guidelines public would enhance the transparency of the CLR while maintaining its robustness.

1.1.b. When estimating failure probabilities, the FRC should constrain itself to a 90-percent confidence interval of the 2-year historical average.¹¹ This recommendation would allow the FRC to apply judgment while limiting its scope, and thereby likely increase both the accuracy and robustness of the CLR. As Exhibit 1-3 shows, restricting the FRC’s discretion to the 90 percent confidence interval would have improved the accuracy of the FRC’s failure-probability estimates in all but one quarter since 1997. The net impact is depicted in Exhibit 1-4, which shows the improved accuracy (approximately 23 percent over the 5-year period shown) that bounding subjectivity confers to the CLR.¹²

D
* Average failure rate over the 5 .buckets. (e.g., CAMELS 4 with more than 2% capital), weighted by the number of institutions in each bucket in the given quarter

Note: Actual failure rate and 2-year moving average failure rate are not weighted for institution size.

Exhibit 1-4

NET IMPROVEMENT TO THE CLR

D
 * Lighter shading starting in 01Q4 indicates probable improvements
** Before any subjective override the FRC may make to account for new information about specific institutions

1.1.c. The FRC should continue not to reserve for unanticipated failures. Although institutions with composite CAMELS ratings 1-3 represent a substantial fraction of failures, losses from these failures do not appear to be “probable and reasonably estimable” in any given year, since the annual loss is usually either zero or a large amount. Reserving based on an historical moving average of past unanticipated losses, regardless of the length of the look-back, would decrease the accuracy of the CLR¹³ because it would result in under-reserving at the time of an unanticipated loss and over-reserving after the loss has occurred (Exhibit 1-5). However, this topic warrants further study, especially if the FDIC begins to experience more than a couple of failures of institutions rated 1-3 in each year.

D
* One-year moving average of actual quarterly losses

1.1.d. The FRC should use the FDIC’s Research Model for loss-rate estimates. FRC’s current CLR methodology takes no explicit account of liability structure in estimating loss rates. Any loss in asset values is considered a loss to the FDIC. In reality, however, the shareholders and creditors of the failed institution share in these losses, in order of their seniority. Because the Research Model specifically accounts for institutional liability structures, it is a better model of the FDIC’s expected losses given failure.

1.1.e. DIR should update the Research Model with more recent data. In current form, the Research Model does not offer a material improvement in accuracy over the existing approach to estimating loss rates, but its accuracy can be improved. The research model was last updated based on data for 1990-97. More recent data almost certainly better reflect current conditions, so additional years of data should be included. Furthermore, DIR should investigate whether loss rates in the early 1990s are still applicable, and if not, DIR should drop those earlier observations from its loss-rate estimates.

1.1.f. DIR should expand the Research Model to incorporate institution size. The current CLR methodology recognizes a strong inverse relationship between institution size and loss rate given failure, assigning a loss rate to the smallest institutions that is three times the rate of very large institutions (24 percent versus 8 percent). Asset size is likely to retain predictive power even when controlling for asset composition, because larger institutions tend to have better internal controls, and because the FDIC often enjoys economies of scale in disposing of larger institutions. It follows that the overall predictive power of the asset-compositionbased Research Model is likely to be enhanced with the addition of a variable representing institution size, and the FRC should explore this possibility. For reasons of sample size, it may be advisable to restrict the analysis to two size categories, large and small.

1.1.g. DIR should expand the Research Model to account for dispositions other than liquidation. The FDIC often is able to dispose of an institution, or part of it, via a Purchase and Assumption (P&A) transaction that captures some of the franchise value and avoids some liquidation costs. By ignoring these kinds of transactions, the current Research Model is biased toward overstating actual losses from failures. Although issues of availability and scope of historical data present analytical obstacles, DIR should thoroughly study the feasibility of expanding the Research Model to account for dispositions other than liquidation.

⁴ A CAMELS rating is a supervisory estimate of the safety and soundness of a depository institution. Each letter of CAMELS corresponds to a dimension considered by the supervisor. C is for capital adequacy; A is for asset quality; M is for management capabilities; E is for earnings quality and quantity; L is for liquidity; and S is for the sensitivity to market risk. Each dimension is scored one to five, with one being the best rating. The supervisor determines the overall, or composite, rating based on the six component ratings, and composite ratings also range from one to five; 1-rated institutions are deemed to be the safest, and they have in fact historically been much less likely to fail.

⁵ This determination is based on either the Division of Resolution and Receiverships’ (DRR) scheduled closing date for the institution, the classification of the institution as “critically undercapitalized,” or the Division of Supervision and Consumer Protection (DSC) identification of the institution as an imminent failure.

⁶ For each of five size groups, the FRC estimates the loss rate as the dollar amount of the FDIC’s losses from failures in that size group divided by the sum of all the assets of institutions in that size group. This amounts to an average loss rate, weighted by each institution’s size by assets.

⁷ Mean squared error (MSE) is the predominant measure of goodness of fit in statistics. When the MSE of a set of estimates is low, then the estimates are accurate, and an MSE of zero corresponds to a perfect fit. To calculate MSE, square the difference (error) between each estimate and its corresponding historical value, and then average these squares. By squaring the errors in this way, all errors are stated as positive numbers (the square of a negative is a positive), and large errors are penalized disproportionately (4²=16 is more than twice as large as 2²=4).

⁸ As discussed later in this report, no analytically sound solution exists in Horizon 1 for correcting the inaccuracy introduced by not reserving for CAMELS 1, 2, and 3 institutions.

⁹ DIR forecasts an institution’s capital using the FDIC’s Pro Forma Model (Pro Forma), which simulates the future financial condition of the institution based on its current financial condition. For example, if the institution’s current income is not sufficient to cover its expenses, then it must be depleting capital to make up the difference, so Pro Forma forecasts that its capital will decline.

¹⁰ The FRC must document and explain any failure rate deviations in a written memorandum to the FDIC’s CFO and the Division of Finance’s Deputy Director responsible for the FDIC’s financial statements.

¹¹ Within each risk bucket (e.g., 4-minus), there were a given number of failures (X) and a given number of institutions (Y) over the prior 2 years. The historical failure rate (F) is the ratio of the two (X/Y). This ratio is a binomial random variable with well-understood statistical properties, making it possible to construct a confidence interval for the true 2-year failure rate. Specifically, Y is almost always greater than 30, so the distribution of the true 2-year failure rate can be closely approximated by a normal random variable with mean F and variance S=[(F)(1-F)/Y]½ . The 90-percent confidence interval for the true failure rate is then F ± (1.96)(S).

¹² We recommend using unweighted averages in calculating historical failure rates, and we have done so in our analysis. The current FRC methodology determines historical failure rates using an asset-weighted average.

¹³ For example, the mean squared error would be higher.

<< PREVIOUS | CONTENTS | NEXT >>