|
Reliability of Supervisory Information Accessed Through the Virtual Supervisory Information on the Net (ViSION) SystemSeptember 2008
|
|
This report presents the results of our audit of the reliability of supervisory information accessed through the ViSION system. ViSION is a mission-critical FDIC system1 that provides access to a broad range of information related to insured financial institutions in support of the Corporation’s insurance and supervision programs. The objective of the audit was to assess the reliability of key supervisory information accessed through the ViSION system. We conducted this performance audit in accordance with generally accepted government auditing standards. Appendix 1 of this report discusses our audit objective, scope, and methodology in detail. BACKGROUNDThe ViSION system is one of the most widely-used Web-based systems at the FDIC. During the first 6 months of 2008, the system recorded approximately 5.7 million pages viewed and served about 3,900 FDIC and outside agency users (primarily other federal and state regulatory agencies). The ViSION system’s primary users within the FDIC are executives, regional managers, case managers, review examiners, and field examiners in the Division of Supervision and Consumer Protection (DSC). DSC personnel use the system to perform supervisory-related functions, such as tracking applications, accessing examination information, and monitoring enforcement actions. Analysts in the Division of Insurance and Research (DIR) also rely on information in the ViSION system to perform insurance-related functions, such as analyzing trends in the banking industry and calculating deposit insurance assessment rates for financial institutions. |
Key Supervisory Information Accessed Through the ViSION System Key supervisory information accessed through the ViSION system includes: (1) financial institution examination ratings (examination ratings); (2) Bank Secrecy Act (BSA) of 1970 examination information (BSA examinations) reported to the Department of the Treasury; (3) safety and soundness Reports of Examination (ROE); and (4) ROE processing dates used to monitor examination frequency and determine deposit insurance assessments for financial institutions. Our audit focused on assessing the reliability of information in these four areas because of their criticality to the success of the FDIC’s insurance and supervision programs. A brief description of each area follows.
2
|
The FDIC has established a Data Stewardship Program4 to enable the Corporation to, among other things, ensure the usefulness, accuracy, timeliness, and accessibility of corporate data. Under the program, divisions and offices designate subject matter experts (SME) who are responsible for preserving the accuracy of data entered into application systems and databases. Within DSC, personnel in the Technology Supervision Branch serve as SMEs for the ViSION system. 3
|
Assessing the Reliability of Key Supervisory Information We used the Government Accountability Office’s (GAO) October 2002 publication entitled, Assessing the Reliability of Computer-Processed Data, as the overarching criteria for assessing the reliability of supervisory information accessed through the ViSION system. The publication states that computer-processed data are reliable when they are accurate (i.e., they reflect the data entered at the source or in the source documents) and complete (i.e., they contain all relevant data elements and records). Based on a random sample of 75 financial institutions for which the FDIC is the primary federal regulator, we verified key supervisory information accessed through the ViSION system to source documentation, such as hard copy safety and soundness ROEs. RESULTS OF AUDITSupervisory information accessed through the ViSION system pertaining to examination ratings, BSA examinations, safety and soundness ROEs, and ROE processing dates was not fully reliable for the 75 financial institutions that we sampled. Specifically, examination ratings and BSA examinations were generally reliable, with some exceptions. Safety and soundness ROEs were not reliable for 33 of the 75 institutions, and ROE processing dates were not reliable for 10 of the 75 institutions. Unreliable information accessed through the ViSION system can limit the efficiencies that the FDIC intended to achieve through automation such as accurate, timely, and consistent data used for off-site monitoring of financial institutions. In addition, because ROE processing dates are used in determining deposit insurance assessments, the reliability of those dates is critical to ensuring the integrity of premiums charged to insured financial institutions. Unreliable ROE processing dates resulted in 1 of 75 sampled institutions being significantly undercharged ($3,050, or about 10 percent) on one of its quarterly deposit insurance assessments. ASSESSMENT OF KEY SUPERVISORY INFORMATION ACCESSED THROUGH THE VISION SYSTEMAs reflected in Table 1 below, supervisory information accessed through the ViSION system pertaining to examination ratings, BSA examinations, safety and soundness ROEs, and ROE processing dates was not fully reliable for the 75 financial institutions that we sampled. Unreliable information accessed through the ViSION system can limit the efficiencies, such as accurate, timely, and consistent data used for off-site monitoring of financial institutions, that the FDIC intended to achieve through automation. In addition, because ROE processing dates are used in determining deposit insurance assessments, the reliability of those dates is critical to ensuring the integrity of premiums charged to insured financial institutions. 4
|
Table 1. Reliability of Key Supervisory Information for 75 Sampled Institutions
Examination Ratings DSC’s Risk Management Examination Manual states that examination ratings are used by regulators to evaluate the safety and soundness of financial institutions and to identify those institutions requiring special supervisory attention or concern. In addition, FDIC Circular 4700.1, Risk Related Premium System, dated June 7, 2007, states that maintaining accurate and complete examination ratings in the ViSION system is “extremely important” because the ratings are used in calculating deposit insurance assessments for financial institutions. Due to erroneous data entry, the ViSION system contained inaccurate component ratings for 2 of the 75 financial institutions that we sampled. We brought these inaccuracies to the attention of DSC officials during our audit, and the ratings were corrected in the ViSION system. The inaccurate ratings resulted in a slight undercharge (less than $15.00) for one institution on its 4th quarter 2007 deposit insurance assessment. BSA Examinations Under the terms of a Memorandum of Understanding between the Federal Banking Agencies (FBA)5 and the Treasury’s Financial Crimes Enforcement Network (FinCEN), the FDIC is required to report information to FinCEN on the BSA examinations the Corporation conducts or reviews. Information typically reported includes, for example, the number of BSA examinations conducted, the number and type of BSA violations identified, and the type of BSA enforcement actions taken. DSC Regional Director Memorandum 03-048, Bank Secrecy Act Examination Violations Codes, dated October 20, 2003, states that information in the ViSION system is used to fulfill the FDIC’s obligation to report BSA violations to FinCEN. The ViSION system did not contain all relevant BSA information for 2 of the 75 financial institutions that we sampled. 5
|
For one institution, the system did not contain a BSA violation cited in the safety and soundness ROE because DSC had not developed a violation code to track the specific type of violation cited.6 As a result, DSC did not include this violation in its BSA reporting to FinCEN. For the remaining institution, the ViSION system contained some, but not all, pertinent BSA information due to an oversight. Specifically, the BSA module in the ViSION system did not contain information regarding whether a BSA examination had been conducted or whether BSA violations had been identified for that institution. Safety and Soundness ROEs DSC Regional Director Memorandum 03-023, Integrity of Data Stored in the Interagency Examination Repository, dated July 1, 2003, emphasizes the importance of maintaining reliable ROEs in the IER to facilitate the off-site analysis of financial institutions. (As previously discussed, users of the ViSION system can access ROEs stored in the IER through a link in the system called the ROE module.) ROEs were not accessible through the ViSION system for 19 (25 percent) of the 75 financial institutions that we sampled. In addition, 14 (25 percent) of the 56 ROEs that were accessible through the ViSION system were draft versions of the final ROEs that did not reflect changes made during the supervisory review process.7 DSC officials informed us that they had identified data reliability concerns with ROEs stored in the IER prior to our audit and attributed these concerns to two principal factors:
6
|
DSC officials informed us that, when fully implemented, these control improvements will significantly increase the reliability of ROE information in the IER. ROE Processing Dates The DSC Risk Management Manual of Examination Policies states that the examination start date and examination completion date are used to monitor compliance with regulatory requirements concerning the length of time between examinations. Circular 4700.1 states that it is “extremely important” for the examination mail date in the ViSION system to be accurate and complete because the Risk Related Premium System (RRPS)8 uses this date to determine when deposit insurance assessment pricing changes become effective for financial institutions. The ViSION system contained unreliable ROE processing dates for 10 of the 75 financial institutions that we sampled. Specifically, the system contained inaccurate examination start dates for two institutions, an inaccurate examination completion date for one institution, and inaccurate or incomplete mail dates for eight institutions.9 Generally, these dates were off by a range of a few days to approximately 1 month. Unreliable ROE processing dates were principally caused by erroneous data entry. Unreliable examination start and completion dates did not negatively impact DSC’s examination schedules for the institutions we reviewed. However, unreliable examination mail dates affected the accuracy of deposit insurance assessments for three FDIC-insured financial institutions. One of the institutions was undercharged $3,050 (about 10 percent of the institution’s fourth quarter 2007 deposit insurance assessment). The monetary errors for the other two institutions were immaterial. Unreliable examination mail dates had no effect on the deposit insurance assessments of the remaining five institutions for two principal reasons: (1) the manner in which the FDIC calculated insurance assessments prior to the implementation of deposit insurance reform legislation differs from current practices and (2) examination ratings, which are a key factor in determining assessments, were substantially the same between the prior and current examinations for some of the institutions. See Appendix 2 for more detailed information regarding how examination mail dates can affect deposit insurance assessments for FDIC-insured financial institutions. 7
|
Strengthening the Reliability of Key Supervisory Information GAO’s November 1999 publication entitled, Standards for Internal Control in the Federal Government, identifies a number of internal control activities that organizations can consider implementing to promote accurate and complete computer-processed data. Such internal control activities include, for example, data edit checks, verifications, and reconciliations. According to the publication, organizations should design and implement internal control activities based on related costs and benefits. In this context, organizations may, based on an assessment of risk, determine that data are reliable even though they are not error free. Within the FDIC, the Division of Resolutions and Receiverships (DRR) took such an approach when it established a formal Data Quality Program in September 2005 to ensure “highly reliable and accurate data” within its priority IT systems.10 Under the program, critical data elements within DRR’s priority IT systems are considered reliable if they demonstrate an accuracy rate of 90 percent or better based on data quality testing. DSC has taken steps to promote the reliability of information accessed through the ViSION system. Such steps include designating SMEs for the ViSION system and periodically assessing the reliability of information accessed through the ViSION system during the division’s internal reviews. However, DSC can improve the reliability of supervisory information accessed through the ViSION system by conducting an assessment of such information to determine an acceptable data accuracy rate. Establishing a data accuracy rate based on an assessment of relevant risks, costs, and benefits can provide DSC a basis for designing and implementing controls over the reliability of information accessed through the ViSION system that are efficient and effective. Recommendation Related to ViSION System Information Reliability We recommend that the Director, DSC, conduct an assessment of supervisory information accessed through the ViSION system in order to define an acceptable accuracy rate and define controls and responsibilities over the reliability of supervisory information consistent with the results of the assessment. CORPORATION COMMENTS AND OIG EVALUATIONOn September 16, 2008, the Director, DSC, provided a written response to the draft of this report. Management’s response is presented in its entirety in Appendix 3 of this report. In its response, DSC concurred with the recommendation and outlined its planned corrective actions. 8
|
To address the recommendation, DSC will conduct a risk-based assessment of supervisory information accessed in ViSION to formalize acceptable data accuracy rates and to refine and clarify controls and responsibilities for monitoring data accuracy. These actions will be completed by June 30, 2009. A summary of management’s response to the recommendation is in Appendix 4 of this report. DSC’s planned actions are responsive to our recommendation. The recommendation is resolved but will remain open until we determine that the agreed-to corrective actions have been completed and are responsive. 9
|
APPENDIX 1OBJECTIVE, SCOPE, AND METHODOLOGY
|
APPENDIX 1
Internal Control We assessed the FDIC’s internal controls designed to ensure the reliability of key supervisory information accessed through the ViSION system. Such controls included relevant FDIC policies, procedures, and guidelines; the role of SMEs in maintaining reliable information in the ViSION system and IER; and DSC’s practices for entering and maintaining key supervisory information into the ViSION system and IER. Also, we considered relevant data quality assurance work conducted by DSC’s Internal Control and Review Section as part of their field territory and regional office reviews. 11
|
APPENDIX 1
Reliance on Computer-processed Information We relied on information in the ViSION system to identify the total number of examined financial institutions for which the FDIC was the primary federal regulator as of April 3, 2008. We used this information as our universe in selecting a random sample of 75 financial institutions for detailed analysis. To assure ourselves that the total number of FDIC-supervised institutions in the ViSION system was sufficiently reliable, we compared this information to a listing of FDIC-supervised financial institutions in the FDIC’s Institution Directory system as of April 3, 2008 and to information included in the FDIC’s 2007 annual report to the Congress. Further, we spoke with DSC officials to obtain their views on the integrity of the information and to discuss the manner in which we were planning to use it. We performed tests of the reliability of ViSION data in order to accomplish our audit objective. Performance Measurement We reviewed the FDIC’s 2005-2010 Strategic Plan, 2008 Annual Performance Plan, 2008 Corporate Performance Objectives, and 2007 Annual Report and found that they did not contain goals, objectives, or performance measures that were specifically relevant to our audit. Compliance With Laws and Regulations We considered the following laws and regulations in determining the supervisory information to be assessed during the audit. Evaluation of compliance with these laws and regulations was not significant to the audit objective.
12
|
APPENDIX 1
Additionally, we assessed the risk of fraud and abuse related to the audit objective in the course of evaluating audit evidence. Prior Coverage We considered the following reports previously issued by the FDIC OIG in planning and conducting our work:
13
|
APPENDIX 2ROLE OF EXAMINATION MAIL DATES IN CALCULATING |
APPENDIX 2
Table 2: Unreliable Examination Mail Dates in the ViSION System
We requested that a DIR analyst review the examination mail dates contained in Table 2 to determine whether the unreliable data had an effect on deposit insurance premiums charged by the Corporation. The analyst concluded that the three blank examination mail dates had no effect on deposit insurance premiums due to the manner in which the Corporation calculated assessments prior to the implementation of deposit insurance reform legislation. The analyst also concluded that inaccurate examination mail dates had no effect on the deposit insurance premiums charged to institutions F and G because the current examination ratings for these institutions were substantially the same as in the prior examinations. Further, the analyst concluded that inaccurate examination mail dates had at least some effect on the deposit insurance premiums for institutions D, E, and H because the current examination ratings for these institutions changed from the prior examinations. Based on information provided by the DIR analyst, we calculated the effect that inaccurate examination mail dates had on the premiums charged to institutions D, E, and H. Table 3 summarizes the results of our calculations. Table 3: Effects of Unreliable Examination Mail Dates on Insurance Assessments
*Parenthetical figures represent undercharges to financial institutions on their quarterly assessments. 15
|
APPENDIX 3CORPORATION COMMENTS
|
The Division of Supervision and Consumer Protection (DSC) has read the subject report and appreciates your finding that DSC has "taken steps to promote the reliability of information accessed through the ViSION system." As you note in your report, DSC is engaged in a comprehensive effort to improve the usefulness and reliability of the Interagency Examination Report (lER) repository. This effort is representative of our strong commitment to data integrity and continual system improvement with the collaboration of our interagency partners. Your recommendation and DSC's response follows: Recommendation We recommend that the Director, DSC, conduct an assessment of supervisory information accessed through the ViSION system in order to define an acceptable accuracy rate and define controls and responsibilities over the reliability of supervisory information consistent with the results of the assessment. DRR Response DSC concurs. We will conduct a risk-based assessment of the supervisory information accessed in ViSION, to formalize acceptable data accuracy rates, and to refine and clarify controls and responsibilities for monitoring data accuracy. These actions will be completed by June 30, 2009. 16
|
APPENDIX 4MANAGEMENT RESPONSE TO RECOMMENDATIONS
|
Corrective Action: Taken or Planned for the recommendation | Expected Completion Date | Monetary Benefits | Resolved:a Yes or No | Open or Closedb |
---|---|---|---|---|
DSC will conduct a risk-based assessment of the supervisory information accessed in ViSION to formalize acceptable data accuracy rates and to refine and clarify controls and responsibilities for monitoring data accuracy. | 6/30/2009 | NA | Yes | Open |
a Resolved – | (1) Management concurs with the recommendation, and the planned corrective action is consistent with the recommendation. |
(2) Management does not concur with the recommendation, but planned alternative action is acceptable to the OIG. | |
(3) Management agrees to the OIG monetary benefits, or a different amount, or no ($0) amount. Monetary benefits are considered resolved as long as management provides an amount. |
APPENDIX 5ACRONYMS USED IN THE REPORT
|
BSA | Bank Secrecy Act |
CAMELS | Capital Adequacy, Asset Quality, Management, Earnings, Liquidity, Sensitivity to Market Risk |
C.F.R. | Code of Federal Regulations |
DIR | Division of Insurance and Research |
DIT | Division of Information Technology |
DRR | Division of Resolutions and Receiverships |
DSC | Division of Supervision and Consumer Protection |
FBA | Federal Banking Agency |
FDI | Federal Deposit Insurance |
FIL | Financial Institution Letter |
FinCEN | Financial Crimes Enforcement Network |
GAO | Government Accountability Office |
IER | Interagency Examination Repository |
IT | Information Technology |
OIG | Office of Inspector General |
OMB | Office of Management and Budget |
ROE | Report of Examination |
RRPS | Risk Related Premium System |
SAR | Suspicious Activity Report |
SME | Subject Matter Expert |
TFR | Thrift Financial Report |
ViSION | Virtual Supervisory Information on the Net |
|