United States Nuclear Regulatory Commission - Protecting People and the Environment
Home > NRC Library > Document Collections > Inspector General Reports > 2000 > OIG-00A-05

OIG-00-A-05 - Review of NRC's Audit Follow-up System

[PDF Version (57KB) PDF Icon]

August 14, 2000

MEMORANDUM TO: William D. Travers
Executive Director for Operations

FROM: Stephen D. Dingbaum
Assistant Inspector General for Audits

SUBJECT: REVIEW OF NRC'S AUDIT FOLLOW-UP SYSTEM

Attached is the Office of the Inspector General's audit report titled, "REVIEW OF NRC'S AUDIT FOLLOW-UP SYSTEM."

This report reflects our evaluation of the agency's audit follow-up system in accordance with the Office of Management and Budget Circular A-50, Revised. Overall, we found that the NRC's audit follow-up system is adequate and, with a few exceptions, generally in compliance with the guidance. Improvements can be made to the audit follow-up system to make it more efficient and effective.

The report includes your response which agrees with the report's findings and recommendations. As a result, all recommendations are considered resolved, but additional actions are needed before they are closed. Please provide the status on actions taken or planned on each of the recommendations within 60 days of the date of this memorandum.

If you have any questions, please call me at 415-5915.

Attachment: As stated

cc: R. McOsker, OCM/RAM
B. Torres, ACMUI
B. Garrick, ACNW
D. Powers, ACRS
J. Larkins, ACRS/ACNW
P. Bollwerk III, ASLBP
K. Cyr, OGC
J. Cordes, Acting OCAA
P. Bird, HR
I. Little, SBCR
W. Kane, NMSS
S. Collins, NRR
A. Thadani, RES
P. Lohaus,
F. Congel, IRO
H. Miller, RI
L. Reyes, RII
J. Dyer, RIII
E. Merschoff, RIV
OPA-RI
OPA-RII
OPA-RIII

To top of page

Report Synopsis

Audit follow-up is the system that Federal agencies use to resolve audit recommendations resulting from audits of Federal programs and operations; to implement and track corrective actions; and to fulfill reporting requirements. The Office of Management and Budget Circular A-50, Revised (Circular A-50), provides the policies and procedures for use by Federal agencies when considering reports issued by the Inspectors General, other executive branch audit organizations, the General Accounting Office, and non-Federal auditors where follow-up is necessary. In addition, Circular A-50 provides that Federal agency audit follow-up systems are to be evaluated on a periodic basis to determine if the systems result in efficient, prompt, and proper resolution and corrective action on audit recommendations. The Nuclear Regulatory Commission's (NRC) Management Directive (MD) 6.1 and associated Handbook provide the agency with guidance to implement Circular A-50 requirements. Our objectives for this review were to determine if the NRC's audit follow-up system was in compliance with applicable requirements and if the system adequately met the intent of those requirements.

Overall, we found that the NRC's audit follow-up system is adequate and, with a few exceptions, generally in compliance with external and internal guidance. However, improvements can be made to the audit follow-up system to make it more efficient and effective. The NRC's electronic tracking system uses an old technology and does not expressly track audit recommendation resolution dates. As the NRC develops a new agencywide tracking system, which will include audit follow-up tracking, it should ensure that the new system includes a unique field for tracking resolution dates. The agency may also be missing opportunities for saving valuable resources by not conducting audit follow-up reviews at a more consistent interval or by not determining trends and identifying system-wide problems as required by Circular A-50. In addition, the NRC does not consistently provide the Office of the Inspector General (OIG) with written notification describing the corrective actions taken in response to OIG audit recommendations, as specified by MD 6.1.

Our report makes four recommendations to bring the NRC into full compliance with Circular A-50 and to improve its audit follow-up effectiveness and efficiency. In addition, our work identified practices used by several Federal agencies. We have included a listing of those practices as an appendix to our report.

To top of page

Introduction

Audit follow-up is the system that Federal agencies use to resolve audit recommendations resulting from audits of Federal programs and operations; to implement and track corrective actions; and to fulfill reporting requirements. The Office of Management and Budget (OMB) Circular A-50, Revised (Circular A-50), requires that Federal agencies evaluate their audit follow-up systems to determine if the systems result in efficient, prompt, and proper resolution and corrective action on audit recommendations. Circular A-50 states that audit follow-up is an integral part of good management and is a shared responsibility of agency management officials and auditors. Furthermore, corrective action taken by management on resolved findings and recommendations is crucial to improving the effectiveness and efficiency of Government operations.

The Office of the Inspector General (OIG) periodically evaluates the adequacy of the Nuclear Regulatory Commission's (NRC) audit follow-up system. In 1993, OIG reviewed the NRC's audit follow-up system and found that, overall, the agency conformed with OMB standards.(1) However, the agency's Audit Follow-up Official (AFO) lacked the authority to track and report on final actions on audit recommendations made to Commission level offices. In response, the Executive Director for Operations (EDO), as the AFO, agreed to begin tracking those recommendations. In 1997, OIG conducted another audit of the agency's follow-up system.(2) OIG found that the agency's guidance for handling and resolving OIG audit recommendations was outdated and needed to be revised. In response, the NRC revised Management Directive (MD) 6.1 and its associated Handbook, both titled "Resolution and Follow up of Audit Recommendations," to identify officials responsible for reviewing and responding to OIG draft reports and resolving disagreements on audit recommendations that arise between OIG and agency officials.

Our objectives for this review were to determine if the NRC's audit follow-up system was in compliance with the Inspector General Act of 1978, as amended, Circular A-50, and MD 6.1 requirements, and if the audit follow-up system adequately met the intent of those requirements. Appendix I contains additional information on our objectives, scope, and methodology.

To top of page

Background

In 1982, OMB issued Circular A-50 to provide Federal agencies with policies and procedures for audit follow-up. Circular A-50 requires each agency to establish systems to assure prompt and proper resolution and implementation of audit recommendations. It also requires that these systems provide a complete record of action taken on monetary and non-monetary findings and recommendations. Furthermore, Circular A-50 assigns responsibilities to agency heads, management officials, Inspectors General, AFOs, and the Comptroller General.

More specifically, Circular A-50 requires each agency head to designate a top management official to serve as the AFO. The AFO has personal responsibility for ensuring that systems of audit follow-up, resolution, and corrective action are documented and in place; timely responses are made to all audit reports; disagreements are resolved; and corrective actions are actually taken.

The NRC's MD 6.1 and associated Handbook provide the guidance to implement the agency's audit follow-up system per Circular A-50 requirements. Through MD 6.1, the EDO is designated as the agency's AFO. To assist the AFO in carrying out his audit follow-up responsibilities, he has a small support staff within the Office of the Executive Director for Operations.

While the EDO serves as the AFO for the entire agency, he is one of three members of the NRC's Executive Council (EC). The EDO, the Chief Financial Officer (CFO), and the Chief Information Officer (CIO) make up the EC. Although the EDO serves as the EC chair, all three EC members currently report directly to the NRC's Chairman. A former NRC Chairman created the EC for the strategic implementation of the Commission's policies and programs while taking an agencywide view of financial management and information technology.

In light of the EC organizational structure, the EDO, as the AFO, is only responsible for resolving disagreements on audit findings and recommendations between the Deputy Executive Directors for Operations (DEDOs) and the Inspector General (IG). The NRC's program offices report to the EDO through the DEDOs (see Appendix III for an NRC organizational chart). For disagreements between Commission-level offices and the IG, the AFO is limited to only facilitating a resolution. The CFO and the CIO are responsible for informing the AFO of any disagreements between their respective offices and the IG on audit findings and recommendations. Therefore, the NRC Chairman ultimately has the final resolution authority unless audit recommendations involve policy formulation or any other matter within the authority of the Commission. In those instances, the full Commission must be consulted.

To top of page

Results of Audit

Overall, we found that the NRC's audit follow-up system is adequate and, with a few exceptions, generally in compliance with external and internal guidance. However, improvements can be made to the audit follow-up system to make it more efficient and effective. We found that the agency does not have a reliable system for tracking audit recommendation resolution dates. And, the agency is not adhering to its own guidance to perform audit follow-up reviews on an annual basis. Furthermore, the agency has not yet performed a trend analysis, which is required by Circular A-50, to identify system-wide problems. We also found that the agency is not consistently notifying OIG, in writing, when it implements corrective actions, as required by MD 6.1.

As part of our review, we visited with three Federal agencies and discussed their audit follow-up systems. Appendix IV offers a sampling of the practices used by these agencies.

To top of page

Resolution Dates Are Not Tracked

The NRC's audit follow-up system generally meets the intent of the Circular A-50 requirement to track audit recommendations throughout the follow-up process. However, the agency's electronic tracking system, Work Item Tracking System (WITS), is a DOS-based system with limited capabilities. Because of its limitations, WITS does not expressly track resolution dates on audit recommendations and, therefore, does not fully comply with Circular A-50.

According to Circular A-50, audit follow-up systems must maintain accurate records of the status of audit reports or recommendations through the entire resolution and corrective action process. Specifically, resolution, which occurs when the agency and OIG agree on the action to be taken to implement a recommendation, shall be made within a maximum of 6 months after issuance of a final report in accordance with the Inspector General Act of 1978, as amended. Furthermore, the Act requires that final action must be completed within 12 months after the issuance of the OIG audit report.

For fiscal years 1995 through 1999, we identified 82 OIG and 11 General Accounting Office recommendations that should be included in the agency's audit follow-up tracking system. We found that each recommendation had a corresponding audit report paper file maintained by the AFO's staff and most of the recommendations were being tracked in WITS. While we were able to identify the dates that the agency implemented corrective actions in response to recommendations, we were not able to determine the recommendation resolution dates. WITS has limited functionality and does not contain a unique field for recommendation resolution dates. Additionally, the AFO's paper files did not contain evidence of when, and if, resolution had been reached on the audit recommendations.

In addition to using WITS for audit follow-up tracking, the EDO also relies on it as a correspondence tracking system. In 1999, the agency requested Arthur Andersen, LLP, to initiate a study to review ways to improve the NRC's support activities. One recommendation from this study was to review the agency's correspondence tracking process and identify ways to eliminate duplicative functions. Most NRC offices have their own electronic tracking systems, which replicate the data in WITS. As a result of Arthur Andersen's recommendation, the offices of the Secretary, CIO, and EDO have been tasked to determine requirements for an agencywide tracking system, which will also include audit follow-up tracking. The task force will look at the NRC's electronic information management system, the Agencywide Documents Access and Management System, as a starting point for developing the new agencywide tracking system.

Because WITS does not have a unique field for tracking resolution dates and the AFO's paper files are not annotated with such information, the agency does not appear to have a reliable system to ensure that resolution is reached within 6 months. Therefore, the agency is not able to fully comply with the Circular

A-50 requirement of maintaining accurate records on the status of audit recommendations through the entire resolution process.

To top of page

Lack of Trend Analysis--Missed Opportunity

Circular A-50 requires that audit follow-up systems provide for a periodic analysis of audit recommendations, resolution, and corrective action to determine trends and system-wide problems, and to recommend solutions. While the agency has conducted several reviews of audit recommendations, resolution, and corrective actions, it has not conducted an analysis to determine trends and system-wide problems. We believe that by not conducting a trend analysis, the agency could be missing the opportunity to identify and correct systemic problems.

The NRC's MD 6.1 takes a more stringent approach than Circular A-50 regarding the periodic analysis of audit recommendations, resolution, and corrective action. MD 6.1 requires the AFO to conduct an annual audit follow-up review to determine if the implemented corrective actions resolve the problems identified in audit reports. The AFO's review should ensure that the corrective actions agreed to as a result of audit recommendations have been implemented. MD 6.1 also tasks the AFO to conduct periodic analyses of audit recommendations, resolution, and corrective action activities to determine trends and problems and to recommend solutions. However, the associated MD 6.1 Handbook does not address this task.

While the NRC does periodically conduct audit follow-up reviews, the reviews are not done on an annual basis. Recently, the AFO staff directed an audit follow-up review that encompassed six audit reports conducted in fiscal years 1994 through 1997. All six of the follow-up reviews have been completed and the reviewers have determined that the intent of the recommendations has been satisfied. An AFO staff member noted that prior to this current review, the AFO conducted an audit follow-up review in 1995 covering three audit reports ranging in dates from 1988 to 1991.

The agency's two previous audit follow-up reviews did not include an analysis to determine trends and system-wide problems. A senior AFO staff member stated that, while the agency has the opportunity to see trends, they have not documented any analyses. The AFO's staff plans to complete and document a trend analysis in the future.

AFO staff members said that annual audit follow-up reviews and trend analyses have not been conducted for a variety of reasons including personnel turnover. In light of these issues, the AFO staff made a conscious decision to place a lower priority on audit follow-up reviews and trend analysis. We also believe that the staff could have overlooked the trend analysis requirement because it is not explicitly addressed in the MD 6.1 Handbook, which provides the implementing guidance.

By not conducting periodic audit follow-up reviews on a consistent interval and not identifying and analyzing trends, we believe the agency could miss the opportunity to identify and correct systemic problems. Furthermore, the agency's early identification and correction might prevent recurring issues in the future. For example, in two 1996 audits,(3) OIG found that the agency did not effectively manage information technology (IT) project development cost and schedule requirements. During audits conducted in 1998(4) and 2000,(5) OIG again had similar findings regarding IT project development.

To top of page

OIG Not Consistently Notified When Corrective Action Completed

The agency does not consistently notify the OIG, in writing, upon completion of corrective actions in response to audit recommendations as required by MD 6.1. Some agency officials that we spoke with were either not clear of the requirement to notify OIG, or were unaware that it was not done consistently. Absent such notification, the NRC is not in full compliance with MD 6.1 and OIG cannot readily determine the status of corrective actions.

MD 6.1 requires that the agency's audited entity advise the AFO and the OIG, in writing, when corrective action is completed and provide a description of the actions taken to implement each recommendation. The written notification should also include, when applicable, a copy of the document closing out the recommendation.

We found that the audited entities within the NRC consistently notify the AFO when corrective actions on audit recommendations have been implemented. The audited entities also include, when possible, evidence of the corrective action taken. However, OIG does not consistently receive this written notification.

We believe the inconsistencies in OIG receiving written notifications may be due to a lack of awareness of MD 6.1 requirements. For example, one agency official, with responsibility for tracking OIG audit recommendations for his office, explained that he provides written notification to the AFO when corrective action is completed but he was not aware of the requirement to also notify OIG in writing. The official surmised that the AFO provided this information to OIG. In addition, a high level AFO staff member believed that the agency had been advising the OIG in writing, but agreed that the NRC staff may not be doing it consistently.

By not consistently notifying OIG, in writing, when corrective action has been implemented, the NRC is not in full compliance with MD 6.1 guidance. Furthermore, without timely, written notification, it is difficult for OIG to assess if potential vulnerabilities still exist in the audited program.

To top of page

Conclusion

Although the NRC's audit follow-up system is generally adequate, several steps can be taken to streamline the system and bring it into full compliance with Circular A-50 and MD 6.1. The agency does not have a reliable system to track recommendation resolution dates, as required by Circular A-50. Therefore, we believe that as the agency develops its new agencywide tracking system, it should include audit follow-up tracking requirements. Additionally, the agency has not fully complied with MD 6.1 guidance that requires an annual audit follow-up review. While the requirement to perform such reviews annually might be too rigorous, we believe that the AFO should conduct the Circular A-50 required periodic reviews on a more consistent interval. By not conducting a trend analysis to determine system-wide problems, the agency is not in full compliance with Circular A-50 and could miss the opportunity to identify and correct such problems. Lastly, the agency should adhere to MD 6.1 guidance and consistently notify OIG, in writing, when corrective actions are implemented. The agency's notification should provide evidence that the deficient condition has been corrected.

To top of page

Recommendations

For the NRC's audit follow-up system to fully comply with Circular A-50 and MD 6.1, we recommend that the Executive Director for Operations in his capacity as the Audit Follow-up Official:

  1. Ensure that, as the agency develops its new agencywide tracking system, the system includes the ability to track audit recommendation resolution dates to meet Circular A-50 audit follow-up tracking requirements.

  2. Revise the MD 6.1 Handbook to ensure that the agency conducts periodic analyses of audit recommendations to determine possible trends and system-wide problems and recommend solutions, as required by Circular A-50.

  3. Assess the MD 6.1 Handbook scheduling requirements for conducting audit follow-up reviews. Whether done annually or periodically, the reviews should be conducted on a consistent frequency.

  4. Ensure that all NRC offices are aware of the MD 6.1 Handbook requirement to advise OIG, in writing, when corrective actions in response to OIG recommendations have been implemented. This notification should also contain a description of the corrective actions taken and a copy of the document (if applicable) closing out the recommendation

To top of page

OIG Comments on The Agency's Response

On August 2, 2000, the EDO responded to the draft report, agreed with the recommendations, and provided implementation dates. The EDO agreed to track resolution dates and he will ensure that the new agencywide tracking system will also include the ability to track audit recommendation resolution dates. Furthermore, the EDO agreed to revise Management Directive 6.1 and associated Handbook. These actions address the intent of the recommendations.

Appendices

To top of page

Appendix I: Objectives, Scope, and Methodology

The objectives of our audit were to determine if the Nuclear Regulatory Commission's (NRC) audit follow-up system was in compliance with applicable requirements and if the system adequately met the intent of those requirements. Our review examined the agency's tracking system and focused on the role and responsibilities of the NRC's Audit Follow-up Official (AFO).

We reviewed the agency's management controls and the effectiveness of those controls related to the NRC's audit follow-up system. Additionally, we identified and reviewed applicable guidance (e.g., Office of Management and Budget Circular A-50, Revised; the Inspector General Act of 1978, as amended; and the NRC's Management Directive 6.1 and associated Handbook). We interviewed NRC staff members in the Office of the Executive Director for Operations, Office of the Chief Financial Officer, Office of the Chief Information Officer, and other agency program offices.

To evaluate if the agency's system adequately met the intent of the guidance on audit follow-up, we reviewed the AFO's paper files and the Work Item Tracking System (WITS). We examined WITS' capabilities and the usefulness of the system. Furthermore, we identified and reviewed Office of the Inspector General and General Accounting Office recommendations for fiscal years 1995 through 1999 to test the reliability of the agency's system.

We also visited with three agencies (i.e., Department of Transportation, Small Business Administration, and Federal Communications Commission) and discussed their audit follow-up systems to compile a collection of practices as a basis for benchmarking with the NRC. Appendix IV contains the results of our work with these agencies.

Our audit was conducted from January 2000 to May 2000 in accordance with generally accepted Government auditing standards.

To top of page

Appendix II: Agency Comments on Draft Report

MEMORANDUM FOR: Stephen D. Dingbaum,
Assistant Inspector General for Audits
FROM: William D. Travers
Executive Director for Operations
SUBJECT: REVIEW OF NRC'S AUDIT FOLLOW-UP SYSTEM

This responds to the July 11, 2000, memorandum transmitting the subject draft audit report. I am pleased to note your conclusion that overall the NRC's audit follow-up system is adequate and is generally in compliance with external and internal guidance. With respect to your specific recommendations, I plan on taking the following actions:

Recommendation 1

"Ensure that, as the agency develops its new agencywide tracking system, the system includes the ability to track audit recommendation resolution dates to meet Circular A-50 audit follow-up tracking requirements."

Response

Agree. When the new agencywide tracking system is developed, it will include the ability to track audit recommendation resolution dates. Additionally, effective with this audit report, the resolution dates will be included in the basis section of the current tracking system and placed in our files.

As part of the effort to revise the MD 6.1 Handbook in response to recommendation 2, the guidance for tracking resolution of audit recommendations will be reviewed to ensure that roles and responsibilities are clearly stated. As discussed in the exit conference, we believe that the difference between resolving and completing an audit recommendation is not clearly understood. The role of the Audit Follow up Official (AFO) in facilitating resolution between the OIG and agency officials is also not fully understood. Appendix III of the draft audit report discussed the agency and IG working together as one of the practices that other federal agencies found beneficial. We would like to use this approach to clarify guidance for issues associated with the resolution of audit recommendations. The results of this effort will be included in the revisions to MD 6.1.

Completion Date: The intent of the recommendation is being implemented immediately by including resolution dates in the current tracking system. Incorporating this requirement into a new agencywide tracking system is dependent upon the schedule and is not likely to occur prior to FY 2002. This recommendation is considered closed.

Recommendation 2

"Revise the MD 6.1 Handbook to ensure that the agency conducts periodic analyses of audit recommendations to determine possible trends and system-wide problems and recommend solutions, as required by Circular A-50."

Response

Agree. The MD 6.1 Handbook will be revised to include a description of the process that will be used to conduct periodic analyses of audit recommendations to determine trends and systemic problems.

Completion Date: June 2001

Recommendation 3

"Assess the MD 6.1 Handbook scheduling requirements for conducting audit follow-up reviews. Whether done annually or periodically, the reviews should be conducted on a consistent frequency."

Response

Agree. As part of revising the MD 6.1 Handbook to include a process for trending audit recommendations, the frequency of audit follow-up reviews will be assessed. The MD 6.1 Handbook will be revised, as appropriate.

Completion Date: June 2001

Recommendation 4

"Ensure that all NRC offices are aware of the MD 6.1 Handbook requirement to advise OIG, in writing, when corrective actions in response to OIG recommendation have been implemented. This notification should also contain a description of the corrective actions taken and a copy of the document (if applicable) closing out the recommendation."

Response

Agree. This has been discussed in staff meetings with Office Directors and Regional Administrators. Additionally, a memorandum will be sent to all offices reiterating this requirement. This memorandum will also be used to highlight the AFO role for ensuring that recommendations are resolved.

Completion Date: November 2000

cc: Chairman Meserve
Commissioner Dicus
Commissioner Diaz
Commissioner McGaffigan
Commissioner Merrifield

To top of page

Appendix III: NRC ORGANIZATIONAL CHART

NRC Organizational Chart

To top of page

Appendix IV: Review of NRC's Audit Follow-Up System

Audit Follow-up Practices Employed by Federal Agencies

Beyond the audit follow-up policies and procedures provided by the Office of Management and Budget's (OMB) Circular A-50, Revised, Federal agencies have developed specific procedures that supplement OMB's overall guidance. During the course of our audit, we visited with three Federal agencies (i.e., Department of Transportation, Small Business Administration, and Federal Communications Commission) and compiled a collection of the practices used by these agencies for audit follow-up. We present them in this appendix to offer options for the Nuclear Regulatory Commission to consider and perhaps to adopt or modify to suit its unique audit follow-up needs.

  • In response to audit recommendations, the agency provides Corrective Action Plans to the Audit Follow-up Official for review. The Corrective Action Plans include benchmark dates, which are monitored by the agency.

  • Posted on its website, the agency has a Fact Sheet with Frequently Asked Questions on how to respond to audit findings.

  • The agency and its Office of the Inspector General worked together to create and publish a brochure titled, What You Should Know About the Office of Inspector General (OIG) Audit Follow-up and Reporting Processes in [the Agency]. The brochure includes concise descriptions on areas from audit entrance conferences through closing audit report recommendations. Also included in the brochure is a section on most commonly asked audit questions and a glossary of audit terms.

  • OIG and the agency review two reports generated monthly from the agencywide, electronic audit follow-up tracking system. The first report titled, Action of Resolution, shows the status of resolution. Resolution is when the agency and OIG agree on actions to be taken in response to audit findings and recommendations. The second report titled, Action on Closure, shows the status of implementation actions to complete audit recommendations.

  • Each organization within the agency has an audit liaison who serves as the point of contact for the OIG. The audit liaison also has a good understanding of the entire audit follow-up process.

To top of page

Appendix V: Major Contributors to this Report

Anthony C. Lipuma
Team Leader

Russell Irish
Staff Assistant for Planning and Reporting

Cheryl Miotla
Senior Management Analyst

David Horn
Senior Auditor

1. Review of NRC's System to Follow-up On Audit Recommendations, OIG/93-08, August 2, 1993.

2. Review of NRC's Management Directive 6.1, OIG/97A-20, September 5, 1997.

3. Improvements Needed in Agency Oversight of Information Resources Management Activities, OIG/96A-11, September 24, 1996, and Review of NRC's Progress in Developing and Implementing an Integrated Payroll/Personnel System (PAY/PERS), OIG/96A-15, September 30, 1996.

4. Review of NRC's Controls Over the PC Refresh Program, OIG/98A-07, October 9, 1998.

5. Review of the Development and Implementation of STARFIRE, OIG-99A-14, June 29, 2000.

Page Last Reviewed/Updated Thursday, March 29, 2012