This report presents the results of our evaluation of the Federal Deposit Insurance Corporation’s
(FDIC) use of performance measures. Congress has established a statutory framework to
improve the performance, management, and accountability of the federal government. The
Government Performance and Results Act of 1993 (GPRA) is a central component of that
framework. GPRA seeks to improve the management of federal programs, as well as their
effectiveness and efficiency, by establishing a system under which agencies set goals for
program performance and measure their results. The FDIC meets GPRA requirements through
the issuance of a strategic plan, an annual performance plan (APP), and a performance and
accountability report (PAR). The FDIC also has implemented additional performance
measurement processes in the form of Corporate Performance Objectives (CPOs) and
balanced scorecards.[ 1 ] We performed this evaluation as a follow-up to a prior evaluation of the
FDIC’s Corporate Planning Cycle.[ 2 ]
EVALUATION OBJECTIVE AND APPROACH
Our objective was to evaluate the performance measurement processes that the FDIC uses to
monitor corporate performance. To accomplish our objective, we evaluated:
- whether the GPRA, CPO, and scorecard measures are aligned with the FDIC’s strategic
goals and objectives,
- FDIC management and stakeholder views on the usefulness of performance measures
in monitoring corporate performance,
- whether there are opportunities for streamlining or combining the FDIC’s performance
measurement processes, and
- whether performance results are transparent to internal and external stakeholders.
Appendix I describes in detail our objective, scope, and methodology.
RESULTS IN BRIEF
The FDIC has developed and implemented multiple performance measurement processes and
approaches that serve various stakeholder needs and that FDIC managers use to varying levels
to manage and monitor program performance. These processes and approaches include:
- the legislatively-mandated GPRA annual performance planning and reporting efforts,
- the internally-focused CPO process,
- division-specific balanced scorecards, and
- other performance metrics related to individual contracts and system development efforts.
Collectively, we found that the FDIC uses performance measures to make various types of
management decisions to improve programs and results. For example, we saw evidence that
the FDIC uses performance measures for:
- identifying problems and taking corrective action,
- developing management strategies and allocating resources,
- recognizing and rewarding performance, and
- identifying and sharing effective processes and approaches.
We also observed that the FDIC employs the following practices that enhance the use of
performance information:
- demonstrating management commitment;
- aligning agency goals, objectives, and measures;
- improving the usefulness of performance information through tools, systems, and
processes; and
- communicating performance information internally.
These observations reflect most of the attributes that the Government Accountability Office
(GAO) has reported as examples of how agencies can use performance information for key
management decisions and agency practices that can enhance or facilitate such use.
We found that FDIC managers have embraced and used some performance measures more
than others to make management decisions and to manage their programs. For example:
- While FDIC senior managers are mindful of the broad, mission-related goals in the Strategic
Plan and APP, most that we interviewed stated that they do not find APP information useful
or use APP information to manage their programs.
- FDIC management has placed considerable emphasis on the CPO process to measure and
internally report on strategic, corporate-wide initiatives that address some of the most
important issues facing the Corporation. The Corporation has also established a policy that
ties executive manager pay to the achievement of CPOs and other performance measures.
- Certain divisions and offices have implemented balanced scorecards. The Division of
Supervision and Consumer Protection (DSC), in particular, uses its scorecard to monitor and
manage day-to-day operations at regional and field offices. Several other divisions and
offices are in the process of developing scorecards, but implementation has been uneven.
While there needs to be flexibility in scorecard design to meet individual divisional needs, the COO and CFO could provide more high-level direction on the use of scorecards and
how scorecards should integrate with the FDIC’s other performance measurement
processes.
The FDIC also communicates performance measures externally by publishing a strategic plan,
APP, and PAR. These documents generally contain information required by GPRA and
implementing guidance issued by the Office of Management and Budget (OMB).
Notwithstanding, we identified the following issues and challenges associated with the
Corporation’s implementation of GPRA and opportunities to enhance the APP and PAR to
increase transparency, improve corporate accountability, and provide information to aid in
congressional decision-making, all key purposes of the GPRA legislation:
- While a majority of CPOs are aligned with APP goals and FDIC strategic objectives, there
are a number of important FDIC initiatives with associated CPOs that would likely be of
interest to the public and the Congress that are not addressed in the APP. Because the
CPOs are distributed internally, they are not transparent to the public or Congress. The
FDIC could benefit from a more structured, documented process for determining which
initiatives should be reported in the APP, as CPOs, or in both sets of performance
measures.
- Consistent with the discretion allowed in OMB guidance, the APP is limited to business line
areas (Insurance, Supervision, and Receivership Management). However, this approach
results in important, financially significant initiatives related to areas such as human capital
and system development efforts not being reflected in the APP.
- FDIC officials have looked for opportunities to establish outcome-related goals but told us
they continue to “struggle” with the validity of such measures because there are factors
beyond the control of the Corporation that influence achievement of the outcomes. GPRA
contains a specific emphasis for agencies to move toward using qualitative outcome
measures that provide a better measurement of program results than quantitative output
measures.
- Finally, the FDIC performs program evaluations of APP goals but could add more structure
and independence to the program evaluation effort.
BACKGROUND
Congress and the OMB have worked to implement a statutory and management reform
framework to improve the performance and accountability of the federal government. Congress
enacted GPRA as a key element of this framework to:
- improve public confidence in the federal government by holding federal agencies
accountable for their program results;
- reform performance by using program goals, measuring results, and publicly reporting the
progress;
- promote service quality and customer satisfaction;
- improve congressional decision-making by providing objective information on agency
program effectiveness and efficiency; and
- improve internal management of the federal government.
In 1995, the FDIC’s Legal Division concluded that the Corporation was subject to certain GPRA
provisions, specifically, the requirement to prepare and submit a strategic plan, an APP, and a
PAR. The FDIC developed its initial strategic plan in 1995. The FDIC Board approved the
current Strategic Plan 2005-2010 in January 2005. The Plan reflects goals, objectives, and
strategies for three major program areas: Insurance, Supervision, and Receivership
Management. The strategic goals and objectives state what must be done to achieve the
FDIC’s mission.
The FDIC’s strategic plan is implemented through the APP, which includes annual performance
goals, indicators, and targets for each strategic objective within the three major program areas.
The performance goals use a mix of output and milestone targets to measure the FDIC’s efforts
toward accomplishing its mission and strategic goals.
In accordance with the provisions of section 17(a) of the Federal Deposit Insurance Act and
GPRA, the FDIC submits a PAR to the President and the Congress that compares actual
performance to the APP annual performance goals. This report is intended to provide important
information to FDIC managers, policymakers, and the public on what the FDIC accomplished
with its budgeted resources.
The Division of Finance’s (DOF) Corporate Planning & Performance Management (CPPM)
section manages the FDIC’s planning and budget process and provides guidance to division
and office staff responsible for business planning, budgeting, and reporting. FDIC Circular
4100.2, Verification and Validation Guidelines for Performance Planning and Reporting, dated
November 30, 2001, provides guidance to divisions and offices responsible for reporting the
APP results in the PAR to Congress. The FDIC recently issued Circular 4100.4, Corporate
Planning and Budget Processes, dated March 21, 2007, which provides an overview of
corporate planning and budget processes and defines the roles and responsibilities of FDIC
employees and organizations in those processes.
CPPM prepares exception-based quarterly performance results memoranda for the Chief
Operating Officer (COO) and Chief Financial Officer (CFO), which include information about
GPRA performance goals that are behind schedule or not on target. The Office of Enterprise
Risk Management (OERM) prepares the FDIC’s PAR with assistance from CPPM and has
responsibility for completing program evaluations of the Corporation’s business lines as
contemplated under GPRA.
Corporate Performance Objectives
The FDIC also develops annual CPOs, which FDIC management defines as a compilation of
specific performance targets and measures developed during the annual planning and budget
process and approved by the Chairman. The CPO document defines performance targets to be
accomplished on an annual basis. These objectives often cut across organizational lines and
provide a mechanism for managing the performance of FDIC organizations from a corporate
perspective. FDIC management characterizes these objectives as the Corporation’s “change
management” agenda, and these objectives are separate and distinct from the mission-critical
APP annual performance goals. DOF disseminates the annual CPO document to all employees
within the Corporation, but not publicly outside of the FDIC.
Balanced Scorecards and Divisional Initiatives
A scorecard system is a strategic management tool that helps manage, monitor, and
communicate strategic goals and objectives. DSC developed a balanced scorecard in 2003 to
measure and compare information such as examiner efficiency between regional and field
offices. FDIC established a 2005 CPO initiative to develop and pilot test for 6 months in all
divisions and appropriate offices, a multi-dimensional scorecard to measure performance
against meaningful costs, quality, timeliness, and customer service benchmarks with the intent
of implementing a corporate scorecard program by March 2006. OERM assumed responsibility
for leading this effort.
Moreover, some divisions identify and track progress on performance initiatives that are
important to their operations but do not rise to the level of a CPO.
Other Performance Measures
The FDIC also has other performance measurement processes for individual contracts,
projects, and initiatives. The FDIC adopted significant changes in 2004 to the sourcing strategy
for obtaining contractor support for its IT functions. These changes incorporate the concept of
using performance-based, results-driven contracts. The structure of the new contracts places
the emphasis on contractor performance and links contractor compensation to results achieved
rather than hours worked. Moreover, the Division of Administration (DOA) and the Division of
Information Technology (DIT) prepare periodic reports for major contract and system
development efforts that include performance metric information.
EVALUATION RESULTS
The FDIC Uses Performance Information to Make Management
Decisions and to Improve Programs and Results
The FDIC has developed and implemented multiple performance measurement processes that
serve various stakeholder needs and that FDIC managers use to varying levels to manage and
monitor program performance. Collectively, these processes provide FDIC managers with
information to make management decisions and to improve programs and results. In addition,
the FDIC has implemented various practices that enhance or facilitate the use of performance
information.
FDIC’s Use of Performance Information
In a 2005 report,[ 3 ] the GAO identified examples of how federal agencies are using performance
information for key management decisions. GAO also identified several categories of
management decisions for which managers can use performance information. We evaluated
the FDIC’s use of performance information using GAO’s report as a guide and identified similar
instances where the FDIC has used performance information for management decision-making
purposes, as illustrated below.
Identify problems in programs and take corrective action: GAO reported that performance
information can be used to identify gaps in performance, improve organizational processes, and
improve program performance. As discussed later, DSC has developed a balanced scorecard
that focuses on quantitative and qualitative success measures and provides information at the
national, regional, and territory level to reflect division results against predetermined targets and
thresholds. DSC uses scorecard information to determine program success or the need for
additional effort or resources.
The FDIC has also developed performance metrics in a red, yellow, and green stoplight fashion
to monitor cost, schedule, and overall performance of large contracts and capital investment
projects. The FDIC uses these metrics to make mid-course corrections, funding decisions, or
project management changes, if needed.
Developing strategies, plan and budget, identify priorities, and allocate resources: GAO noted
that agencies can use performance information to make decisions that affect future strategies,
to infuse performance concerns into planning and budgetary deliberations, and to compare
program results with goals and thus determine where to target program resources to improve
performance. We have previously reported that the FDIC has integrated its planning and
budget process.[ 4 ] One senior FDIC manager indicated that the FDIC uses its performance
measures to focus and drive the strategic direction of the Corporation. As discussed later, the
CPOs represent strategic, corporate-wide initiatives that address the most important issues
facing the Corporation. Several CPOs have been related to providing more attention and
additional resources to risks facing the banking industry or the FDIC. For example, the FDIC
had three CPOs during 2005 related to the Bank Secrecy Act (BSA), which included developing
an industry outreach effort, developing and implementing BSA and anti-money laundering
examination procedures, and doubling the number of BSA subject matter experts.
Recognize and reward performance: Agencies can use performance information to affect pay
decisions and to reward individuals. Using performance information this way reinforces
accountability and creates incentives for achieving results. GAO notes that high-performing
public-sector organizations create a clear “line of sight” between individual performance and
organizational success showing how team, unit, and individual performance can contribute to
overall organizational success. At the FDIC, corporate manager and executive manager pay
and funding for bonuses is based in part on the Corporation’s level of success in meeting
corporate goals and objectives.
Identify and share more effective processes and approaches: GAO reported that successful
organizations continuously assess and benchmark performance and efforts to improve
performance. These organizations evaluate their efforts using fact-based understandings of
how their activities contribute to accomplishing agency missions and achieving results, and
optimize their efforts through continuous improvement. Managers can use performance
information to identify and increase the use of program approaches that are working well, and
consider alternative processes in areas where goals are not being met. We identified instances
where the FDIC expanded the use of successful CPO initiatives to other areas of the
Corporation. For example, as discussed later, DSC developed a balanced scorecard in 2003 to
monitor and manage day-to-day operations at regional and field offices. The Corporation
subsequently expanded the use of balanced scorecards to other divisions and offices.
In a second case, the FDIC developed a Talent Review[ 5 ] program at the executive manager
level in response to a 2006 CPO.[ 6 ] Due to the success of this program, the FDIC established a
2007 CPO to expand the Talent Review to the Corporate Manager ranks.
Practices Used by the FDIC to Enhance Performance Information
The GAO also stated that agencies can adopt or apply a number of practices that can enhance
and facilitate greater use of performance information for policy and program decisions. We
noted the following practices used by FDIC management to enhance the use of performance
information:
Demonstrating management commitment: GAO reported that the commitment of agency
managers to results-oriented management is critical to increased use of performance
information for policy and program decisions. We observed that the 2007 Planning and Budget
Conference was attended by the FDIC’s Vice Chairman, a Board member, a Chairman’s
representative, the COO, the CFO, and most division and office directors. Moreover, the CPPM
provides a quarterly performance review report to the CFO and COO that presents the status of
each CPO and related initiatives and provides a quarterly exception report of APP goals and
targets that are not being met. These quarterly reports are also provided to the FDIC Chairman,
Vice Chairman, and FDIC Director.
Aligning agency-wide goals, objectives and measures: GAO reported that agencies can
encourage greater use of performance information by aligning agency-wide goals and
objectives, and by aligning program performance measures at each operating level with those
goals and objectives. GAO notes that “cascading” goals and objectives throughout the
organization and aligning performance measures to the objectives from the executive level
down to the operational levels forms hierarchies of goals and objectives and performance
information that are appropriate to the managerial responsibilities and controls at each level of
the organization. The FDIC’s strategic goals and objectives cascade down through the
organization from the APP and CPOs to more detailed divisional initiatives and balanced
scorecard information and, in some cases, detailed performance-based contracting measures.
As discussed later in this report, we found that a majority of the CPOs are aligned with APP
goals and strategic objectives. However, we did identify opportunities to increase alignment,
particularly by including goals in the APP related to key mission support activities.
Improving the usefulness of performance information: GAO notes that to ensure that
performance information is both useful and used in decision making, agencies need to consider
users’ differing policy and management information needs and ensure that performance
information meets users’ needs for completeness, accuracy, consistency, timeliness, validity,
and ease of use. GAO notes that involving managers in the development of performance goals
and measures is critical to increasing the relevance and therefore the usefulness of
performance information to their day-to-day activities. As discussed later, FDIC senior level
managers are involved in developing, reviewing and approving APP and CPO goals, targets,
and indicators during the annual planning and budget process. DOF also issued Circular
4100.2, Verification and Validation Guidelines for Performance Planning and Reporting, dated November 30, 2001, which revised the roles and responsibilities for ensuring the accuracy of
FDIC Quarterly and Annual Performance Reports data.
DOF has also established a Cost Management Program designed to provide reliable and timely
information about the full cost of the FDIC’s business processes, activities, and outputs on a
regular basis to enable managers to make decisions about allocating resources, evaluating
program performance, and improving efficiency and effectiveness. DOF has implemented an
Activity Based Management module to the FDIC’s New Financial Environment and is presently
working with divisions and offices to design management reports to support specific cost
management and performance measurement needs.
Communicating performance information frequently and effectively: GAO reported that
improving the communication of performance information among staff and stakeholders can
facilitate the use of performance information by agency managers and that frequent, regular
communication is key for managers to inform staff and other stakeholders of their commitment
to achieve the agency’s goals and to keep those goals in mind as they pursue their day-to-day
activities. As discussed later in this report, we concluded that the FDIC sufficiently
communicates performance information internally, but could increase accountability for, and
transparency of, performance information to external stakeholders. DOF posts the CPOs and a
summary of year-to-date cumulative results on the FDIC’s internal Web site. DOF also provides
a detailed quarterly performance review of the CPOs to the COO, CFO, FDIC Chairman, and
internal FDIC Board Members.
During his tenure, the former FDIC Chairman issued a Quarterly Letter to Stakeholders, which
included key indices for the Corporation such as insurance fund levels and financial institutions
supervised, as well as summary information about the status of selected CPOs. This Quarterly
Letter was posted to the FDIC’s external Web site. The FDIC last posted this publication to the
FDIC Web site in October 2005 following the former Chairman’s departure from the FDIC.
Finally, DSC’s balanced scorecard is available to all DSC staff and provides detailed information
about strategic objectives, success indicators, and performance targets to provide a
comprehensive view of business operations at the national, regional, and territory level. DOF’s
balanced scorecard is also available to FDIC employees and presents performance
measurement information about DOF operations, strategies, and initiatives.
FDIC Managers’ Use of Performance Information Varies
We found that FDIC managers use some performance measurement processes more than
others to make management decisions and to manage their programs. We interviewed eight
senior managers to determine whether they find GPRA, CPO, and balanced scorecard
measures useful and to what extent they use performance information in managing their day-today
operations. Most senior managers commented that the government-mandated GPRA
measures were not as useful as the internally-focused CPOs. In addition, while some divisions
and offices have developed scorecards to manage their daily operations, implementation of
scorecard efforts has been uneven.
Usefulness of GPRA
GAO has reported that the benefit of collecting performance information is only fully realized
when this information is actually used by managers to bring about desired results.[ 7 ] Most senior
FDIC managers reported that they do not find GPRA APP information useful and do not use
GPRA annual performance goals to manage their programs. One FDIC senior manager told us
that FDIC participates in GPRA primarily because the Corporation is statutorily required to do
so. Further, a representative from one business line division told us that GPRA is not used to
manage and monitor division programs. Instead, this representative uses the division’s
balanced scorecard to manage programs.
Although many FDIC managers understand and use results-oriented management concepts in
their day-to-day activities, such as strategic planning and performance measurement, they do
not always connect these concepts to the requirements of GPRA. In one instance, an FDIC
manager was not aware of the annual performance goals for his organization. There also
appeared to be uncertainty among some FDIC managers on whether certain goals were GPRAor
CPO-related.
Notwithstanding, several managers stated that they do use annual performance goals to
manage their operations. For example, one representative from a business line division
reported using an annual performance goal to manage operations. In addition, a senior
manager from another business line division stated that it varied from year-to-year as to which
goals his office used to manage day-to-day operations. For 2006, his office managed by the
CPOs. However, he stated that there was alignment between those CPOs and the annual
performance goals.
A senior DOF official noted that although Corporation managers may respond that they do not
use annual performance goals, the information underlying these goals is critical to the
achievement of the FDIC’s mission and thus very important to Corporation managers. For
example, the indicator and target for the Supervision Annual Performance Goal #1 requires the
FDIC to conduct 100 percent of required risk management examinations in accordance with
statutory requirements. In the senior DOF official’s view, FDIC’s planning process is fully
integrated with the internal management processes of the business divisions, and the annual
performance goals reflect the mission priorities of the FDIC to such an extent that FDIC
managers do not see the annual performance goals as being separate from FDIC mission
priorities.
Corporate Performance Objective Process
Since 2003, the FDIC has structured the CPOs consistent with the corporate priorities of
Stability of the industry and the insurance funds, Sound Policy positions supported by
substantive research and led by comprehensive deposit insurance reform legislation, and
Stewardship of the Corporation and insurance funds to ensure the FDIC accomplishes its
mission in the most efficient and effective manner possible. In 2007, the FDIC changed the
corporate priorities to Depositor Protection, Mission Support, and Resource Management.
FDIC management has characterized the CPOs as “stretch goals” that the Corporation may not
always meet and that the Corporation prefers to report internally. According to the prior FDIC
Chairman who established the CPO process, these objectives must continually challenge the FDIC to achieve more, to reach for the higher bar, and to strive to change things, even in an
environment in which the FDIC cannot fully control the outcome, in order to be the best
organization it can be.
CPO Development and Reporting Process: FDIC executives propose CPOs for the upcoming
budget year at an annual Planning and Budget Conference held in late September. This year
we noted that the attendees were directed to develop CPOs that: (1) represented the priorities
of the Corporation, (2) were balanced initiatives that were important for all employees and not
just a few, and (3) were measurable. The executives’ discussions were open, constructive, and
strategic in nature. The proposed CPOs appeared to represent the most important issues
facing the Corporation. The attendees classified some proposed CPOs that were narrowly
focused objectives as divisional initiatives. The executives were receptive to developing CPOs
that could be measured and were challenging, yet attainable.
Based on the Planning and Budget Conference discussions and guidance from the Chairman,
DOF’s CPPM section revised the proposed 2007 CPOs and redistributed them to division and
office directors for final review. DOF presented the proposed CPOs and FDIC budget to the
Chairman and the Board of Directors for review in November. The Chairman approved the
CPOs in early December at the same time that the Board approved the FDIC’s budget.
Once approved, the Corporation determines the divisions and offices responsible for the CPOs
and identifies detailed action items and milestones necessary to achieve the objectives.
Divisions and offices are required to submit performance results to DOF on a quarterly basis.
DOF prepares a corporate quarterly performance report on program performance results based
on the information submitted by the divisions and offices. This report provides the basis for a
formal Quarterly Performance Review that the COO and CFO conducts with all division and
office directors.
FDIC Manager Views on the Usefulness of CPOs: Virtually all of the senior FDIC managers
that we interviewed responded that the CPO process was effective and each manager used
these measures to manage their operations. However, one manager thought that the CPO
process was too project-oriented and not program-oriented. This manager noted that under the
CPO process, executives report that their divisions completed a project but not whether the
project made the operation better.
Further, according to FDIC Circular 2200.1, Management and Executive Compensation,
Incentive, Benefits, and Performance Management Programs, dated October 13, 2005,
executive manager pay and performance is directly tied to the Corporation’s level of success in
meeting corporate goals and objectives and is one element considered by the Chairman at the
end of the year in establishing an annual pay increase and bonus pool for executive and
corporate managers. This connection to executive pay creates an additional incentive to
achieve positive performance results.
Opportunities for Streamlining CPO Reporting: The number of CPOs and underlying initiatives
has fluctuated each year. The FDIC has significantly reduced the number of CPOs and
performance initiatives since 2004. For 2006, the FDIC established 15 CPOs with 36
associated initiatives to accomplish the objectives. For 2007, the Chairman approved 17 CPOs
with 81 initiatives, in the following corporate priorities:
- Depositor Protection: 6 CPOs and 33 related CPO initiatives
- Mission Support: 6 CPOs and 28 related CPO initiatives
- Resource Management: 5 CPOs and 20 related CPO initiatives.
Several managers commented that reporting on the CPO initiatives can be a burdensome and
arduous process. The quarterly report is a lengthy document that was approximately 50 pages
during the 2006 quarterly reporting process. DOF is researching the possibility of a Web-based
technical solution to improve the reporting process. Because the number of CPO initiatives has
more than doubled for 2007, performance reporting will likely be more intensive.
The FDIC’s Implementation of Balanced Scorecards
Certain FDIC divisions and offices have implemented balanced scorecards to measure
performance against meaningful quality, timeliness, and customer service benchmarks. These
divisions and offices are at varying levels in implementing their scorecards. Further, some
divisions and offices have opted not to develop scorecards. While there needs to be flexibility in
scorecard design to meet individual divisional needs, the COO and CFO could provide more
high-level corporate guidance on the appropriate use of scorecards and how scorecards should
integrate with the FDIC’s other performance measurement processes.
Scorecards are intended to improve current performance measurement systems by providing
alternatives to managing organizational performance exclusively through financial measures.
Scorecards should be aligned with an entity’s goals and strategies by developing a strategy
map of the organization’s strategic objectives.
DSC, in particular, uses the balanced scorecard to monitor and manage day-to-day operations
at regional and field offices. In 2003, DSC implemented its initial scorecard which focused
primarily on operational measures such as average hours per exam. In November 2005, DSC
implemented its new balanced scorecard that combines internal productivity with other
important factors to create a balanced approach focused on strategic objectives.
As part of a 2005 CPO, OERM established a project plan for divisions to develop and pilot test a
multi-dimensional scorecard to measure performance. The project plan stated that it was
designed to support the development of a corporate-wide balanced scorecard to be operational
by September 30, 2005.[ 8 ] OERM also developed parameters for divisions to follow in developing
their scorecards:
- divisions were to follow the balanced scorecard approach in developing their scorecards;
- division scorecards were to be aligned with corporate and division goals and strategies;
- divisions were to produce their scorecard on a monthly basis, though some measures could be reported on a quarterly or yearly basis;
- divisions were to strive to measure outcomes, rather than just outputs and efficiency;
- divisions were not required to link scorecard results to employee pay in 2005; and
- divisions did not need to develop or acquire new software to automate their scorecards.
We reviewed the 4th quarter 2005 corporate CPO performance report and noted that the OERM
project manager made a presentation to senior management for the corporate-wide scorecard
program. This presentation suggested the Corporation integrate the scorecards with other
performance measurement processes. For example, the scorecard would consist of a combination of various measures/targets that are included in the FDIC APP and certain
initiatives that are included as CPOs.
The 4th quarter corporate CPO performance report also indicated the divisional scorecard
initiative was completed as of October 31, 2005 and that no corporate wide scorecard program
would be pursued at that time. OERM stated that senior executive management has not made
the scorecard mandatory for all FDIC divisions and offices. As a result, the Corporation’s
divisions are at various stages in implementing balanced scorecards as shown in Table 1.
Table 1: Status of Scorecard Efforts by Division
Division/Office |
Status |
DSC |
Balanced scorecard implemented in regional and field offices in 2003 and revised in
November 2005. |
DIR |
Balanced scorecard has been implemented in certain branches of DIR. |
DRR |
Balanced scorecard initiative is in process. Implementation expected in 2007. |
DOF |
Balanced scorecard implemented in September 2005. |
DIT |
Balanced scorecard has been implemented. |
DOA |
Balanced scorecard implemented in one branch of DOA. |
Legal |
Has not implemented a balanced scorecard. Efforts have been made to establish a draft scorecard. |
OERM |
Balanced scorecard has been implemented. |
Source: Results of OIG interviews with division representatives.
Managers in those divisions that implemented the scorecard characterized it as an important
management tool. However, to date, scorecard implementation has been uneven and
development of a corporate-wide balanced scorecard is uncertain. While there needs to be
flexibility in scorecard design to meet individual divisional needs, the COO and CFO could
provide greater structure to guide division and office scorecard efforts.
Recommendation
We recommended that the Director, OERM:
- Coordinate with DOF, the COO, and the CFO, to provide high-level direction on the
appropriate use of balanced scorecards, including addressing whether there is a need
for a corporate-wide scorecard and how divisional and office scorecards should integrate
with the FDIC’s other performance measurement processes.
The FDIC’s Government Performance and Results Act Process Could
Be Enhanced
We concluded that the FDIC is technically meeting the requirements of GPRA by developing a
strategic plan, APP, and PAR, and these documents generally contain information required by
the GPRA legislation and OMB. Notwithstanding, we identified opportunities to enhance the
APP and PAR to increase transparency, improve corporate accountability, and provide
information to aid in congressional decision-making, all key purposes of the GPRA legislation,
by:
- establishing and documenting a process for determining what initiatives should be reported in the APP and PAR,
- including goals for the FDIC’s support divisions and offices,
- establishing outcome performance measures instead of output measures,
- enhancing the effectiveness of FDIC program evaluations.
Our observations on these areas are discussed in the following sections.
Legislative Requirements for GPRA Plans and Performance Reports
We evaluated the FDIC Strategic Plan, APP, and PAR against applicable legislative
requirements.
Strategic Plan: Section 3 of GPRA requires the FDIC to develop and submit to Congress and
OMB by September 30, of each year, a 5-year strategic plan covering the FDIC’s activities that
are listed in the annual budget of the United States. We reviewed the FDIC’s Strategic Plan to
determine whether it contained the following required elements:
- A comprehensive agency mission statement;
- Agency-wide long-term goals and objectives for all major functions and operations;
- Approaches (or strategies) and the various resources needed to achieve goals and objectives;
- A description of the relationship between long-term goals and objectives and the annual performance goals;
- An identification of the key factors, external to the agency and beyond its control that could significantly affect achievement of the strategic goals; and
- A description of how program evaluations were used to establish or revise strategic goals and a schedule for future evaluations.
We determined that the FDIC’s Strategic Plan addressed most of the above items. However,
the strategic plan did not clearly describe how program evaluations were used to establish or
revise strategic goals or provide a schedule for future evaluations. This issue is discussed in
greater detail later in this report.
Annual Performance Plan: Section 4 of GPRA requires that the FDIC prepare and submit to the
OMB an APP that covers each program activity set forth in the FDIC’s budget. The FDIC’s
2006 APP includes strategic goals, strategic objectives, and annual performance goals. The
goals address the FDIC’s major business lines of Insurance, Supervision, and Receivership
Management as outlined in Table 2.
Table 2: Overview of 2006 Annual Performance Plan Goals and Objectives
Program |
Strategic Goals |
Strategic Objectives |
Performance Goals |
Insurance
|
1
|
3
|
5
|
Supervision
|
2
|
3
|
10
|
Receivership Management
|
1
|
3
|
4
|
Total
|
4
|
9
|
19
|
Source: OIG Review of Annual Performance Plan.
We reviewed the 2006 APP and confirmed that it contained the following six key elements
required by GPRA:
- Goals established to define the level of performance to be achieved;
- Goals expressed in an objective, quantifiable, and measurable form;
- A description of the operational processes, skills and technology, and the human capital information or other resources required to meet the performance goals;
- An identification of the performance indicators to be used in measuring and assessing the outputs, service levels, and outcomes of each program activity;
- A description of the basis for comparing actual program results with the established goals; and
- A description of the means to be used to verify and validate measured values.
In addition, the APP has a separate section, Effective Management of Strategic Resources,
which discusses management of financial resources, human capital management, and
information technology resources.
Performance and Accountability Report: GPRA also requires that the FDIC submit a PAR to the
President and the Congress that compares actual performance to the annual performance
goals. We reviewed the FDIC’s 2005 PAR and confirmed that it substantially addressed the
following required items:
- A discussion of the success of achieving the performance goals of the fiscal year;
- An evaluation of the [upcoming] APP relative to the performance achieved toward the
performance goals in the fiscal year covered by the report;[ 9 ] and
- An explanation, when a performance goal has not been met, why the goal was not met;
plans and schedules for achieving the goal; if a goal is impractical or infeasible, why that
is the case and what action is recommended; and a summary of findings for those
program evaluations completed during the fiscal year covered by the report.
Alignment and Transparency of Performance Measures
Although certain CPOs are aligned with APP goals and FDIC strategic objectives, a number of
important FDIC issues with associated CPOs that would likely be of interest to the public and
the Congress are not addressed in the APP. We concluded that FDIC could benefit from a
more structured, documented process for determining which initiatives should be reported in the
APP, as CPOs, or in both sets of performance measures. We compared the 2006 CPOs with
the 2006 GPRA performance goals and found that the measures within the two separate sets of performance measurement structures were aligned (linked) to some degree but were not
aligned in a number of cases as shown in Table 3.
Table 3: Alignment of 2006 CPOs and APP
FDIC 2006 Strategic Priorities |
Number of FDIC 2006 CPO Initiatives |
CPO Initiatives Addressed in APP Annual Performance Goals, Indicators, Targets, Means & Strategies |
CPO Initiatives Addressed in APP Effective Management of Strategic Resources Section |
CPO Initiatives Addressed in APP Appendix |
CPO Initiatives Not Addressed in APP |
Sound Policy
|
8
|
6
|
1
|
0
|
1
|
Stability
|
15
|
13
|
0
|
2
|
0
|
Stewardship
|
13
|
1
|
11
|
0
|
1
|
Total
|
36
|
20
|
12
|
2
|
2
|
Source: OIG review of 2006 CPOs and 2006 APP.
As shown, the FDIC addressed 12 of the CPOs in the Effective Management of Strategic
Resources narrative section of the APP; however, this section does not include goals,
indicators, or targets for these initiatives.
We also reviewed the 2007 CPOs and draft 2007 APP and identified several important issues
for which the FDIC had established CPOs that were not addressed in the APP. In our view,
many of the 2007 CPOs involve initiatives that would likely be of interest to the public or the
Congress. Table 4 presents a comparison of important issues facing the FDIC and whether
they are included in the draft 2007 APP or 2007 CPOs.
Table 4: Analysis of 2007 Performance Measures for Important Issues Facing the FDIC
Issue |
Addressed in Draft 2007
Corporate Performance
Objectives? |
Included in Draft 2007 Annual
Performance Plan as an Annual
Performance Goal (APG),
Indicator, or Target? |
New Deposit Insurance Assessment System Implementation
|
YES
|
YES
Insurance Program (APG 1.3-1)
|
FDIC Actions to Address Risks Posed by
Different Types of Industrial Loan Companies
|
YES
|
NO
|
Restructuring the Large Insured Depository Institution Analysis Program
|
YES
|
NO
|
Enhancing the FDIC’s Ability to Insure, Supervise, & Resolve Large/Complex Insured Institutions
|
YES
|
PARTIALLY ADDRESSED Insurance Program (APG 1.1-1) 1 of 6 CPO initiatives addressed
|
Protecting the U.S. Banking System Against Terrorist Financing, Money Laundering, & Other Financial Crimes
|
YES
|
PARTIALLY ADDRESSED Supervision Program (APG 2.1-1) 1 OF 3 CPO initiatives addressed
|
Expanding Program and Strategy Implementation to Encourage & Promote Broader Economic Inclusion in U.S. Banking System
|
YES
|
YES Supervision Program (APG 3.2-2) 4 of 6 CPO Initiatives addressed
|
Reducing Regulatory Burden While Maintaining Consumer Protection & Safety & Soundness Safeguards
|
YES
|
NO
|
Pursuing Research Agenda to Identify, & Assist in Managing/ Mitigating Current & Emerging U.S. Banking System Risks
|
YES
|
PARTIALLY ADDRESSED
Insurance Program (APG 1.2-2)
1 of 6 CPO Initiatives addressed
|
Developing Effective Succession Management Strategies
|
YES
|
NO *
|
Ensuring Necessary Workforce Skills to
Effectively Address Emerging Safety &
Soundness & Compliance Risks
|
YES
|
NO *
|
Strengthening Data Security & Privacy
Protections Including a Coordinated Data
Security Approach with Insured Financial
Institutions
|
YES
|
NO *
|
Source: OIG review of Draft 2007 Annual Performance Plan and 2007 CPOs.
* These issue areas were discussed in the Effective Management of Strategic Resources section of the APP;
however, this section does not include goals, indicators, or targets.
As discussed earlier, the CPOs are internal goals established annually through a dialogue
between the Chairman and FDIC senior executives. Consequently, information about the CPOs
or related performance results is not distributed publicly outside of the FDIC. According to a
CPPM representative, CPOs are intended to be challenging, may not always be readily
achievable, and for this reason the Corporation prefers to report on CPOs internally.
We discussed the issue of transparency with several FDIC senior level executives. These
executives stressed that the internal nature of the CPOs was critical to the success of this
performance measurement effort. One executive noted that CPOs are often new initiatives for
the FDIC that are still in the developmental stage and that are not ready for public consumption.
This executive noted that most of these initiatives result in notices of proposed rulemaking
through the Federal Register which achieves the goal of transparency and provides the public
with a formal opportunity to comment on proposed initiatives, at the point that the FDIC is ready
to unveil those issues publicly.
In order to strike the right balance between those organizational goals that are made public and
those that are kept internally, the FDIC could develop a documented process for determining
whether initiatives warrant recognition under GPRA, should be measured as CPOs, or should
be included in the APP and the CPOs. Ideally, this process would include criteria or
considerations for evaluating whether to measure the initiative internally or externally. Such
criteria could include whether: an initiative impacts multiple agencies, an initiative would spark
Congressional or public interest, or an initiative is still in the developmental stages.
Recognition of Mission Support Activities and Operations
GPRA requires that the FDIC prepare and submit to the OMB an APP that covers each program
activity set forth in the FDIC’s budget. OMB Circular A–11 also gives agencies the discretion to
omit support-type activities and operations. The FDIC’s budget includes line items for its
mission-related activities of Insurance, Supervision, and Receivership Management and other
program activities such as general and administrative and working capital outlays. The general
and administration category includes expenses for human capital and information technology.
While some of the FDIC’s 2006 annual performance goals address how the FDIC will use
human capital to achieve individual goals, the Corporation has not established separate goals
and objectives for mission support initiatives. Rather, the FDIC's APP goals and objectives are
limited to business line (DSC, DIR, DRR) areas and thus, do not include goals or objectives
related to areas such as human capital (e.g., succession planning or the corporate employee
program) or system development efforts (e.g., the New Financial Environment, Central Data
Repository, or the Claims Administration System).[ 10 ] Moreover, the FDIC’s APP does not
include goals or objectives related to any of the President’s Management Agenda initiatives
(e.g., human capital, competitive sourcing, integration of budget and performance information,
etc…). The FDIC has a number of CPOs and related initiatives for many of these mission
support activities.
A senior DOF official noted that OMB Circular A-11 allows agencies discretion in including
mission support activities and operations in agency strategic plans and that the FDIC is only
required to report those goals and objectives related to the major mission activities of Insurance,
Supervision, and Receivership Management. We confirmed that Section 210 of OMB Circular
No. A-11, Preparing and Submitting a Strategic Plan, allows such discretion.
In this regard, in June 2000, the President issued a memorandum to the heads of executive
departments and agencies detailing actions to further improve the management of human
capital. Among other things, the memorandum directed agencies to clearly state specific
human capital management goals and objectives in their strategic and annual performance
plans.
We performed research to determine to what extent other federal agencies included goals and
objectives for their support functions or organizations in their annual performance plans. Table
5 presents the results of our review of six federal agencies’ APPs.
Table 5: Goals and Objectives Related to Mission Support Functions at Other Agencies
Agency Reviewed |
Coverage of Support
Functions |
Comments |
Federal Reserve Board
|
YES
|
Goals and objectives related to planning and budget, human capital, and diversity.
|
Securities and Exchange Commission
|
YES
|
Goals and measures related to human capital, IT, and SEC’s financial statements.
|
Pension Benefit Guaranty Corporation
|
YES
|
Objectives related to human capital strategies, effective use of IT tools, and financial management systems.
|
Social Security Administration
|
YES
|
Objectives and measures related to financial statements and human capital.
|
Federal Communication Commission
|
YES
|
Goals and objectives related to human capital strategies.
|
Environmental Protection Agency
|
NO
|
No goals or objectives identified for support-type functions.
|
Source: OIG research of agency APPs and PARs.
We believe that the FDIC could make its APP and PAR more informative and provide greater
transparency about internal FDIC operations to the Congress and the public by including goals
and objectives related to key initiatives within the divisions of Administration, Finance, and Information Technology. These divisions contribute to the success of the FDIC achieving its
mission and require a substantial outlay of funds.
Output Versus Outcome Measures
Performance measures can be expressed as output measures, outcome measures, or some
combination of the two to reflect the agency’s intended performance. Output goals refer to the
tabulation, calculation, or recording of activity or effort and can be expressed in a quantitative
manner. Outcome measures refer to an assessment of the results of a program activity
compared to its intended purpose and are generally qualitative in nature. GPRA performance
reporting allows agencies to use a range of performance measures but contains a specific
emphasis on the use of outcome measures.
Our assessment of the FDIC’s 2006 GPRA goals found that most of the goals are output-related
goals rather than outcome-oriented goals. Approximately 22 of 32 annual performance goals,
targets, and indicators in the 2006 APP were output measures. For example, one of the
performance goals for the Supervision business line is to “effectively meet the statutory
mandate to investigate and respond to consumer complaints about FDIC-supervised financial
institutions.” FDIC established an indicator and target that responses are provided to 90
percent of written complaints within time frames established by policy. While this goal is a
necessary and important measure, it does not measure whether responses were effective in
resolving complaints. A more outcome-oriented measure would be to assess the customer
satisfaction with the FDIC’s resolution of complaints. The FDIC began gathering such
qualitative information and could use the information in support of an outcome-oriented annual
performance goal.
The GAO acknowledged that agencies found it challenging to distinguish between outputs and
outcomes in the early implementation of GPRA.[ 11 ] GAO found that agencies were struggling
with the basic meaning of the concept of outcomes and some agencies argued that the nature
of their mission made it hard to develop measurable outcomes.
We encourage the FDIC to take steps to develop and state its performance expectations in
outcome-oriented terms when developing its annual performance goals.
Program Evaluations
We identified opportunities for adding more structure and independence to the GPRA-related
program evaluations performed by OERM to include a schedule for program evaluations,
description of the evaluation methodology performed in program evaluations, and a discussion
of how the Corporation uses performance evaluations to improve the performance
measurement process.
Although GPRA does not require federal agencies to conduct formal program evaluations,[ 12 ] it
does require agencies to (1) measure progress toward achieving their goals, (2) identify which
external factors might affect such progress, and (3) explain why a goal was not met. Strategic
plans are to describe the program evaluations that were used in establishing and revising goals
and to include a schedule for future evaluations. Agencies are to summarize program
evaluation findings in the PAR.
GAO has reported that program evaluations are important for assessing the contributions that
programs are making to results, determining factors that are affecting performance, and
identifying opportunities for improvement. In 2000 GAO reported that evaluations could:
- provide data on program results that were otherwise unavailable,
- prepare the groundwork and pilot-test future performance measures,
- validate the accuracy of existing data,
- address operational concerns about a program,
- explain the reasons for observed performance or identify ways to improve performance,
and
- assess a program’s net impact or contributions to agency results.
FDIC Circular 4010.3, FDIC Enterprise Risk Management Program, delineates that OERM is
responsible for conducting program evaluations of the Corporation’s business lines as required
under GPRA. OERM officials indicated that OMB Circular A-11, Section 230.2 (j) is the
guidance used to perform its program evaluations. Section 230.2 (j) specifies that an agency’s
annual report include a summary of the findings and recommendations of the program
evaluations completed during the fiscal year, and references question 2.6 of OMB’s
Performance Assessment and Rating Tool (PART)[ 13 ] guidance for more information on program
evaluations.
While PART is not applicable to the FDIC, PART does provide guidance and criteria for program
evaluations. The guidance conveys that program evaluations must meet four criteria: high
quality, sufficient scope, independent, and conducted on a regular basis to support program
improvements. The evaluations should apply evaluation methods that provide evidence of a
program’s effectiveness. The most significant aspect of program effectiveness is impact—the
outcome of the program, which otherwise would not have occurred without the program. The
evaluations should have sufficient scope to improve planning with respect to the effectiveness of
a program. To be independent, the evaluations should be conducted by non-biased parties with
no conflict of interest. The evaluations must also be conducted on a recurring basis based on
the needs and resources of the specific programs, but all programs should have plans to repeat
evaluations. Finally, to provide evidence of results, an evaluation must have been completed or
at minimum have produced some interim findings.
We reviewed the FDIC’s current 2005-2010 Strategic Plan, the 2006 APP, and the 2005 PAR
for discussion of program evaluations. Although the Strategic Plan made reference to a future
evaluation schedule, there was no detailed discussion of the program evaluations intended to
be completed over the planning period, and the plan did not address how the FDIC intended to
use evaluations to establish new or to revise existing strategic goals, as envisioned by GPRA.
The 2006 APP Appendix provided a summary discussion of the purpose of program evaluations
and areas reviewed in 2005 as well as areas planned for review in 2006. The 2005 PAR
included a summary of findings on DSC program evaluations.
Additionally, we reviewed OERM’s program evaluation reports for 2003 through 2005 and
interviewed OERM personnel to determine how program evaluations were used for performance
reporting and what circumstances led OERM to conduct these evaluations. OERM officials
characterized program evaluations as an ongoing management process where OERM participates with the driver divisions’ internal review staff to review business line programs.
OERM considers joint participation critical to fully understanding the program being evaluated
and contends that it has no impact on OERM’s independence in conducting the evaluations.
We confirmed that this description of OERM’s program evaluation approach was consistent with
the 2006 APP discussion of the program evaluation function.
Evaluations conducted by the program itself (e.g., program internal review groups) are generally
not considered independent under PART provisions. PART guidance states that for evaluations
to be independent, nonbiased parties with no conflict of interest, for example, GAO or an
Inspector General Office, should conduct the evaluation. OMB guidance also recognizes a
contracted third party or independent program evaluation office as possibly being sufficiently
independent, subject to examination on a case-by-case basis. Although OERM views the joint
program evaluations with internal review staff to be independent, such arrangements may not
provide the independence that OMB envisioned.
We also concluded that OERM could add more structure to its program evaluation efforts. First,
we noted that OERM does not develop an annual program evaluation schedule or staffing
requirements for its program evaluation work. OERM told us that it assigns one employee
25 percent of the time during the year to perform program evaluations. OERM issues a
program evaluation report covering one of the Corporation’s three business lines each year.
Second, OERM could do more to define the scope and methodology of the evaluation work
performed. For example, the 2005 DSC evaluation report methodology section listed
documents reviewed but did not include a description of procedures performed. The report
provided information about how DSC processes worked but did not conclude on how well these
processes were operating or whether they were achieving desired outcomes.
Finally, although program evaluations of business line program areas are repeated on a cyclical
basis, we could not readily determine how OERM’s program evaluations were used to improve
program performance. We observed that the program evaluation reports contained no
recommendations for program improvement. OERM indicated that it identifies program issues
and resolves these issues through meetings without reporting formal findings and
recommendations. For example, OERM officials stated that, as part of OERM’s 2006 program
evaluation effort, it charted and identified new and changed processes associated with Deposit
Insurance Reform as well as initiated the development of key procedures for the Division of
Insurance and Research to complete to implement the reforms.
In technical comments to our draft report, DOF indicated that our description of OERM’s
program evaluation approach was too narrow. DOF noted that the 2007 APP provides a clear
and succinct explanation of the approach to program evaluation currently being pursued by
OERM. We confirmed that the 2007 APP contains the following expanded description of
OERM’s program evaluation approach.
The FDIC’s Office of Enterprise Risk Management (OERM) has primary responsibility for the
program evaluation function within the Corporation’s program. It carries out this role in
several ways:
- It performs studies and evaluations of selected programs, making appropriate
recommendations to improve their operational effectiveness and monitoring the
implementation of accepted recommendations.
- It reviews the results of program studies and evaluations undertaken by other
independent organizations, such as the GAO and the Office of the Inspector General
(OIG), to identify key recommendations to improve the operational effectiveness of
these programs and monitor the implementation of accepted recommendations.
- It reviews the results of program studies and evaluation studies undertaken by
independent internal review units within selected FDIC divisions to identify key
recommendations to improve the operational effectiveness of these programs and
monitor the implementation of accepted recommendations. In some cases, it may
also partner with such units to conduct joint program evaluations.
As discussed in greater detail in the following section of our report entitled, Corporation
Comments and OIG Evaluation, it is our view that the expanded approach described in the 2007
APP takes into account activities that are more closely related to OERM’s responsibilities under
the FDIC’s internal control program. Further, these efforts have primarily been focused on
mission-support activities for which the Corporation has elected not to establish annual
performance goals. Accordingly, we concluded that recommendation 3 stated below still
warranted consideration by management.
Recommendations
We recommended the Director, Division of Finance:
- Enhance the Annual Performance Plan and Performance Accountability Report to
increase transparency, public accountability, and information to aid in congressional
decision-making by developing a process and procedures for determining which
corporate performance objectives or initiatives deserve GPRA recognition.
We recommended the Director, Office of Enterprise Risk Management:
- Take steps to add greater independence and structure to GPRA program evaluation
efforts. Such structure could include developing an annual program evaluation
schedule, defining the scope and methodology of evaluation procedures performed, and
reporting recommendations for program improvements.
CORPORATION COMMENTS AND OIG EVALUATION
The DOF and OERM Directors, with the concurrence of the Deputy to the Chairman and CFO
(hereinafter referred to as management) provided a written response, dated April 5, 2007, to a
draft of this report. The response is presented in its entirety in Appendix II. In addition, DOF
management provided clarifications and editorial comments, which we have incorporated in our
final report, where appropriate. The Corporation’s response noted that management carefully
considered the three recommendations presented in the report, but after discussions with the
Chairman, did not plan to adopt any of the specific recommendations presented. The response
further indicated that management has taken or plans to take certain actions that may address
some of the concerns underlying OIG recommendations.
Recommendation 1 involved developing a process and procedures for determining which
CPOs deserved GPRA recognition to increase transparency, public accountability, and
information to aid in Congressional decision-making. Management responded that it has always
made an intentional effort to ensure that the two sets of performance measures were
appropriately aligned and that the Corporation’s senior leaders have been specifically asked
each year to consider whether key CPOs should have a parallel APG in the Corporation’s
Annual Performance Plan. The response also identified a potential modification that may
enhance this process. Historically, the APGs and CPOs were presented to the Chairman for
consideration at separate times. Going forward, the proposed 2008 CPOs and APGs will be
presented to the Chairman simultaneously in order to facilitate a more holistic review of the two
sets of proposed performance measures and to ensure that they are appropriately aligned.
Management believes that this change in the timing of the presentation of the APGs to the
Chairman is consistent with the OIG’s concerns about improved alignment and transparency of
the two sets of performance measures.involved developing a process and procedures for determining which
CPOs deserved GPRA recognition to increase transparency, public accountability, and
information to aid in Congressional decision-making. Management responded that it has always
made an intentional effort to ensure that the two sets of performance measures were
appropriately aligned and that the Corporation’s senior leaders have been specifically asked
each year to consider whether key CPOs should have a parallel APG in the Corporation’s
Annual Performance Plan. The response also identified a potential modification that may
enhance this process. Historically, the APGs and CPOs were presented to the Chairman for
consideration at separate times. Going forward, the proposed 2008 CPOs and APGs will be
presented to the Chairman simultaneously in order to facilitate a more holistic review of the two
sets of proposed performance measures and to ensure that they are appropriately aligned.
Management believes that this change in the timing of the presentation of the APGs to the
Chairman is consistent with the OIG’s concerns about improved alignment and transparency of
the two sets of performance measures.
The response also noted that the Chairman is giving consideration to the OIG’s suggestion that
the FDIC resume publication of the quarterly Letters to Stakeholders to increase transparency.
Regarding the inclusion of mission support activities and operations in the APP, management
noted that the FDIC will soon be reporting to the Office of Personnel Management on a variety
of performance metrics as part of a human capital planning process that is being established by
regulation. Thus, in management’s view, establishing and reporting on human capital-related
annual performance goals would be unnecessarily duplicative.
The changes proposed by management address the intent of our recommendation by
establishing a process for determining what initiatives warrant GPRA recognition and should
increase the transparency of CPO results. As a result, we consider the recommendation
closed. However, we encourage the FDIC to consider documenting this process in FDIC
Circular 4100.4, Corporate Planning and Budget Processes, dated March 21, 2007. Doing so
would help the Corporation (1) address any scrutiny it may receive from external parties
regarding which performance measures and initiatives are included in the APP and how those
determinations were made and (2) institutionalize the process so that it is more likely to be
repeated as changes occur in senior corporate leadership.
Recommendation 2 involved providing high-level direction on: the appropriate use of balanced
scorecards, whether there is a need for a corporate-wide scorecard, and how division and office
scorecards should integrate with other performance measurement processes. Management
indicated that OERM is nearing completion of a final report on the balanced scorecard pilot
project and had briefed the Chairman on the report findings. The response indicated that the Chairman had endorsed OERM’s recommendations that no corporate scorecard be pursued at
this time and that use of the scorecarding tool be optional for all divisions/offices. The Chairman
also confirmed the current requirement that all divisions and offices be required to have annual
division/office plans/performance objectives that are aligned with and supplement the corporate
goals and objectives. The response indicated that this guidance would be communicated to all
division/office directors by the COO and CFO following issuance of OERM’s report.
Although management stated that it did not plan to adopt this specific recommendation, the
actions taken by the Chairman provide the high-level direction on the appropriate use of
balanced scorecards as recommended in the draft report. Accordingly, management’s actions
meet the intent of this recommendation, and we consider it closed.
Recommendation 3 involved adding greater independence and structure to GPRA program
evaluation efforts. Management responded that it is comfortable with the current structure of
and approach to the program evaluation function and believes that this approach complies fully
with GPRA requirements and is consistent with the spirit of PART requirements for
“independent” program evaluations. The response notes that the FDIC’s approach to program
evaluations has changed over the past 2 years and OERM now relies on a broad range of
program studies and evaluations in performing the program evaluation function, including
studies by the GAO, OIG, and independent internal review units in program offices as well as
studies performed by OERM itself. Consistent with the intent of GPRA, the FDIC states that this
information is directly used to make improvements to program performance and results.
It is our view that the expanded description of OERM’s program evaluation approach that was
included in the 2007 APP also encompasses OERM’s responsibilities under the FDIC’s internal
control program. While the purpose of program evaluations and internal control reviews can
overlap, the focus of the two is generally different. Program evaluations, as described earlier,
are results-oriented and address whether agency goals and objectives are being addressed.
Internal control reviews, on the other hand, are oriented towards evaluating operations,
systems, and procedures to provide assurance that laws and regulations are being followed and
data and reporting are reliable; and to minimize exposure to waste, fraud, and mismanagement.
Moreover, OERM’s internal control-related efforts have primarily focused on mission-support
type functions—IT projects, procurement, equal employment complaint processing—for which
the Corporation has elected not to develop annual performance goals. We confirmed that
OERM’s GPRA-related work during 2006 consisted of OERM staff providing support in the
documentation and analysis of current business processes associated with the implementation
of deposit insurance reform.
After considering management’s response and clarifications, we concluded that the program
evaluation function could still benefit from greater structure and does not meet the definition of
“independent” as defined by PART guidance. While the individual internal review units play an
important role in ensuring that divisions and offices meet their goals and objectives, these units
are not organizationally independent from their parent divisions and offices. Further, we
understand that OERM will not be issuing program evaluation reports for 2006 evaluation efforts
or going forward. Rather, OERM will discuss any findings with division representatives at the
time of fieldwork and will continue to report program evaluation findings in the Corporation’s
annual Performance and Accountability Report. This action further reduces the structure,
transparency, and possibly the effectiveness of the program evaluation function.
Nevertheless, as discussed earlier, GPRA does not specifically require agencies to conduct
formal program evaluations. Thus, the FDIC’s approach to program evaluations, despite our concerns, does not violate GPRA. Accordingly, we are accepting management’s decision on
this recommendation, and we consider it closed.
OBJECTIVE, SCOPE, AND METHODOLOGY
This assignment was a follow-on evaluation from earlier work that our office performed related
to the FDIC’s corporate planning and budget process. The objective of this review was to
evaluate the FDIC’s performance measurement processes used to monitor corporate
performance. We evaluated the performance measurement processes under the Government
Performance and Results Act of 1993 (GPRA), the FDIC’s Corporate Performance Objective
(CPO) process, and the FDIC’s Balanced Scorecard Initiative. We evaluated: (1) whether the
GPRA, CPO, and scorecard measures are aligned with the FDIC’s strategic goals and
objectives, (2) FDIC management and stakeholder views on the usefulness of performance
measures in monitoring corporate performance, (3) whether there are opportunities for
streamlining or combining the FDIC’s performance measurement processes, and (4) whether
performance results are transparent to internal and external corporate stakeholders.
Scope and Methodology
We performed fieldwork in FDIC divisions located in Washington, D.C., and the DSC Dallas
Regional Office. In addition, we performed fieldwork in one office, OERM, responsible for
facilitating the implementation of the 2005 CPO initiative of developing a balanced scorecard
and performing program evaluations as part of GPRA. We performed our evaluation from
September 2006 through December 2006, in accordance with the Quality Standards for
Inspections. To accomplish our objectives, we performed the following.
- Identified and reviewed applicable laws, regulations, criteria on planning and
performance monitoring and reporting:
- the GPRA of 1993;
- OMB Circular A-11;
- FDIC Directive 4100.2, Verification and Validation Guidelines for Performance
Planning and Reporting, dated November 30, 2001;
- FDIC Draft Directive, Corporate Planning and Budget Processes; not dated or
formally issued (upon issuance of this directive FDIC Directive 4100.1, FDIC’s
2001 Corporate Planning and Budget Guide, dated November 2, 2000, will be
rescinded and superseded) ;
- FDIC Directive 4010.3, FDIC’s Enterprise Risk Management Program, dated
September 25, 2006; and
- DIT Policy Memorandum 03-009: Information Technology (IT) Performance
Measurement Program, dated September 30, 2003.
- Researched and reviewed:
- the National Performance Review’s June 1997 Benchmarking Study Report,
Serving the American Public: Best Practices in Performance Management;
- GAO Report No. GAO-06-28 dated October 2005 entitled, Performance
Budgeting: PART Focuses Attention on Program Performance, but More Can
Be Done to Engage Congress;
- GAO Report No. GAO-06-67 dated October 2005 entitled, Program Evaluation:
OMB’s PART Reviews Increased Agencies’ Attention to Improving Evidence of
Program Results;
- GAO/GGD-00-204 dated September 2000 entitled, GAO/GGD-00-204 dated September 2000 entitled, Studies Help Measure or
Explain Performance;
- GAO Executive Guide: Effectively Implementing Government Performance
and Results Act; and
- President’s Management Agenda’s report dated August 2004 entitled,The
Federal Government is Results-Oriented, a Report to Federal Employees.
- Reviewed the FDIC’s:
- 2005-2010 Strategic Plan;
- 2005 and 2006 APP and draft 2007 APP;
- 2005 PAR and draft 2006 PAR;
- 2005, 2006 CPOs and proposed 2007 CPOs;
- 2005 and 2006 Quarterly Performance Reports;
- 2006 balanced scorecards for DSC, DIR, DRR, DIT, DOA, Legal Division, and
DOF; and
- 2006 Divisional Performance Objectives for DSC, DIR, DRR, DIT, DOA, and
DOF.
- Obtained and reviewed prior related OIG evaluations:
- July 2001, Study of FDIC’s Corporate Planning Cycle, an OIG joint review with
OERM; and
- September 2005, Follow-up Evaluation of the FDIC’s Corporate Planning Cycle
(Report No. 05-032).
- Reviewed OIG’s Office of Quality Assurance and Oversight 2004 and 2005 studies and
analysis of the linkage of the FDIC’s GPRA and CPO performance goals and objectives,
and prepared a similar analysis for 2006;
- Reviewed copies of FDIC Legal Division legal opinions relating to the applicability of
GPRA to FDIC, and interviewed FDIC Legal Division representatives about those
opinions;
- Interviewed officials in DOF’s CPPM Section, to determine (1) status of the new
corporate directive on the FDIC’s planning and budgeting process and (2) whether the
CPO process would continue with the appointment of the new FDIC Chairman;
- Developed and used a standard questionnaire to conduct interviews with division and
office planning representatives;
- Interviewed planning representatives of the FDIC headquarters divisions and OERM to
gain an understanding of whether:
- GPRA, the CPOs, and balanced scorecard performance measures aligned with
the FDIC’s strategic goals and objectives;
- FDIC management and stakeholders viewed performance measures to be
useful in monitoring corporate performance;
- performance reporting was transparent to internal and external corporate
stakeholders; and
- the performance measurement processes could be streamlined or combined.
- Attended and observed the 2007 Corporate Planning and Budget Conference held on
September 28, 2006;
- Obtained and reviewed a copy of OERM’s 2005 OERM Program Evaluation report
prepared as part of GPRA, interviewed OERM staff to obtain an understanding of the
process for performing these evaluations, and assessed the extent the reviews are used
in the Strategic and APP planning processes and whether the program evaluation
process could be improved;
- Reviewed planning documents included in DOF’s Budget Send and Budget Receive for
2007; and
- Reviewed and analyzed the strategic plans, APPs, and/or PARs for selected agencies to
determine whether their performance goals addressed agency support-type operations.
Evaluation of Internal Controls
We gained an understanding of the control environment, corporate risk assessment, relevant
control activities, information and communications flow, and monitoring of internal controls by
reviewing the FDIC’s processes for (1) strategic planning, annual performance planning, and
reporting; (2) developing CPOs and related monitoring and reporting of performance results;
and (3) developing division scorecard measures and monitoring and reporting of performance
results for those scorecards. We assessed the control environment by interviewing corporate
executives as well as senior and mid-level management officials in six divisions and one office.
We reviewed applicable policies and procedures related to the FDIC’s use of performance
measures. We examined how the performance measures were defined through corporate risk
assessments. We also reviewed the flow of information to determine whether employees at all
levels of the Corporation understood their roles in achieving the corporate mission and
objectives. Finally, we reviewed the monitoring of internal controls through the assessment of
the usefulness and transparency of performance reporting. We used GAO’s Standards for
Internal Control in the Federal Government as a guide for evaluating internal controls.
|
DATE: |
April 5, 2007 |
|
MEMORANDUM TO: | Stephen M. Beard |
|
Assistant Inspector General for Evaluations |
|
THROUGH: |
Steven O. App [SIGNED] |
|
Deputy to the Chairman and
Chief Financial Officer |
|
FROM: |
Fred S. Selby [SIGNED] |
|
Director, Division of Finance |
|
|
James H. Angel, Jr. [SIGNED] |
|
Director, Office of Enterprise Risk Management |
|
SUBJECT: |
Response to Draft Report Entitled, "Evaluation of the FDIC’s
Use of Performance Measures" (Assignment No. 2006-029) |
|
We have reviewed the attached Office of Inspector General (OIG) draft report entitled
Evaluation of the FDIC’s Use of Performance Measures (Assignment No. 2006-029). We were
especially pleased to note that your evaluation confirmed that the FDIC is in full compliance
with all statutory and regulatory requirements related to the Government Performance Results
Act (GPRA); that it is effectively using the performance information derived from its various
performance management/measurement processes to make management decisions and improve
program results; and that it employs a number of recognized “best practices” in the
administration of its performance management system. As you know, we have diligently sought
to improve the integration and effectiveness of our planning and performance management
processes in recent years.
We have carefully considered the three recommendations presented in your report. After
discussing them with the Chairman, we do not plan to adopt any of the specific
recommendations presented. As indicated below, however, we have taken or plan to take
certain actions that may address some of the concerns underlying those recommendations.
-
That the Director, DOF, enhance the Annual Performance Plan and Performance
Accountability Report to increase transparency, public accountability and information to aid
in congressional decision-making by developing a process and procedures for determining
which corporate performance objectives or initiatives deserve GPRA recognition.
Management Response: During the annual senior management strategic planning
conference, the Corporation’s senior leadership works from division/office input to finalize for presentation to the Chairman both proposed corporate performance objectives (CPOs)
and proposed annual performance goals. We have always made an intentional effort to
ensure that those two sets of performance measures were appropriately aligned. To that end,
the Corporation’s senior leaders have been specifically asked each year to consider whether
key CPOs should have a parallel APG in the Corporation’s Annual Performance Plan, and
that process has almost always resulted in the addition of new APGs. In the 2007 planning
process, that effort was also extended to the Chairman’s review. In her initial review of the
2007 APGs earlier this year, Chairman Bair requested several adjustments to add items that
had been proposed for inclusion only in the 2007 CPOs, and she added one final 2007 APG
from the 2007 CPOs during her final review of the Corporation’s 2007 Annual Performance
Plan in February, just prior to its transmittal to the Congress.
However, we have identified a potential modification that may enhance our current process.
Heretofore, the APGs and CPOs have been presented to the Chairman for consideration at
separate times. Following a discussion with Chairman Bair, we have determined that the
proposed 2008 CPOs and APGs will be presented to her simultaneously in order to facilitate
a more holistic review of the two sets of proposed performance measures and to ensure that
they are appropriately aligned. We believe that this change in the timing of the presentation
of the APGs to the Chairman is consistent with the OIG’s concerns about improved
alignment and transparency of the two sets of performance measures.
In addition, the Chairman is giving serious consideration to the OIG’s suggestion that the
FDIC resume publication of the quarterly Letters to Stakeholders. As the OIG’s draft report
noted, that publication improved the transparency of many of the CPO initiatives in past
years by publicly reporting their results. The FDIC’s senior leadership (including the
Chairman) continues to believe, however, that the Corporation realizes substantial benefits
from the maintenance of a set of CPOs that are separate from the APGs and are not broadly
publicized externally. Inclusion of all “significant” initiatives in the Annual Performance
Plan would undermine the basic concept of “stretch” objectives that challenge the FDIC to
pursue many important change objectives that would not be appropriate for inclusion in that
document.
We also discussed with the Chairman the OIG’s suggestion that APGs be included in the
Annual Performance Plan for major human capital and other support initiatives, but have
decided to continue with our present approach of discussing those initiatives in the
“Effective Management of Strategic Resources” section of the plan without establishing
specific performance goals, indicators, or targets. As the draft report notes, this approach is
in full compliance with all regulatory requirements. Moreover, in the human capital area,
the FDIC will soon be reporting to OPM on a variety of performance metrics as part of a
Human Capital planning process that is being established by OPM regulation and will be
mandatory for all covered agencies. The Annual Performance Plan will continue to address
in narrative form the initiatives covered in more detail in the Human Capital Plan, but
establishing and reporting on APGs in both venues would be unnecessarily duplicative.
-
That the Director, OERM, coordinate with DOF, the COO, and the CFO to provide highlevel
direction on the appropriate use of balanced scorecards, including addressing whether
there is a need for a corporate-wide scorecard and how divisional and office scorecards
should integrate with the FDIC’s other performance measurement processes.
Management Response: OERM is nearing completion of a final report on the balanced
scorecard pilot project initiated in 2005. It will note that there were varying degrees of
applicability, benefits, enthusiasm, and success for scorecarding among the various divisions
that were invited to participate in the pilot. The Chairman was recently briefed on these
findings and endorsed OERM’s recommendations that no corporate scorecard be pursued at
this time and that use of the scorecarding tool be optional for all divisions/offices. In lieu of
a required scorecard, the Chairman confirmed the current requirement that all divisions and
offices are required to have annual division/office plans/performance objectives that are
aligned with and supplement the corporate goals and objectives (APGs and CPOs). This
guidance will be communicated to all division/office directors by the COO and CFO
following issuance of OERM’s report.
-
That the Director, OERM, take steps to add greater independence and structure to GPRA
program evaluation efforts. Such structure could include developing an annual program
evaluation schedule, defining the scope and methodology of evaluation procedures
performed, and reporting recommendations for program improvements.
Management Response: The FDIC is comfortable with the current structure of and
approach to the program evaluation function and believes that the way the function is being
administered complies fully with GPRA requirements and is consistent with the spirit of
PART requirements for “independent” program evaluations (although the FDIC is not
formally subject to PART requirements). The FDIC’s approach to program evaluations has
changed over the past two years. The current approach is articulated very clearly in
Appendix C to the 2007 Annual Performance Plan that was recently submitted to the
Congress and OMB. As indicated in that document, OERM now relies on a broad range of
program studies and evaluations in performing the program evaluation function, including
studies by the GAO, OIG, and independent internal review units in program offices as well
as studies performed by OERM itself. Consistent with the intent of GPRA, this information
is directly used to make improvements to program performance and results.
The current approach to program evaluations is not compatible with publication of a longterm
schedule of specific planned evaluation studies. As a result, the FDIC’s long-term
Strategic Plan (scheduled to be updated this year) will necessarily address program
evaluation plans at a high level. We will, however, address each year in the Corporation’s
Annual Performance Plan the studies, if any, that are planned for the year covered by the
plan and will discuss how the findings of previous evaluation studies have impacted the
establishment of new APGs or the revision of existing ones. We will also continue to report
program evaluation findings in the Corporation’s Annual Report.
We appreciate the opportunity to respond to your draft report. If you have questions or wish to
discuss our comments in more detail, please contact Thomas E. Peddicord at X26252.
|
MANAGEMENT RESPONSE TO RECOMMENDATIONS |
This table presents the management response on the recommendations in our report and the status of the recommendations as of the date of report issuance. |
|
|
Rec. Number |
|
Corrective Action: Taken or Planned/Status |
|
Expected Completion Date |
|
Monetary Benefits |
|
Resolved: [ a ] Yes or No |
|
Open or Closed [ b ] |
|
|
|
|
|
Management did not concur with this recommendation. Management
responded that it has always made an intentional effort to ensure that
the two sets of performance measures were appropriately aligned and
that the Corporation’s senior leaders have been specifically asked
each year to consider whether key CPOs should have a parallel APG
in the Corporation’s Annual Performance Plan. Management also
noted that going forward, the proposed 2008 CPOs and APGs will be
presented to the Chairman simultaneously to facilitate a more holistic
review of the two sets of proposed performance measures and to
ensure that they are appropriately aligned.
The response also noted that the Chairman is giving consideration to
the OIG’s suggestion that the FDIC resume publication of the quarterly
Letters to Stakeholders to increase transparency.
Regarding the inclusion of mission support activities and operations in
the APP, management noted that FDIC will soon be reporting to the
Office of Personnel Management on a variety of performance metrics
as part of a human capital planning process that is being established
by regulation.
The changes proposed by management address the intent of our
recommendation by establishing a process for determining what
initiatives warrant GPRA recognition and should increase the
transparency of CPO results.
|
|
N/A |
|
$0 |
|
Yes |
|
Closed |
|
|
|
|
|
Management did not concur with this recommendation. Management
indicated that OERM is nearing completion of a final report on the
balanced scorecard pilot project and had briefed the Chairman on the
report findings. The response indicated that the Chairman had
directed that no corporate scorecard be pursued at this time; that use
of scorecards be optional for all divisions/offices; and that all divisions
and offices have annual division/office plans/performance objectives that are aligned with and supplement the corporate goals and
objectives. The response indicated that this guidance would be
communicated to all division/office directors by the COO and CFO
following issuance of OERM’s report.
Although management stated that it did not plan to adopt this specific
recommendation, the actions taken by the Chairman provided the
high-level direction on the appropriate use of balanced scorecards as
recommended.
|
|
N/A |
|
$0 |
|
Yes |
|
Closed |
|
|
|
|
|
Management did not concur with this recommendation. Management
responded that it is comfortable with the current structure of and
approach to the program evaluation function and believes that this
approach complies fully with GPRA requirements and is consistent
with the spirit of PART requirements for “independent” program
evaluations.
We reiterated our concerns with the structure, transparency, and
independence of the program evaluation function, but acknowledged
that GPRA does not specifically require agencies to conduct formal
program evaluations. Thus, the FDIC’s approach to program
evaluations, despite our concerns, does not violate GPRA.
Accordingly, we accepted management’s decision on this
recommendation and we consider it closed.
|
|
N/A |
|
$0 |
|
Yes |
|
Closed |
|
|
a Resolved – |
(1) Management concurs with the recommendation, and the planned corrective action is consistent with the recommendation. |
|
(2) Management does not concur with the recommendation, but planned alternative action is acceptable to the OIG. |
|
(3) Management agrees to the OIG monetary benefits, or a different amount, or no ($0) amount. Monetary benefits are considered resolved as long as management provides an amount. |
b Once the OIG determines that the agreed-upon corrective actions have been completed and are effective, the recommendation can be closed. |
|
Last updated 05/11/2007 |
|