This is the accessible text file for GAO report number GAO-09-82 
entitled 'Results-Oriented Management: Opportunities Exist for Refining 
the Oversight and Implementation of the Senior Executive Performance-
Based Pay System' which was released on December 17, 2008. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to Congressional Requesters: 

United States Government Accountability Office: 

GAO: 

November 2008: 

Results-Oriented Management: 

Opportunities Exist for Refining the Oversight and Implementation of 
the Senior Executive Performance-Based Pay System: 

SES Performance-Based Pay: 

GAO-09-82: 

GAO Highlights: 

Highlights of GAO-09-82, a report to congressional requesters. 

Why GAO Did This Study: 

Agencies are allowed to raise pay caps for their Senior Executive 
Service (SES) members if the Office of Personnel Management (OPM) 
certifies and the Office of Management and Budget (OMB) concurs that 
their appraisal systems meet applicable criteria. As requested, this 
report examines selected agencies’ policies and procedures for (1) 
factoring organizational performance into SES appraisal decisions, (2) 
making meaningful distinctions in SES performance and (3) building 
safeguards into SES systems. Also, this report examines OPM and OMB 
oversight in certifying the pay systems through their statutory roles. 
GAO selected six agencies based on mission, structure, and number of 
career SES variations. GAO analyzed the agencies’ policies and fiscal 
year 2007 aggregate SES appraisal data and OPM guidance. 

What GAO Found: 

Factoring organizational performance into senior executive appraisal 
decisions: All of the selected agencies—the U.S. Departments of 
Defense, Energy, State, and the Treasury; U.S. Nuclear Regulatory 
Commission; and USAID—have policies in place that require senior 
executives’ performance expectations to be aligned with organizational 
results and organizational performance to be factored into appraisal 
decisions. While almost all of the agencies provided and communicated 
the importance of considering organizational performance, USAID did not 
provide its performance review board members (PRB) and other reviewing 
officials with any specific information on organizational performance 
to help inform their executive appraisal recommendations. 

Making meaningful distinctions in senior executive performance: All of 
the selected agencies have multiple rating levels in place for 
assessing senior executive performance. For the fiscal year 2007 
appraisal cycle, senior executives were concentrated at the top two 
rating levels, which raises questions about the extent to which 
meaningful distinctions based on relative performance are being made 
and how OPM applies this criterion. OPM has an opportunity to 
strengthen its communication with agencies and executives on the 
importance of using a range of rating levels when assessing performance 
while avoiding the use of forced distributions. 

Building safeguards into senior executive performance appraisal and pay 
systems: All of the selected agencies have safeguards, including higher 
level reviews of performance appraisal recommendations, PRBs, and 
transparency in communicating the aggregate results, although agencies 
varied in how they implemented such safeguards. 

While generally satisfied with OPM’s and OMB’s oversight, officials at 
the selected agencies said OPM could strengthen its communication with 
agencies and executives on how it uses the SES performance appraisal 
data and correlation between ratings and performance pay in determining 
whether agencies are making meaningful distinctions based on relative 
performance. Further communication from OPM is important in order for 
agencies to have a better understanding of how they are being held 
accountable for these certification criteria and make the necessary 
improvements to their systems to maintain certification. Further, 
senior-level officials at the selected agencies suggested options—such 
as moving to an electronic submission process and lengthening the 
certification coverage beyond 2 years once their systems are operating 
at the fully certified level—to increase the efficiency of the process. 
Moving forward, it will be important for OPM and OMB to identify ways 
to improve the certification process and make it more streamlined while 
ensuring that agencies have the guidance, tools, and training they need 
to implement effective performance appraisal and pay systems for their 
senior executives. 

What GAO Recommends: 

GAO recommends that the United States Agency for International 
Development (USAID) provide uniform organizational performance 
assessments to reviewing officials to help inform their executive 
appraisal recommendations. GAO makes several recommendations to OPM and 
OMB to strengthen its communication to agencies on certification 
decisions and to improve the efficiency of the certification process. 
USAID, OPM, and OMB generally agreed with the recommendations. 

To view the full product, including the scope and methodology, click on 
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-09-82]. For more 
information, contact Robert Goldenkoff at (202) 512-6806 or 
goldenkoffr@gao.gov. 

[End of section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

Selected Agencies Are Generally Addressing Key Areas Related to the 
Certification Criteria through Their SES Performance-Based Pay Systems, 
but an Opportunity for Refinement Exists: 

OPM and OMB Provide Oversight through Various Formats; Selected Agency 
Officials Suggested Opportunities for Refinements: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: Certification Criteria for the Senior Executive 
Performance- Based Pay System: 

Appendix III: Comments from the Department of Defense: 

Appendix IV: Comments from the Office of Personnel Management: 

Appendix V: GAO Contact and Staff Acknowledgments: 

Tables: 

Table 1: NRC's SES Position Groups with Basic Pay Ceilings and 
Resulting Bonus Amounts Based on Position Group and Performance Ratings 
for the Fiscal Year 2007 Appraisal Cycle: 

Table 2: DOD's Performance Scores and Performance Payout Shares by 
Performance Rating for the Fiscal Year 2007 Appraisal Cycle: 

Table 3: Percentage of Eligible Senior Executives Who Received Bonuses 
or Pay Adjustments and the Average Amounts at the Selected Agencies for 
the Fiscal Year 2007 Appraisal Cycle: 

Table 4: OPM's and OMB's Senior Executive Performance Appraisal System 
Certification Criteria: 

Figures: 

Figure 1: Percentage of Senior Executives by Rating Level at the 
Selected Agencies for the Fiscal Year 2007 Appraisal Cycle: 

Figure 2: Percentage of Eligible Senior Executives Who Received Bonuses 
and the Average Bonus Amounts by Rating Level at the Selected Agencies 
for the Fiscal Year 2007 Appraisal Cycle: 

Figure 3: Percentage of Eligible Senior Executives Who Received Pay 
Adjustments and the Average Pay Adjustment Amounts by Rating Level at 
the Selected Agencies for the Fiscal Year 2007 Appraisal Cycle: 

Figure 4: Percentage of Senior Executives at the Selected Agencies Paid 
According to Executive Schedule Pay Levels in 2008: 

Abbreviations: 

CHCO: chief human capital officer: 

DOD: Department of Defense: 

DOE: Department of Energy: 

EX: Executive Schedule: 

NRC: Nuclear Regulatory Commission: 

OMB: Office of Management and Budget: 

OPM: Office of Personnel Management: 

PAAT: Performance Appraisal Assessment Tool: 

PAR: performance and accountability report: 

PART: program assessment rating tool: 

PMA: President's Management Agenda: 

PRB: performance review board: 

SES: Senior Executive Service: 

SFS: Senior Foreign Service: 

SL: senior level: 

SPO: senior performance official: 

ST: scientific or professional: 

USAID: United States Agency for International Development: 

November 21, 2008: 

The Honorable Daniel K. Akaka: 
Chairman: 
The Honorable George V. Voinovich: 
Ranking Member: 
Subcommittee on Oversight of Government Management, the Federal 
Workforce, and the District of Columbia: 
Committee on Homeland Security and Governmental Affairs: 
United States Senate: 

The Honorable Byron L. Dorgan: 
Chairman: 
Subcommittee on Interstate Commerce, Trade, and Tourism: 
Committee on Commerce, Science and Transportation: 
United States Senate: 

To successfully respond to the array of complex challenges facing the 
nation--including long-term fiscal imbalance, evolving national and 
homeland security threats, and turmoil in global financial markets--the 
federal government must build the capacity to plan more strategically, 
react more expeditiously, and focus on achieving results. Strategic 
human capital management is the centerpiece of federal agencies' 
efforts to transform to meet these challenges. Senior executives need 
to lead the way in transforming their agencies to become more results- 
oriented, collaborative in nature, and customer-focused. In 2003, 
Congress and the administration modernized the performance appraisal 
and pay systems for members of the Senior Executive Service (SES) by 
requiring a clearer link between individual performance and 
pay.[Footnote 1] Specifically, agencies are allowed to raise SES basic 
pay and total compensation caps if the Office of Personnel Management 
(OPM) certifies, with the concurrence of the Office of Management and 
Budget (OMB), that their performance appraisal systems, among other 
things, link performance for senior executives to the organization's 
goals and make meaningful distinctions based on relative 
performance.[Footnote 2] 

In our past work on performance management and pay issues, we have 
reported that performance-based pay cannot be simply overlaid on most 
organizations' existing performance management systems.[Footnote 3] 
Rather, as a precondition to effective pay reform, individual 
expectations must be clearly aligned with organizational results, 
communication on individual contributions to annual goals must be 
ongoing and two-way, meaningful distinctions in employee performance 
must be made, and cultural changes must be undertaken. Most important, 
leading organizations have recognized that effective performance 
management systems create a "line of sight" showing how unit and 
individual performance can contribute to overall organizational goals 
and can help them drive internal change and achieve external 
results.[Footnote 4] Effective performance management systems that hold 
executives accountable for results can help provide continuity during 
times of leadership transition, such as the upcoming change in 
administration, by maintaining a consistent focus on organizational 
priorities. 

OPM's recently released governmentwide SES survey results found that 
senior executives across the government recognize the importance of 
linking pay to performance, with about 93 percent of the respondents 
strongly agreeing or agreeing that pay should be based on performance. 
In addition, the majority of senior executives reported that their 
salary increases and bonuses were linked to their performance ratings 
to a very great or great extent. However, senior executives recognized 
the challenge of making meaningful distinctions in performance--a key 
criterion for agencies' certification of their SES appraisal systems. 
Specifically, less than a third of senior executives governmentwide 
strongly agreed or agreed that bonuses or pay distinctions were 
meaningfully different among executives. 

At your request, this report examines selected agencies' policies and 
procedures for their career SES performance appraisal and pay systems 
in three key areas: (1) factoring organizational performance into 
senior executive performance appraisal decisions, (2) making meaningful 
distinctions in senior executive performance, and (3) building 
safeguards into senior executive performance appraisal and pay systems. 
Collectively, these three areas cover half of the nine criteria that 
are critical to appraising and rewarding senior executive performance 
that agencies must meet to be certified by OPM and OMB.[Footnote 5] In 
addition, this report examines how OPM and OMB are providing oversight 
to the certification of the senior executive performance-based pay 
system through their statutory roles. In July 2008, we testified before 
the Senate Subcommittee on Oversight of Government Management, the 
Federal Workforce, and the District of Columbia on our preliminary 
results of the selected agencies' policies and procedures for their SES 
performance-based pay systems.[Footnote 6] This report supplements the 
information provided in our July 2008 testimony. 

For this review, we selected the U.S. Departments of Defense (DOD), 
Energy (DOE), State, and the Treasury; the U.S. Nuclear Regulatory 
Commission (NRC); and the United States Agency for International 
Development (USAID) based on variations in agency mission; 
organizational structure; size of their career SES workforces to 
reflect agencies with a large, average, and small number of executives; 
and results of their SES performance appraisal systems in terms of the 
percentage of SES rated at the highest rating levels and the percentage 
that received performance awards or bonuses from fiscal years 2004 to 
2006, according to OPM's governmentwide data reports. To meet our 
objectives, we analyzed these agencies' SES performance management 
documents; analyzed OPM's guidance; interviewed cognizant senior-level 
agency officials regarding their SES performance-based pay systems and 
their views on OPM's and OMB's oversight of the certification process; 
interviewed senior-level OPM and OMB officials who oversee the 
certification review process on their oversight of the SES performance- 
based pay system and certification process; and analyzed aggregate SES 
performance appraisal and compensation data as provided by the agencies 
and comparable governmentwide data as reported by OPM for fiscal year 
2007. Appendix I contains a detailed discussion of our objectives, 
scope, and methodology. 

We checked the agency SES performance appraisal data for reasonableness 
and the presence of any obvious or potential errors in accuracy and 
completeness. We also reviewed related agency documentation, 
interviewed agency officials knowledgeable about the data, and brought 
to the attention of these officials any concerns or discrepancies we 
found with the data for correction or updating. The agency officials 
confirmed the correctness of the data or in some cases provided 
corrections to the data, which we used in our analysis. On the basis of 
these procedures, we believe the data are sufficiently reliable for use 
in the analyses presented in this report. 

The examples of the selected agencies' policies and procedures for 
their SES performance-based pay systems are not generalizable to the 
governmentwide SES population and all executive branch agencies. We did 
not assess how the selected agencies are implementing all the policies 
and procedures for their SES performance-based pay systems. We 
recognize that focusing on 1 year of appraisal data may not show the 
complete picture of how agencies are making meaningful distinctions in 
senior executive performance. An agency may have implemented a policy 
related to the study objectives even if it is not specifically 
highlighted in this report. We conducted our work from October 2007 
through November 2008 in accordance with generally accepted government 
auditing standards. Those standards require that we plan and perform 
the audit to obtain sufficient, appropriate evidence to provide a 
reasonable basis for our findings and conclusions based on our audit 
objectives. We believe that the evidence obtained provides a reasonable 
basis for our findings and conclusions based on our audit objectives. 

Results in Brief: 

All of the selected agencies require senior executives' performance 
expectations to be aligned with organizational results and are 
factoring organizational performance into appraisal decisions. While 
almost all of the selected agencies provided the organizational 
performance assessments and communicated the importance of considering 
organizational performance to individuals involved in appraisal 
decisions, USAID did not provide its performance review board (PRB) 
members and other reviewing officials with any specific information on 
organizational performance to help inform their senior executive 
appraisal recommendations. As for making meaningful distinctions in 
senior executive performance, all of the selected agencies have 
multiple rating levels in place for assessing senior executive 
performance and five of the selected agencies have tier structures or 
prescribed performance payout ranges based on the resulting performance 
rating. For the fiscal year 2007 appraisal cycle, senior executives 
were concentrated at the top two rating levels, which raises questions 
about the extent to which meaningful distinctions based on relative 
performance are being made and how OPM applies this criterion. At 
almost all of the agencies, the highest-performing executives, rated as 
"outstanding" (level 5), made up the greatest percentage of eligible 
executives receiving bonuses with the largest bonuses on average, but 
at some agencies, these executives did not make up the greatest 
percentage of executives receiving pay adjustments with the largest 
increases on average. Specifically, at Treasury, executives rated at 
levels 5, 4, and 3 on average received about the same pay adjustment 
amounts, primarily due to pay cap issues. As part of making meaningful 
distinctions in performance, OPM has emphasized to agencies through its 
certification guidance that its regulations prohibit forced 
distribution of performance ratings and that agencies must avoid 
policies or practices that would lead to forced distributions or even 
the appearance of it. OPM has not provided specific guidance to 
agencies on how to make meaningful distinctions in senior executive 
performance while avoiding the perception of forced distributions of 
performance ratings. OPM has an opportunity to strengthen its 
communication with agencies and executives regarding the importance of 
using a range of rating levels when assessing performance while 
avoiding the use of forced distributions. Communicating this 
information to agencies can help agencies begin to transform their 
cultures to one where a fully successful rating is valued and rewarded. 
All of the selected agencies have safeguards including higher-level 
reviews of performance appraisal recommendations, PRBs, and 
transparency in communicating the aggregate results, although agencies 
varied in how they implemented such safeguards. 

OPM and OMB, as applicable, provide continuing oversight by issuing 
guidance to agencies on revisions to the certification process, using 
tools and other initiatives to help assess how agencies are 
implementing their systems, providing training and forums, and 
interacting with agencies on the review of their certification 
submissions. While generally satisfied with OPM's and OMB's oversight, 
officials at the selected agencies said OPM could strengthen its 
communication with agencies and executives on how it uses the SES 
performance appraisal data and the correlation between ratings and 
performance pay in determining whether agencies are making meaningful 
distinctions based on relative performance. Further communication from 
OPM is important in order for agencies to have a better understanding 
of how they are being held accountable for these certification criteria 
and make the necessary improvements to their systems to maintain 
certification. The senior-level officials at the selected agencies also 
suggested options--such as moving to an electronic submission process 
and lengthening the certification coverage beyond 2 years once their 
systems are operating at the fully certified level--to increase the 
efficiency of the certification submission process. OPM recognizes that 
extending coverage would require a statutory change, and OPM is not 
taking a position on lengthening coverage at this time. 

To help ensure consistency and clarity in how organizational 
performance is considered in appraising executive performance, we 
recommend that the Administrator of USAID provide uniform 
organizational performance assessments to PRB members and other 
reviewing officials to help inform their appraisal recommendations for 
senior executives at the end of the performance appraisal cycle. To 
help improve agencies' understanding of certain aspects of the 
certification decisions, we recommend two areas for the Acting Director 
of OPM to take action to strengthen OPM's communication with agencies 
and executives on the importance of making meaningful distinctions in 
performance while avoiding the use of forced distributions and that a 
fully successful rating is valued and rewarded; and how it uses the SES 
performance appraisal data and the correlation between ratings and 
performance pay in determining whether agencies are making meaningful 
distinctions based on relative performance as measured through the pay 
and performance differentiation certification criteria. In addition, to 
help improve the efficiency of the certification submission process for 
agencies, we recommend that the Acting Director of OPM and Director of 
OMB explore opportunities for streamlining the certification process, 
such as electronic submissions or lengthening the full certification 
coverage beyond 2 years for agencies that received full certification. 

We provided a copy of the draft report to the Secretaries of Defense, 
Energy, State, and the Treasury; the Commissioners of NRC; the 
Administrator of USAID; the Acting Director of OPM; and the Director of 
OMB for their review and comment. DOE had no comments on the draft 
report. We received written comments from DOD and OPM, which are 
included in appendixes III and IV. NRC, OMB, State, Treasury, and USAID 
provided clarifying and technical comments, which we incorporated as 
appropriate. Regarding our recommendations, USAID, OPM, and OMB 
expressed general agreement. The Acting Director stated that OPM looks 
forward to working with agencies and OMB to find ways to further 
improve communications with the agencies concerning the certification 
process. OMB generally agreed with our assessment and recommendation 
regarding the possibilities of streamlining the certification process 
to improve efficiency and potentially extending full certification 
coverage beyond 2 years. OMB stated that it agrees with OPM that 
careful review of the newly passed law and its effect will be necessary 
before considering such an extension. Regarding our discussion of pay 
compression, OPM stated that it is not comfortable with the 
identification of tiers as a means to address SES pay compression. 
While we recognize OPM's concern about agencies' use of tiers, we are 
not recommending the use of tiers as a way for agencies to address 
future problems with pay compression and have revised the language in 
the report to clarify this point. 

Background: 

Since 2000, Congress and OPM have gradually shifted to performance- 
based pay for senior executives through legislative and regulatory 
changes. 

* In October 2000, OPM amended its senior executive performance 
management regulations requiring agencies to (1) hold senior executives 
accountable for their individual and organizational performance by 
linking performance management with the results-oriented goals of the 
Government Performance and Results Act of 1993; (2) evaluate senior 
executive performance using measures that balance organizational 
results with customer satisfaction, employee perspectives, and any 
other measures agencies decide are appropriate; and (3) use performance 
results as a basis for pay, awards, and other personnel 
decisions.[Footnote 7] While emphasizing the use of performance results 
as the basis for pay and other awards, members of the SES still 
received the annual across-the-board and locality pay adjustments. 

* In 2002, Congress raised the total annual compensation limit--from 
Executive Schedule (EX) level I to the total annual compensation 
payable to the Vice President--for senior executives and other senior 
professionals in agencies with systems that have been certified by OPM 
with OMB concurrence as having performance appraisal systems which as 
designed and applied make meaningful distinctions based on relative 
performance.[Footnote 8] The act instructed OPM and OMB to promulgate 
regulations regarding certification that, if met by an agency, would 
allow it to access the higher total compensation cap, which includes 
bonuses and other forms of compensation. 

* In 2003, Congress changed the basis for how agencies pay their senior 
executives and the overall SES pay structure.[Footnote 9] Beginning in 
January 2004, senior executives no longer received annual across-the- 
board or locality pay adjustments. Agencies are to base pay adjustments 
for senior executives on individual performance and contributions to 
the agency's performance by considering the individual's 
accomplishments and such things as unique skills, qualifications, or 
competencies of the individual and the individual's significance to the 
agency's mission and performance, as well as the individual's current 
responsibilities. In addition, the SES pay structure changed from six 
pay levels to a single, open-range pay band with a higher basic pay 
cap--EX-level III for agencies without certified appraisal systems and 
EX-level II for agencies with such systems. For calendar year 2008, the 
pay caps are $158,500 for basic pay (EX-level III) with a senior 
executive's total compensation not to exceed $191,300 (EX-level I). If 
an agency's senior executive performance appraisal system is certified 
by OPM and OMB concurs, the caps are increased to $172,200 for basic 
pay (EX-level II) and $221,100 for total compensation (the total annual 
compensation payable to the Vice President). 

To qualify for senior executive pay flexibilities, agencies' 
performance appraisal systems are evaluated against nine certification 
criteria and any additional information that OPM and OMB may require to 
make determinations regarding certification. OPM's and OMB's 
certification criteria are broad principles that position agencies to 
use their pay systems strategically to support the development of a 
stronger performance culture and the attainment of their mission, 
goals, and objectives. (See app. II for additional information on the 
certification criteria). Two levels of performance appraisal system 
certification are available to agencies--full and provisional. Through 
a law passed in October 2008, an agency's certification now lasts for 
up to 24 months, with the possibility of a 6-month extension by the OPM 
Director, rather than a calendar-year-based coverage.[Footnote 10] 
Previously, an agency's certification lasted for 2 calendar years for 
full certification and 1 calendar year for provisional certification. 

In addition to SES employees, many agencies use senior employees with 
scientific, technical, and professional expertise, commonly known as 
senior-level (SL) and scientific or professional (ST) positions. An 
agency may apply to OPM and OMB for certification of its SL/ST 
performance management system, and if its system is certified as making 
meaningful distinctions in relative performance, an agency may raise 
the total annual compensation maximum for SL/ST employees to the salary 
of the Vice President. Beginning in April 2009, the recently passed law 
allows certified agencies to raise the basic pay cap for SL/ST 
employees to EX-level II--the same maximum rate of basic pay as SES 
members, and also exempts SL/ST employees from receiving locality 
pay.[Footnote 11] Previously, SL/ST employees under certified appraisal 
systems had a maximum rate of basic pay equal to EX-level IV plus 
locality pay up to EX-level III. However, unlike the SES, their 
individual rate of pay does not necessarily have to be based on 
individual or agency performance. 

OPM has a key leadership and oversight role in the design and 
implementation of agencies' SES performance-based pay systems by 
certifying that the agencies' systems meet the certification criteria 
before they can receive the pay flexibilities.[Footnote 12] In our 
January 2007 report examining the senior executive performance-based 
pay system, we made a series of recommendations to OPM designed to 
address issues specific to the performance-based pay system, such as 
sharing best practices, tracking progress towards goals, and developing 
a timeline for issuance of certification guidance. We are following up 
on the status of these recommendations through this report. 

Selected Agencies Are Generally Addressing Key Areas Related to the 
Certification Criteria through Their SES Performance-Based Pay Systems, 
but an Opportunity for Refinement Exists: 

The selected agencies are generally addressing three key areas related 
to OPM's and OMB's certification criteria through their SES performance-
based pay systems--factoring organizational performance into senior 
executive performance appraisal systems, making meaningful distinctions 
in senior executive performance, and building safeguards into senior 
executive performance appraisal and pay systems. However, USAID did not 
provide its PRB members and other reviewing officials with any specific 
information on organizational performance to help inform their senior 
executive appraisal recommendations. 

Selected Agencies Have Policies for Factoring Organizational 
Performance into Appraisal Decisions, but USAID Did Not Provide 
Organizational Performance Information to Reviewing Officials: 

In our past work on performance management, we identified the alignment 
of individual performance expectations with organizational goals as a 
key practice for effective performance management systems.[Footnote 13] 
Having a performance management system that creates a "line of sight" 
showing how unit and individual performance can contribute to overall 
organizational goals helps individuals understand the connection 
between their daily activities and the organization's success. To 
receive certification of their systems, agencies are to align senior 
executive performance expectations with the agency's mission, strategic 
goals, program and policy objectives, or annual performance plan and 
budget priorities. While many agencies are doing a good job overall of 
aligning executive performance plans with agency mission and goals, 
according to OPM some of the plans do not fully identify the measures 
used to determine whether the executive is achieving the necessary 
results, which can affect the executive's overall performance 
appraisal. This challenge of explicitly linking senior executive 
expectations to results-oriented organizational goals is consistent 
with findings from our past work on performance management.[Footnote 
14] 

To help hold senior executives accountable for organizational results, 
beginning in 2007, OPM required agencies to demonstrate that at least 
60 percent of each senior executive's performance plan is focused on 
achieving results and has clear measures associated with those results 
to show whether the goals have been achieved to be certified. The 
selected agencies have designed their appraisal systems to address 
OPM's requirement of aligning individual expectations with 
organizational goals. For example, in setting expectations for 
individual performance plans, DOE requires senior executives and 
supervisors to identify three to five key performance requirements with 
metrics that the executive must accomplish in order for the agency to 
achieve its strategic goals. Weighted at 60 percent of the summary 
rating, the performance requirements are to be specific to the 
executive's position and described in terms of specific results with 
clear, credible measures (e.g., quality, quantity, timeliness, cost- 
effectiveness) of performance, rather than activities. For each 
performance requirement, the executive is to identify the applicable 
strategic goal in the performance plan. To ensure that agencies are 
implementing their policies for alignment of performance expectations 
with organizational goals, OPM requires agencies as part of their 
certification submissions to provide a sample of executive performance 
plans, the strategic plan or other organizational performance documents 
for establishing alignment, and a description of the appraisal system 
outlining the linkage of executive performance with organizational 
goals. 

Further, OPM requires agencies to factor organizational performance 
into senior executive performance appraisals to receive certification 
of their SES appraisal systems. According to OPM and OMB officials 
overseeing the certification review process, the main sources of 
organizational performance that agencies use are the performance and 
accountability reports (PAR); program assessment rating tool (PART) 
summaries, which capture agencywide as well as program-or office- 
specific performance; and the President's Management Agenda (PMA) 
scorecards, as applicable. However, agencies have the flexibility to 
determine the format and type of organizational performance information 
for the performance appraisal process and certification submissions, 
according to OMB's lead official for the certification review process. 

All of the selected agencies have policies in place for factoring 
organizational performance into senior executive appraisal decisions 
and have identified common organizational assessments--such as the PMA, 
PAR, or PART results--for highlighting organizational performance 
results. As a next step, a few of the agencies, such as NRC and 
Treasury, have developed customized tools summarizing organizational 
performance at different levels of the organization, such as the 
bureau, office, or program levels to help ensure that senior executive 
appraisal decisions are consistent with organizational performance. For 
example, NRC provides summary reports capturing office-level 
performance to rating and reviewing officials to ensure that these 
officials have the information they need to make consistent assessments 
between senior executive and organizational performance. At the 
midpoint and end of the appraisal cycle, NRC's senior performance 
officials (SPO)--two top-level executives responsible for assessing 
organizational performance--conduct assessments for each office that 
take into account quarterly office performance reports on their 
operating plans, an interoffice survey on the office's performance 
completed by the other directors as identified by NRC, as well as the 
office director's self-assessment of the office's performance. 

To assess bureau-level performance, Treasury uses a departmentwide 
organizational assessment tool that provides a "snapshot" of each 
bureau's performance across various indicators of organizational 
performance, such as the PAR, PART results, PMA areas, OPM's Federal 
Human Capital Survey results, budget data, and information on material 
weaknesses. PRB members and reviewing officials receive copies of the 
organizational performance assessments, which serve as a basic 
framework for reviewing and recommending senior executive ratings, pay, 
and bonuses to help ensure ratings and pay are consistent with the 
organization's performance. According to Treasury's Deputy Assistant 
Secretary for Human Resources and Chief Human Capital Officer (CHCO), 
the indicators of organizational performance are updated throughout the 
year as organizational performance is always changing and the senior 
executives need to have a sense of the organization's performance at 
all times. 

Prior to the completion of individual performance ratings, agencies are 
to communicate organizational performance to senior executives, PRB 
members, and other reviewing officials--including supervisors who 
complete the ratings--involved in appraisal decisions to ensure they 
understand the effect organizational performance can have on rating 
distributions. Almost all of the selected agencies provided 
organizational performance assessments and communicated the importance 
of considering organizational performance in individual appraisals 
through briefings, training, or document packages for the PRB meetings. 
One agency, however, did not provide any specific information regarding 
organizational performance to PRB members and other reviewing 
officials. 

* DOD provided the heads of its components with a departmentwide 
organizational assessment against its overall priorities for fiscal 
year 2007 that was to be used in appraising senior executive 
performance and, as a check across the components, asked for copies of 
the training given to PRB members and other reviewing officials on 
factoring organizational performance into senior executive appraisal 
recommendations. According to the Principal Director to the Deputy 
Under Secretary of Defense for Civilian Personnel Policy, the 
components had the flexibility to use the departmentwide assessment and 
to develop their own organizational assessments. Component 
organizational assessments were required to be linked to the 
departmentwide priorities and assessment. Component organizational 
assessments can provide a level of specificity that enables a clearer 
connection or "line of sight" between individual executive and 
organizational performance. Having the components provide the 
department with their communications of organizational performance and 
how it was used to inform executive rating decisions provides 
accountability across the components for the departmental performance 
management policies, according to this official. 

* DOE provides its PRB members with snapshots of the Consolidated 
Quarterly Performance Reports relevant to the senior executives that 
measure how each departmental element performed respective to the goals 
and targets in its annual performance plan. According to the Director 
of the Office of Human Capital Management, the Deputy Secretary also 
verbally briefed PRB members on the importance of considering 
organizational performance in appraising executive performance. 

* For its most recently completed appraisal cycle, State for the first 
time provided PRB members an organizational assessment composed of 
various indicators from the most recent PART, PMA scorecard, and 
PAR.[Footnote 15] For the previous appraisal cycle, PRB members 
received various documents, such as senior executives' performance 
plans and appraisals and the performance management policy, but did not 
receive any specific assessments of organizational performance. 
According to a senior human resources official at State, based on OPM's 
and OMB's feedback for its 2008 certification submission, the agency 
has committed to providing organizational performance results in its 
guidance to the PRB members on how to consider organizational 
performance in making individual senior executive appraisal 
recommendations, among other things. 

* In contrast, USAID did not provide its PRB members and other 
reviewing officials with any specific information on organizational 
performance to help inform their senior executive appraisal 
recommendations for the fiscal year 2007 appraisal cycle. According to 
a senior human resources official at USAID, the agency does not provide 
PRB members and reviewing officials with these organizational 
performance assessments because they know where to find the relevant 
information applicable for each senior executive's performance 
appraisal given the small size of the agency. Nevertheless, providing 
and communicating uniform organizational performance assessments can 
help ensure consistency and clarity in how organizational performance 
is considered in appraising executive performance among PRB members, 
rating officials, and other reviewers. According to USAID's Deputy 
Director for Human Resources, USAID has developed various indicators of 
organizational performance-- such as individual operating unit reports, 
the Agency Financial Report, the PMA, PART results, and the 
Congressional Budget Justification outlining agency performance and 
other information--which are readily available for use by PRB members 
and other reviewing officials responsible for appraising senior 
executive performance. 

Selected Agencies Use Various Mechanisms to Help Make Meaningful 
Distinctions in Performance, but Senior Executives Were Concentrated at 
the Top Two Rating Levels and Received Varying Performance Payout 
Amounts: 

Effective performance management systems make meaningful distinctions 
between acceptable and outstanding performance of individuals and 
appropriately reward those who perform at the highest level. In order 
to receive OPM certification and OMB concurrence, agencies are to 
design and administer performance appraisal systems that make 
meaningful distinctions based on relative performance through 
performance rating and resulting performance payouts (e.g., bonuses and 
pay adjustments). To address the certification criteria of performance 
and pay differentiation, agencies are to use multiple rating levels-- 
four or five levels including a level for outstanding performance--and 
recognize the highest performing executives with the highest ratings 
and largest pay adjustments and bonuses, among other things. 

Selected Agencies Designed Their Appraisal Systems to Help Make 
Meaningful Distinctions in Senior Executive Performance: 

Five of the selected agencies designed their appraisal systems to help 
allow for differentiations when assessing and rewarding executive 
performance by establishing tier structures or prescribed performance 
payout ranges based on the resulting performance rating. 

* For example, NRC uses three tiers called position groups to 
differentiate its senior executives' basic pay and the resulting bonus 
amounts based on ratings received at the end of the appraisal cycle. 
NRC divides its executives into three groups (A, B, and C) based on the 
position's difficulty of assignment and the scope of responsibilities 
and annually sets basic pay ceilings for each of the groups tied to the 
EX pay levels. NRC uses the position groups and resulting performance 
ratings as the basis for its bonus structure to help ensure that 
executives in the higher position groups with the higher performance 
ratings receive the larger bonuses, as shown in table 1. In fiscal year 
2007, an executive in the highest position group A who received an 
outstanding rating was to receive a $30,000 bonus, while an executive 
in the lowest group C with the same rating was to receive a $20,000 
bonus. According to a senior human resources official at NRC, the bonus 
range for executives in group C with excellent ratings was intended to 
help allow for meaningful distinctions in performance to be made within 
that group, as well as to give the agency flexibility in the monetary 
amounts of the bonuses awarded. 

Table 1: NRC's SES Position Groups with Basic Pay Ceilings and 
Resulting Bonus Amounts Based on Position Group and Performance Ratings 
for the Fiscal Year 2007 Appraisal Cycle: 

Examples of SES positions by group: A: Executive Director for 
Operations, Chief Financial Officer, General Counsel, and major program 
office directors (e.g., Director of the Office of Nuclear Reactor 
Regulation); 
Basic pay ceiling (comparable to EX pay): $172,200: (EX- level II); 
Resulting bonus amount based on performance rating received: 
Outstanding: $30,000; 
Resulting bonus amount based on performance rating received: Excellent: 
$25,000; 
Resulting bonus amount based on performance rating received: Meets 
expectations: $0. 

Examples of SES positions by group: B: Support and small program office 
directors (e.g., Directors of the Offices of Administration and Human 
Resources), Deputy Directors of the Offices of the General Counsel, and 
the Chief Financial Officer; 
Basic pay ceiling (comparable to EX pay): 165,350: (Midpoint between EX-
levels II and III); 
Resulting bonus amount based on performance rating received: 
Outstanding: 25,000; 
Resulting bonus amount based on performance rating received: Excellent: 
20,000; 
Resulting bonus amount based on performance rating received: Meets 
expectations: 0. 

Examples of SES positions by group: C: All other SES members; 
Basic pay ceiling (comparable to EX pay): 158,500 (EX-level III); 
Resulting bonus amount based on performance rating received: 
Outstanding: 20,000; 
Resulting bonus amount based on performance rating received: Excellent: 
8,000-13,800; 
Resulting bonus amount based on performance rating received: Meets 
expectations: 0. 

Source: NRC. 

Notes: NRC has a five-level appraisal system, but senior executives in 
the two lowest rating categories--unsatisfactory and needs improvement-
-are not eligible to receive bonuses based on their performance 
ratings. The governmentwide basic pay cap for SES under certified 
performance appraisal systems is EX-level II. 

[End of table] 

* State uses a six-tier structure to help differentiate executive 
performance based on the ratings and bonuses and allocate pay 
adjustment amounts for its senior executives, with senior executives 
who are placed in the highest tier (I) receiving a larger percentage 
pay adjustment than those in a lower tier (V), who received the annual 
percentage adjustment to the EX pay schedule--2.5 percent in 2008. 

* In 2008, DOD implemented a departmentwide tier structure to help 
ensure comparability and transparency in SES position and compensation 
management with pay ceilings for each of the tiers tied to EX-level II 
and III pay rates. Specifically, DOD assigned SES positions to three 
tiers based on the position's impact on mission, level of complexity, 
span of control, and influence in joint, national security matters, 
among other things. According to the Principal Director, DOD is now 
using the tier structure to differentiate executive performance payouts 
to recognize that high-level performance in some positions has more 
impact than comparable performance in other positions. Further, DOD 
uses a mathematical formula to differentiate the performance payout 
amounts among its senior executives based on the recommended 
performance rating, performance score, and performance payout shares, 
as shown in table 2. In determining the number of performance payout 
shares to recommend, rating officials are to consider areas such as the 
executive's level of responsibility, mission impact, current basic pay, 
and performance against the relative performance of other executives, 
if applicable. The formula for computing the actual amount of the 
performance payout takes into account various indicators, such as the 
budget for bonuses and pay increases, annual adjustment to the EX pay 
rates, and total salaries and number of performance shares for all the 
senior executives in the pay pool. 

Table 2: DOD's Performance Scores and Performance Payout Shares by 
Performance Rating for the Fiscal Year 2007 Appraisal Cycle: 

Performance rating: Exceptional results; 
Performance score: 95-100; 
Performance payout shares: 11, 12, 13, 14, 15, or 16. 

Performance rating: Exceeds expected results; 
Performance score: 86-94; 
Performance payout shares: 7, 8, 9, or 10. 

Performance rating: Achieved expectations; 
Performance score: 70-85; 
Performance payout shares: 1, 2, 3, 4, 5, or 6. 

Performance rating: Minimally satisfactory; 
Performance score: 51-69; 
Performance payout shares: 0. 

Performance rating: Unsatisfactory; 
Performance score: 0-50; 
Performance payout shares: 0. 

Source: DOD. 

[End of table] 

* DOE sets prescribed ranges tied to performance ratings for its senior 
executives prior to finalizing ratings to help create a greater 
distinction between bonus amounts for top and middle performers and 
differentiate pay adjustment caps. Specifically, for fiscal year 2007, 
DOE required that all executives receiving an outstanding rating 
receive a bonus of 12 to 20 percent of basic pay, while executives 
receiving a meets expectations rating were eligible to receive a bonus 
of 5 to 9 percent at management's discretion.[Footnote 16] For pay 
adjustments, executives were eligible to receive a discretionary 
increase of up to 5 or 7 percent of basic pay if rated at meets 
expectations or outstanding, respectively. Executives who received 
needs improvement or unsatisfactory ratings were not eligible for any 
bonuses or pay increases.[Footnote 17] 

Selected Agencies Rated Their Senior Executives at the Top Two Rating 
Levels with Varying Bonus and Pay Adjustment Amounts: 

We have reported that using multiple rating levels provides a useful 
framework for making distinctions in performance by allowing an agency 
to differentiate among individuals' performance.[Footnote 18] As 
required for certification, all of the selected agencies have four or 
five rating levels in place for assessing senior executive performance. 
For the fiscal year 2007 appraisal cycle, senior executives were 
concentrated at the top two rating levels, as shown in figure 1. At 
State and USAID, about 69 percent and 60 percent of senior executives, 
respectively, received the top performance rating. At the other four 
agencies, the largest percentage of executives received the second 
highest rating--ranging from about 65 percent at NRC to 45 percent at 
Treasury. Conversely, less than 1 percent of senior executives across 
the selected agencies received a rating below fully successful (level 
3). As a point of comparison, about 47 percent of career SES 
governmentwide received the top performance rating for fiscal year 
2007, according to governmentwide data as reported by OPM. Similar to 
the selected agencies, less than 1 percent of career senior executives 
governmentwide received ratings below fully successful for fiscal year 
2007. 

Figure 1: Percentage of Senior Executives by Rating Level at the 
Selected Agencies for the Fiscal Year 2007 Appraisal Cycle: 

This figure is a combination bar graph showing the percentage of senior 
executives by rating level at the selected agencies for the fiscal year 
2007 appraisal cycle. The X axis represents the agencies, and the Y 
axis represents the percentage. The bars represent rating level 5 
(highest performance rating), rating level 4, rating level 3, rating 
level 2, and rating level 1 (lowest performance rating). 

Agency: DOD;	
Rating level 5 (highest performance rating): 32; 
Rating level 4: 54.3; 
Rating level 3: 13.4; 
Rating level 2: 0.3; 
Rating level 1 (lowest performance rating): 0. 

Agency: NRC; 
Rating level 5 (highest performance rating): 29.2; 
Rating level 4: 64.6; 
Rating level 3: 6.3; 
Rating level 2: 0; 
Rating level 1 (lowest performance rating): 0. 

Agency: State; 
Rating level 5 (highest performance rating): 69; 
Rating level 4: 27.4; 
Rating level 3: 3.5; 
Rating level 2: 0; 
Rating level 1 (lowest performance rating): 0. 

Agency: Treasury; 
Rating level 5 (highest performance rating): 43.9; 
Rating level 4: 44.9; 
Rating level 3: 10.7; 
Rating level 2: 0.5; 
Rating level 1 (lowest performance rating): 0. 

Agency: USAID; 
Rating level 5 (highest performance rating): 60; 
Rating level 4: 30; 
Rating level 3: 10; 
Rating level 2: 0; 
Rating level 1 (lowest performance rating): 0. 

Agency: DOE; 
Rating level 5 (highest performance rating): 37; 
Rating level 4: 0; 
Rating level 3: 62.4; 
Rating level 2: 0.5; 
Rating level 1 (lowest performance rating): 0. 

[See PDF for image] 

Source: GAO analysis of agency data. 

Note: The percentages may not total 100 percent due to rounding. 

[A] DOE uses a four-level appraisal system with no rating level between 
outstanding (rating level 5) and meets expectations (rating level 3). 

[End of figure] 

While OPM officials have certified that the selected agencies' systems 
are making meaningful distinctions, performance ratings at the selected 
agencies raise questions about the extent to which meaningful 
distinctions based on relative performance are being made and how OPM 
applies this criterion, as indicated in figure 1. As part of making 
meaningful distinctions in performance, OPM has emphasized to agencies 
through its certification guidance that its regulations prohibit forced 
distribution of performance ratings and that agencies must avoid 
policies or practices that would lead to forced distributions or even 
the appearance of it. A senior OPM official acknowledged that it is 
difficult for OPM to determine if an agency is using forced 
distributions through its review of agencies' aggregate appraisal 
results and policy documents. The official indicated that OPM looks at 
trends in the data across different components of agencies for 
statistical improbabilities, such as a certain percentage of SES 
members receiving an outstanding rating each year within an office that 
could be explained by a quota system. OPM has not provided specific 
guidance to agencies on how to make meaningful distinctions in senior 
executive performance while avoiding the perception of forced 
distributions of performance ratings. OPM has an opportunity to 
strengthen its communication with agencies and executives regarding the 
importance of using a range of rating levels when assessing performance 
while avoiding the use of forced distributions. Communicating this 
information to agencies will help them begin to transform their 
cultures to one where a fully successful rating is valued and rewarded. 

Senior-level officials at three of the selected agencies recognized the 
challenge in using a range of rating levels when appraising senior 
executive performance. In a memo to all SES members, DOE's Deputy 
Secretary stated his concern with the negligible difference in bonuses 
and pay adjustments among executives receiving the top two rating 
levels and stressed the importance of making meaningful distinctions in 
the allocation of compensation tied to performance ratings in the 
upcoming appraisal cycle. According to State's Deputy Assistant 
Secretary for the Bureau of Human Resources, historically the vast 
majority of senior executives have received the highest rating of 
outstanding, including for fiscal year 2007. Since the implementation 
of performance-based pay, this official said State has struggled with 
changing the culture and general perception among senior executives 
that any rating less than outstanding is a failure. According to DOD's 
Principal Director, DOD is communicating the message that the SES 
performance-based pay system recalibrates performance appraisals as a 
way to help change the culture and make meaningful distinctions in 
performance. A fully successful or equivalent rating is a high standard 
as well as a valued and quality rating. Levels above fully successful 
require extraordinary results. Part of this communication is developing 
common benchmark descriptors for the performance elements at the 5, 4, 
and 3 rating levels. The Principal Director said she hopes that 
developing common definitions for the performance elements at all three 
levels will aid the development of a common understanding and in turn 
make more meaningful distinctions in ratings. The agency official 
recognizes that this shift will require a significant cultural change, 
and that such cultural transformation takes time. 

The percentage of eligible executives who received bonuses or pay 
adjustments varied across the selected agencies for fiscal year 2007, 
as shown in table 3. The percentage of eligible senior executives who 
received bonuses ranged from about 92 percent at DOD to about 30 
percent at USAID, with the average dollar amount of bonuses ranging 
from $11,034 at State to about $17,917 at NRC.[Footnote 19] All 
eligible executives at State received pay adjustments, while about 88 
percent of eligible executives at DOE received adjustments, with the 
average dollar amount of such adjustments ranging from about $5,414 at 
NRC to about $6,243 at DOE. As a point of comparison, about 75 percent 
of career senior executives received bonuses with an average dollar 
amount of $14,221 for fiscal year 2007, according to OPM's 
governmentwide data report. The governmentwide percentage of career 
senior executives receiving pay adjustments and the average dollar 
amount of such adjustments in the aggregate are not available from 
OPM's governmentwide data report for fiscal year 2007. 

The selected agencies have policies in place where only senior 
executives who receive a rating of fully successful (level 3) or higher 
are eligible to receive bonuses or pay increases. Also affecting 
executives' bonus eligibility are the agencies' policies on awarding 
bonuses to executives who also received Presidential Rank Awards that 
year, which varied among the selected agencies.[Footnote 20] NRC, 
State, and Treasury do not allow executives to receive both awards in 
the same year, while DOD, DOE, and USAID allow the practice. 

Table 3: Percentage of Eligible Senior Executives Who Received Bonuses 
or Pay Adjustments and the Average Amounts at the Selected Agencies for 
the Fiscal Year 2007 Appraisal Cycle: 

Agency: DOD; 
Bonuses: Percentage who received bonuses: 92; 
Bonuses: Average amount: $13,934; 
Pay adjustments: Percentage who received pay adjustments: 95; 
Pay adjustments: Average amount: $5,739. 

Agency: DOE; 
Bonuses: Percentage who received bonuses: 82; 
Bonuses: Average amount: 14,116; 
Pay adjustments: Percentage who received pay adjustments: 88; 
Pay adjustments: Average amount: 6,243. 

Agency: NRC; 
Bonuses: Percentage who received bonuses: 87; 
Bonuses: Average amount: 17,917; 
Pay adjustments: Percentage who received pay adjustments: 95; 
Pay adjustments: Average amount: 5,414. 

Agency: State; 
Bonuses: Percentage who received bonuses: 55; 
Bonuses: Average amount: 11,034; 
Pay adjustments: Percentage who received pay adjustments: 100; 
Pay adjustments: Average amount: 6,148. 

Agency: Treasury; 
Bonuses: Percentage who received bonuses: 77; 
Bonuses: Average amount: 16,074; 
Pay adjustments: Percentage who received pay adjustments: 93; 
Pay adjustments: Average amount: 6,120. 

Agency: USAID; 
Bonuses: Percentage who received bonuses: 30; 
Bonuses: Average amount: 11,083; 
Pay adjustments: Percentage who received pay adjustments: 90; 
Pay adjustments: Average amount: 6,227. 

Source: GAO analysis of agency data. 

Notes: In calculating the percentage of eligible senior executives who 
received bonuses or pay adjustments and average amounts, we excluded 
executives who received a rating less than fully successful since those 
executives are not eligible to receive bonuses or pay increases, 
according to the selected agencies' policies. We also excluded SES 
members at NRC, State, and Treasury who received Presidential Rank 
Awards because according to the agencies' policies, those individuals 
were not considered for bonuses. For all agencies, we included senior 
executives who were rated but left their positions--because of 
retirement, attrition, or assignment to a lower grade--prior to 
performance payouts being made. 

[End of table] 

According to OPM regulations, agencies are to recognize the highest 
performing executives with the highest ratings and largest bonuses and 
pay adjustments.[Footnote 21] At five of the selected agencies, the 
highest performing executives (rated at level 5) made up the greatest 
percentage of eligible executives receiving bonuses. At NRC, all 
eligible executives rated at the top two levels received a bonus. At 
all the agencies, the executives rated at the highest level received 
the largest bonuses on average--about $23,333 at NRC compared to about 
$11,034 at State. State only awarded bonuses to executives receiving 
outstanding ratings for fiscal year 2007. According to State's senior 
human resources official, State does not have an official policy 
prohibiting those receiving ratings of exceeds expectations or fully 
successful from receiving a bonus. Rather, the agency official stated 
that State's decision to only award bonuses to executives who received 
outstanding ratings was due to budget constraints and an effort to keep 
the SES parallel with the SFS in the allocation of bonuses and pay 
adjustments. In addition, senior executives at NRC and USAID rated at 
fully successful (level 3) did not receive bonuses. (see fig. 2). 

Figure 2: Percentage of Eligible Senior Executives Who Received Bonuses 
and the Average Bonus Amounts by Rating Level at the Selected Agencies 
for the Fiscal Year 2007 Appraisal Cycle: 

This figure is a combination of two bar graphs, showing the percentage 
of eligible senior executives who received bonuses and the average 
bonus amounts by rating level at the selected agencies for the fiscal 
year 2007 appraisal cycle. The X axis for the left graph represents 
percentage, and the X axis for the right graph represents dollars in 
thousands. The Y axis represents agencies. The bars represent rating 
level 5 (highest performance rating), rating level 4, rating level 3, 
rating level 2, and rating level 1 (lowest performance rating). * 
signifies that these were not eligible to receive bonuses at these 
rating levels. 

Percentage of eligible SES members receiving a bonus by rating level: 

Agency: DOD;	
Rating level 5 (highest performance rating): 99; 
Rating level 4: 96; 
Rating level 3: 61; 
Rating level 2: *; 
Rating level 1 (lowest performance rating): *. 

Agency: DOE; 
Rating level 5 (highest performance rating): 94; 
Rating level 4: N/A[A]; 
Rating level 3: 74; 
Rating level 2: *; 
Rating level 1 (lowest performance rating): *. 

Agency: NRC; 
Rating level 5 (highest performance rating): 100; 
Rating level 4: 100; 
Rating level 3: 3.5; 
Rating level 2: *; 
Rating level 1 (lowest performance rating): *. 

Agency: State; 
Rating level 5 (highest performance rating): 83; 
Rating level 4: 0; 
Rating level 3: 0; 
Rating level 2: *; 
Rating level 1 (lowest performance rating): *. 

Agency: Treasury; 
Rating level 5 (highest performance rating): 99; 
Rating level 4: 72; 
Rating level 3: 13; 
Rating level 2: *; 
Rating level 1 (lowest performance rating): *. 

Agency: USAID; 
Rating level 5 (highest performance rating): 42; 
Rating level 4: 17; 
Rating level 3: 0; 
Rating level 2: *; 
Rating level 1 (lowest performance rating): *. 

Average bonus by rating level: 

Agency: DOD;	
Rating level 5 (highest performance rating): $18,373; 
Rating level 4: $12,100; 
Rating level 3: $8,456; 
Rating level 2: *; 
Rating level 1 (lowest performance rating): *. 

Agency: DOE; 
Rating level 5 (highest performance rating): $20,326; 
Rating level 4: N/A[A]; 
Rating level 3: $9,258; 
Rating level 2: *; 
Rating level 1 (lowest performance rating): *. 

Agency: NRC; 
Rating level 5 (highest performance rating): $23,333; 
Rating level 4: $15,640; 
Rating level 3: $0; 
Rating level 2: *; 
Rating level 1 (lowest performance rating): *. 

Agency: State; 
Rating level 5 (highest performance rating): $11,034; 
Rating level 4: $0; 
Rating level 3: $0; 
Rating level 2: *; 
Rating level 1 (lowest performance rating): *. 

Agency: Treasury; 
Rating level 5 (highest performance rating): $19,195; 
Rating level 4: $12,389; 
Rating level 3: $8,885; 
Rating level 2: *; 
Rating level 1 (lowest performance rating): *. 

Agency: USAID; 
Rating level 5 (highest performance rating): $11,500; 
Rating level 4: $9,000; 
Rating level 3: $0; 
Rating level 2: *; 
Rating level 1 (lowest performance rating): *. 

[See PDF for image] 

Source: GAO analysis of agency data. 

Notes: In calculating the percentage of eligible senior executives who 
received bonuses and average amounts, we excluded executives who 
received a rating less than fully successful since those executives are 
not eligible to receive bonuses, according to the selected agencies' 
policies. We also excluded SES members at NRC, State, and Treasury who 
received Presidential Rank Awards because according to the agencies' 
policies, those individuals were not considered for bonuses. For all 
agencies, we included senior executives who were rated but left their 
positions--because of retirement, attrition, or assignment to a lower 
grade--prior to performance payouts being made. 

[A] DOE uses a four-level appraisal system with no rating level between 
outstanding (rating level 5) and meets expectations (rating level 3). 

[End of figure] 

In a memo to agencies on the certification process, OPM has stated that 
it expects senior executives who receive a fully successful or higher 
rating and are paid at a level consistent with their current 
responsibilities will receive a performance-based pay increase. 
According to a senior OPM official, agencies are not required to give 
these executives pay increases, but OPM considers fully successful to 
be a good rating and encourages agencies to recognize and reward 
executives performing at this level. At the selected agencies, the 
majority of eligible senior executives rated at fully successful 
received pay adjustments for fiscal year 2007, as shown in figure 3. 

The highest-performing executives (rated at level 5) did not make up 
the greatest percentage of executives receiving pay adjustments with 
the largest increases on average at some of the selected agencies. 
Specifically, at Treasury, about 95 percent of eligible executives 
rated at level 4 received a pay adjustment, compared with about 91 
percent of eligible executives rated at level 5 and about 90 percent 
rated a level 3. At NRC, all of the eligible executives rated at level 
5 and level 3 received pay adjustments compared with about 92 percent 
of eligible executives rated at level 4[Footnote 22]. For all the 
agencies except Treasury, the executives rated at the highest level 
received the largest pay adjustments on average--about $7,473 at USAID 
compared to about $6,133 at NRC. At Treasury, executives rated at 
levels 5, 4, and 3 on average received about the same pay adjustment 
amounts, primarily due to pay cap issues. 

Figure 3: Percentage of Eligible Senior Executives Who Received Pay 
Adjustments and the Average Pay Adjustment Amounts by Rating Level at 
the Selected Agencies for the Fiscal Year 2007 Appraisal Cycle: 

This figure is a combination of two graphs showing the percentage of 
eligible senior executives who received pay adjustments and the average 
pay adjustment amounts by rating level at the selected agencies for the 
fiscal year 2007 appraisal cycle. The X axis on the left represents 
percentage, and the X axis on the right represents dollars in 
thousands. The Y axis represents the agencies. The bars represent 
rating level 5 (highest performance rating), rating level 4, rating 
level 3, rating level 2, and rating level 1 (lowest performance 
rating). * signifies that these were not eligible to receive bonuses at 
these rating levels. 

Percentage of eligible SES members receiving a pay adjustment by rating 
level: 

Agency: DOD; 
Rating level 5 (highest performance rating): 96; 
Rating level 4: 95; 
Rating level 3: 86; 
Rating level 2: *; 
Rating level 1 (lowest performance rating): *. 

Agency: DOE; 
Rating level 5 (highest performance rating): 95; 
Rating level 4: N/A[A]; 
Rating level 3: 84; 
Rating level 2: *; 
Rating level 1 (lowest performance rating): *. 

Agency: NRC; 
Rating level 5 (highest performance rating): 100; 
Rating level 4: 92; 
Rating level 3: 100; 
Rating level 2: *; 
Rating level 1 (lowest performance rating): *. 

Agency: State; 
Rating level 5 (highest performance rating): 100; 
Rating level 4: 100; 
Rating level 3: 100; 
Rating level 2: *; 
Rating level 1 (lowest performance rating): *. 

Agency: Treasury; 
Rating level 5 (highest performance rating): 91; 
Rating level 4: 95; 
Rating level 3: 90; 
Rating level 2: *; 
Rating level 1 (lowest performance rating): *. 

Agency: USAID; 
Rating level 5 (highest performance rating): 92; 
Rating level 4: 83; 
Rating level 3: 100; 
Rating level 2: *; 
Rating level 1 (lowest performance rating): *. 

Average pay adjustments by rating level: 

Agency: DOD;	
Rating level 5 (highest performance rating): $6,572; 
Rating level 4: $5,601; 
Rating level 3: $4,143; 
Rating level 2: *; 
Rating level 1 (lowest performance rating): *. 

Agency: DOE; 
Rating level 5 (highest performance rating): $6,496; 
Rating level 4: N/A[A]; 
Rating level 3: $6,073; 
Rating level 2: *; 
Rating level 1 (lowest performance rating): *. 

Agency: NRC; 
Rating level 5 (highest performance rating): $6,133; 
Rating level 4: $5,216; 
Rating level 3: $4,028; 
Rating level 2: *; 
Rating level 1 (lowest performance rating): *. 

Agency: State; 
Rating level 5 (highest performance rating): $6,651; 
Rating level 4: $5,142; 
Rating level 3: $4,150; 
Rating level 2: *; 
Rating level 1 (lowest performance rating): *. 

Agency: Treasury; 
Rating level 5 (highest performance rating): $6,002; 
Rating level 4: $6,304; 
Rating level 3: $5,802; 
Rating level 2: *; 
Rating level 1 (lowest performance rating): *. 

Agency: USAID; 
Rating level 5 (highest performance rating): $7,473; 
Rating level 4: $4,517; 
Rating level 3: $3,652; 
Rating level 2: *; 
Rating level 1 (lowest performance rating): *. 

[See PDF for image] 

Source: GAO analysis of agency data. 

Notes: In calculating the percentage of eligible senior executives who 
received pay adjustments and average amounts, we excluded executives 
who received a rating less than fully successful since those executives 
are not eligible to receive pay increases, according to the selected 
agencies' policies. For all agencies, we included senior executives who 
were rated but left their positions--because of retirement, attrition, 
or assignment to a lower grade--prior to performance payouts being 
made. 

[A] DOE uses a four-level appraisal system with no rating level between 
outstanding (rating level 5) and meets expectations (rating level 3). 

[End of figure] 

The Proximity of Senior Executives' Basic Pay to the Governmentwide 
Basic Pay Cap Varies Across the Selected Agencies: 

We have reported that the federal government as a whole may face 
challenges in offering competitive compensation to its senior 
leaders.[Footnote 23] In 2003, about 70 percent of senior executives 
received the same basic pay due to compression--which occurred when 
their pay reached the statutory cap of EX-level III. In 2004, the SES 
performance-based pay system and certification process provided an 
interim solution to this issue of pay compression by creating a single, 
open-range pay band and allowing agencies to increase the basic pay cap 
for their senior executives to EX-level II upon certification of their 
performance appraisal systems by OPM with OMB concurrence. 

While the pay cap was raised for certified agencies, agencies are to 
reserve the pay rates above EX-level III for truly outstanding 
performers only, which in effect slows the growth of senior executives' 
pay within the pay band. According to OPM regulations, the rates of 
basic pay higher than EX-level III but less than or equal to EX-level 
II are generally reserved for senior executives who have demonstrated 
the highest levels of individual performance and/or made the greatest 
contributions to the agency's performance, or newly appointed senior 
executives who possess superior leadership or other competencies. 

The basic pay for senior executives at the selected agencies shows that 
pay compression may be a problem in the future for some agencies. 
Overall, however, only a small percentage of senior executives at the 
selected agencies have their basic pay capped out at EX-level II 
($172,200) in 2008. Specifically, about half to three-quarters of 
senior executives at the selected agencies are paid at or above EX- 
level III ($158,500) in 2008, after performance-based pay adjustments 
were made for the fiscal year 2007 performance appraisal cycle, as 
shown in figure 4.[Footnote 24] For example, at State, about 76 percent 
of senior executives are paid at or above EX-level III, while about 50 
percent of senior executives at DOD and USAID are paid at these rates. 
Of the senior executives paid at or above $158,500, the percentage of 
senior executives who are paid at the governmentwide pay cap for 2008-
-$172,200--varies across the agencies. Specifically, at State and DOE, 
about 26 percent of senior executives are paid at the pay cap for 2008, 
while only 1 percent of senior executives at DOD and none at USAID are 
paid at this cap. OPM has found that about 13 percent of SES members 
governmentwide are paid at the pay cap based on the fiscal year 2007 
performance appraisal data they received from agencies. 

Figure 4: Percentage of Senior Executives at the Selected Agencies Paid 
According to Executive Schedule Pay Levels in 2008: 

This figure is a combination bar graph showing percentage of senior 
executives at the selected agencies paid according to executive 
schedule pay levels in 2008. The X axis represents the agency, and the 
bars represent percentage of SES members paid at or above EX-level III 
($158,500), percentage of SES members paid below EX-level III, 
percentage of SES members paid at EX-level II ($172,200). 

Agency: DOD; 
Percentage of SES members paid at or above EX-level III ($158,500): 50; 
Percentage of SES members paid below EX-level III: 50; 
Percentage of SES members paid at EX-level II ($172,200): 1. 

Agency: DOE; 
Percentage of SES members paid at or above EX-level III ($158,500): 70; 
Percentage of SES members paid below EX-level III: 30; 
Percentage of SES members paid at EX-level II ($172,200): 26. 

Agency: NRC; 
Percentage of SES members paid at or above EX-level III ($158,500): 72; 
Percentage of SES members paid below EX-level III: 28; 
Percentage of SES members paid at EX-level II ($172,200): 4. 

Agency: State; 
Percentage of SES members paid at or above EX-level III ($158,500): 76; 
Percentage of SES members paid below EX-level III: 24; 
Percentage of SES members paid at EX-level II ($172,200): 26. 

Agency: Treasury; 
Percentage of SES members paid at or above EX-level III ($158,500): 65; 
Percentage of SES members paid below EX-level III: 35; 
Percentage of SES members paid at EX-level II ($172,200): 2. 

Agency: USAID; 
Percentage of SES members paid at or above EX-level III ($158,500): 50; 
Percentage of SES members paid below EX-level III: 50; 
Percentage of SES members paid at EX-level II ($172,200): 0. 

[See PDF for image] 

Source: GAO analysis of agency data. 

[End of figure] 

According to a senior OPM official, the SES performance-based pay 
system was never intended to fix the problem of pay compression that 
occurred prior to 2004 or be the answer to future pay compression 
issues. OPM recognizes that pay compression is a problem in the SES. 
While the majority of senior executives at the selected agencies have 
yet to reach the governmentwide basic pay cap, officials from two of 
the selected agencies recognized the challenge of making distinctions 
in executive performance given potential pay compression issues. 
Specifically, Treasury's Deputy Assistant Secretary for Human Resources 
and CHCO said an agency does not have enough room in the governmentwide 
pay band to fully recognize the outstanding performers through the 
appraisal system since the best performers are already near the top of 
the pay range and their performance payouts in the form of basic pay 
increases are limited. DOD's Principal Director said when pay increases 
are not possible given salary cap issues, bonuses are a tool for 
components to reward their executives' performance for achieving 
results. 

NRC's three position groups and the associated pay ceilings are 
intended, in part, to help reserve pay above EX-III for those 
executives who demonstrate the highest levels of performance, including 
the greatest contribution to organizational performance as determined 
through the appraisal system, according to a senior human resources 
official at NRC. A senior executive would not receive a pay increase if 
the executive had already reached the pay ceiling for the applicable 
position group. While there is little room for pay increases within the 
pay bands for each position group, the agency official indicated that 
NRC tries to give pay adjustments and generous bonus amounts when 
possible to acknowledge that their senior executives are high- 
performing individuals. To identify possible areas and options for 
improvement to its performance appraisal and pay system including the 
tier structure, NRC convened an executive working group of PRB members 
and senior executives from different areas and position groups. As part 
of its June 2008 findings, the working group recommended retaining the 
three position groups and existing pay ceilings with slight revisions 
to the positions that fall within each group. NRC management accepted 
this recommendation, according to the agency official. 

Selected Agencies Have Built Safeguards into Their Senior Executive 
Performance Appraisal and Pay Systems to Help Ensure Fairness and 
Transparency: 

We have reported that agencies need to have modern, effective, 
credible, and validated performance management systems in place with 
adequate safeguards to ensure fairness and prevent politicization and 
abuse.[Footnote 25] All of the selected agencies have safeguards 
including higher-level reviews of performance appraisal 
recommendations, PRBs, and transparency in communicating the aggregate 
results, although agencies varied in how they implemented such 
safeguards. 

Higher-level reviews. By law, as part of their SES appraisal systems, 
all agencies must provide their senior executives with an opportunity 
to view their performance appraisals and to request a review of the 
recommended performance ratings by higher-level officials, before the 
ratings become final.[Footnote 26] The higher-level reviewer cannot 
change the initial rating given by the supervisor, but may recommend a 
different rating in writing to the PRB that is shared with the senior 
executive and the supervisor. For example, according to State's policy, 
an executive may request a higher-level review of the initial rating in 
writing prior to the PRB convening, at which time the initial summary 
rating, the executive's request, and the higher-level reviewer's 
written findings and recommendations are considered. The PRB is to 
provide a written recommendation on the executive's summary rating to 
State's Director General of the Foreign Service and Director of Human 
Resources, who makes the final appraisal decisions. 

Performance review boards. All agencies must establish one or more PRBs 
to help ensure that performance appraisals reflect both individual and 
organizational performance and rating, bonus, and pay adjustment 
recommendations are consistently made. The PRB is to review senior 
executives' initial summary performance ratings and other relevant 
documents and make written recommendations on the performance of the 
senior executives to the agency head or appointing authority. When 
appraising a career appointee's performance or recommending a career 
appointee for a bonus, more than one-half of the PRB's members must be 
SES career appointees. The selected agencies varied in their PRB 
structures and who provided the final approval of the appraisal 
decisions. On the one hand, given its small number of senior 
executives, USAID has one PRB that is responsible for making 
recommendations to the Administrator for his/her final approval on all 
rated career executives' annual summary ratings, bonuses, performance- 
based pay adjustments, and Presidential Rank Award nominations. On the 
other hand, DOD has multiple PRBs within and across its components and 
agencies with separate authorizing officials who give the final 
approval of rating and performance payout recommendations. As another 
level of review after the PRB, DOE convenes a Senior Review Board-- 
comprised mainly of political appointees--to review and approve the PRB 
recommendations for ratings, pay adjustments, and bonuses, and look for 
consistency in recommendations across the senior executives in 
headquarters, the field, and the various organizations within DOE. The 
Director of the Office of Human Capital Management said the Deputy 
Secretary, who serves as the chair, ultimately makes the final 
decisions on senior executives' ratings, pay adjustments, and bonuses. 

Transparency in communicating aggregate appraisal results. Agencies 
should communicate the overall aggregate results of the performance 
appraisal decisions--ratings, bonuses, and pay adjustment 
distributions--to senior executives while protecting individual 
confidentiality, and as a result, provide a clear picture of how the 
executive's performance compares with that of other executives in the 
agency. Further, as part of its certification decisions, OPM requires 
agencies to brief their SES members on the results of the completed 
appraisal process to make sure that the dynamics of the general 
distribution of ratings and accompanying rewards are fully understood. 
All the selected agencies communicated the aggregate appraisal results 
to senior executives, although their methods of communication and the 
types of information provided varied. 

* Treasury and DOD posted the aggregate rating, bonus, and pay 
adjustment distributions for senior executives on their Web sites with 
comparison of data across previous fiscal years. 

* NRC sent an e-mail to all senior executives providing the percentage 
of executives at each rating level and the percentage who received 
bonuses and pay adjustments, as well as the average dollar amounts. 
According to a senior human resources official at NRC, the agency 
periodically holds agencywide "all hands" SES meetings where the 
results of the appraisal cycle, among other topics, are communicated to 
executives. 

* The Deputy Secretary of DOE provides a memo to all senior executives 
summarizing the percentage of executives at the top two rating levels 
and the average bonus and pay adjustment amounts, as well as OPM's 
governmentwide results as a point of comparison. 

* USAID communicated the aggregate SES appraisal results to SES members 
throughout the appraisal cycle. In a February 2008 notice, USAID 
communicated to all SES members the pay adjustment distributions in 
ranges by rating level for the fiscal year 2007 appraisal cycle. In a 
September 2008 e-mail to all SES members and rating officials at the 
end of the appraisal cycle, USAID communicated the aggregate 
performance rating distributions for the past two appraisal cycles for 
fiscal years 2006 and 2007. 

While the selected agencies all shared aggregate appraisal results with 
their senior executives, the results of the OPM SES survey show that 
the communication of overall performance appraisal results is not 
widely practiced throughout the government. Specifically, 65 percent of 
respondents said that they were not given a summary of their agency's 
SES performance ratings, bonuses, and pay adjustments. At the June 2008 
forum with agency executive resources staff where it shared the survey 
results, OPM officials emphasized the importance of communicating 
aggregate appraisal results to all senior executives. According to a 
senior OPM official, agencies need to figure out how best to 
communicate aggregate appraisal results in a way that supports their 
different cultures and practices. The official said OPM plans to 
continually monitor how well the agencies are communicating aggregate 
appraisal results through the certification review process. 

OPM and OMB Provide Oversight through Various Formats; Selected Agency 
Officials Suggested Opportunities for Refinements: 

To ensure agencies' senior executive appraisal systems are designed and 
implemented to address the certification criteria, OPM and OMB, as 
applicable, provide continuing oversight by issuing guidance to 
agencies on revisions to the certification process, using tools and 
other initiatives to help assess how agencies are implementing their 
SES performance-based pay systems, providing training and forums, and 
interacting with agencies on the review of their certification 
submissions. While generally satisfied with OPM's and OMB's oversight, 
officials at the selected agencies said OPM could strengthen its 
communication with agencies and executives on how it uses the SES 
performance appraisal data and the correlation between ratings and 
performance pay in determining whether agencies are making meaningful 
distinctions based on relative performance. In addition, senior-level 
officials at the selected agencies identified a need for increased 
efficiency in the certification submission process. 

Issuing Guidance to Agencies on Revisions to the Certification Process: 

Providing agencies with clear and timely guidance is one way for OPM 
and OMB to effectively communicate with agencies upcoming revisions to 
the certification process. OMB does not issue its own guidance to 
agencies, but reviews OPM's guidance to agencies, according to OMB's 
lead official in the certification review process. Officials at five of 
the selected agencies said that, in the past, OPM has revised its 
guidance midway through the appraisal cycle, which did not allow 
agencies sufficient time to change their systems in order to receive 
certification for that calendar year. Recognizing that it was late in 
issuing the guidance in 2006, OPM since has issued guidance in the fall 
via memos to agency heads. In the future, OPM plans to continue issuing 
any changes to the guidance in the fall, according to a senior OPM 
official, since this is when agencies are finishing up the performance 
appraisal cycle and starting to set expectations for the next cycle. 
This should also provide agencies with adequate time to revise their 
appraisal systems to reflect any new requirements before the 
certification submission deadline at the end of June. In light of 
changes to the law in October 2008, in the near future OPM plans to 
issue regulations and revised certification guidance to agencies 
reflecting the modifications to SL/ST basic pay rates for certified 
appraisal systems and changing the certification cycle coverage to up 
to 24 months from a calendar-year-based coverage.[Footnote 27] 

Using Tools and Other Initiatives to Help Assess How Agencies Are 
Implementing Their SES Performance-Based Pay Systems: 

OPM and OMB use tools and other initiatives--such as the SES 
Performance Appraisal Assessment Tool (SES-PAAT), the correlation 
coefficient, and the governmentwide survey to all SES members on 
performance-based pay--to help assess how agencies are implementing 
their performance-based pay systems and addressing the certification 
criteria. Overall, selected agency officials were in favor of OPM and 
OMB using these tools and initiatives, although officials from three of 
the selected agencies expressed concern about how OPM calculated the 
correlation coefficient and the effect it had on the resulting score. 

SES-PAAT: 

In 2007, OPM developed the SES-PAAT to help streamline the 
certification process, improve the efficiency of its oversight process, 
and offer a more transparent and organized way for agencies, OPM, and 
OMB to examine SES appraisal systems, among other things. OPM first 
required agencies with fully certified SES appraisal systems to use 
this tool when requesting full certification for 2009 and 2010. Based 
on a set of questions that relate to the certification criteria, the 
SES-PAAT helps clarify the certification criteria and quantifies 
aspects of the certification package that agencies had previously 
supported through narrative form. In making an agency's certification 
decision, OPM and OMB consider the SES-PAAT score and the quality of 
the provided supporting documentation, such as the sample of individual 
performance plans. 

OPM is hopeful that the SES-PAAT will improve the efficiency of its 
oversight process and the feedback provided to the agencies, but 
officials from our selected agencies that completed the SES-PAAT have 
mixed views on using this tool. OPM removed one office from the review 
process for SES-PAAT submissions with the intention of spending less 
time overseeing the continuation of full certifications and more time 
focusing on provisional certifications or those agencies that were in 
danger of dropping from full to provisional certification, according to 
a senior OPM official. Officials from the two selected agencies that 
completed the SES-PAAT for the first time said the certification 
submission process was more efficient with less documentation submitted 
overall; however, the process was still labor intensive and time 
consuming, specifically with regard to the sample of performance plans 
required. Even though the SES-PAAT requires agencies to submit a 
smaller sample of performance plans, a senior human resources official 
said her agency needed to do a complete review of its SES performance 
plans to ensure that all the plans were adequate for certification 
approval. 

Correlation Coefficient: 

To help assess in part how agencies are meeting the pay differentiation 
certification criterion, OPM is using a metric based on a correlation 
coefficient that summarizes the strength of the relationship between 
SES members' ratings and their performance-based pay adjustments and 
bonuses as part of the Human Capital Assessment and Accountability 
Framework's systems, standard, and metrics.[Footnote 28] Given that at 
least 60 percent of executives' performance ratings are to be based on 
organizational results, a senior OPM official said calculating the 
relationship between executive ratings and performance pay provides an 
indication of how well an agency is recognizing its executives based on 
organizational results achieved. 

Officials from three of the selected agencies expressed concern about 
how OPM calculated the correlation coefficient and the effect it had on 
the resulting score. Specifically, OPM decided to include in its 
calculations those senior executives who received Presidential Rank 
Awards, but because of their agencies' policies were not eligible for 
and did not receive bonuses. As a result, the coefficients for those 
agencies may show a weaker connection between SES ratings and 
performance pay because highly rated executives did not receive 
bonuses. A senior OPM official said OPM recognizes that the decision to 
include all executives regardless of their bonus eligibility in the 
correlation coefficient may have a negative effect on an agency's 
coefficient, especially in the case of smaller agencies with few SES 
members, but it is a policy decision that OPM has made to ensure that 
the coefficients were calculated consistently across the government. 
For small agencies, OPM said that a correlation coefficient may not be 
appropriate for determining how the agency is addressing the pay 
differentiation criteria; rather, OPM will review the mean, median, and 
mode of the agencies' total compensation including pay adjustments and 
bonuses, as applicable to determine whether higher-rated executives 
were rewarded appropriately. 

SES Survey: 

In January 2008, OPM conducted a governmentwide survey of all SES 
members to evaluate the performance-based pay system. While OPM found 
considerable variability in the executives' responses across the 
different agencies, according to OPM the overall results show that the 
vast majority of executives believe pay should be based on performance 
and that areas for improvement exist, for example, in communicating 
aggregate appraisal results to senior executives. According to selected 
agency officials, the SES survey results were very helpful and useful 
to their agency. 

We previously recommended that OPM develop a strategy to allow it, 
other executive agencies, and Congress to monitor the progress of 
implementation of the senior executive performance-based pay 
system.[Footnote 29] The SES survey could be a vehicle for regularly 
monitoring progress in the future. OPM has not committed to 
administering the survey on an ongoing basis, in part due to the 
concern of over-surveying agency officials, but plans to revisit the 
idea of administering the survey again in the next several years. 
According to a senior OPM official, the Federal Human Capital Survey is 
administered every 2 years and provides OPM with the opportunity to 
monitor senior executives' satisfaction with the appraisal process, 
including whether they consider their appraisals to be a fair 
reflection of their performance, based on the senior executives who 
responded. Rather, when OPM decides to administer the survey again, 
according to the OPM official, it plans to target the relevant issues 
of the day with some of the original questions in order to track trends 
over time. 

Providing Training and Forums for Agency Officials to Obtain 
Information from OPM and Discuss Relevant Issues Regarding the SES 
Performance-Based Pay System: 

To help facilitate its communications and interactions with agency 
officials and the executive resources community, OPM periodically 
provides training and holds forums for agency officials to discuss 
different aspects of the SES performance-based pay system and the 
certification process, among other topics. Selected agency officials 
found the forums to be useful and helpful in understanding the 
certification process and requirements while allowing agencies to share 
lessons learned from and experiences with the certification process. 
OPM also finds these forums helpful in gathering agency feedback, which 
it considers in future revisions to the certification process and other 
human capital initiatives, according to a senior OPM official. For 
example, in December 2007 OPM conducted four training workshops for all 
interested agency executive resources officials on how to complete the 
SES-PAAT and plans to hold sessions in December 2008 and January 2009. 
OPM also holds forums five times a year for executive resources staff 
from all agencies. At these forums, OPM and agency officials have the 
opportunity to discuss common concerns, obtain status updates on 
various OPM initiatives, and learn about future plans for the 
certification process and other human capital areas. 

In addition, the CHCO Council chaired by the OPM Director works with 
agencies to develop and share leading practices in implementing human 
capital initiatives. For example, the CHCO Council periodically holds 
training academy sessions that are open to agency officials other than 
CHCOs to highlight and showcase human capital practices related to 
senior executive pay and certification issues. Specifically, over the 
last 2 years, the CHCO Council has held several training academy 
sessions related to SES performance management and pay systems, the SES-
PAAT, and lessons learned from the governmentwide SES survey results. 
In our past work we recommended that OPM work with the CHCO Council to 
develop a formal mechanism for sharing leading practices for 
implementing human capital initiatives, such as the SES certification 
process.[Footnote 30] OPM has addressed this recommendation by inviting 
all levels of agency officials to attend CHCO Council training academy 
sessions when relevant topics, such as the SES performance management 
and SES survey results, were featured. Moving forward, the CHCO 
subcommittee on performance management plans to partner with OMB's 
Performance Improvement Council to make improvements on SES 
certification and other human capital efforts.[Footnote 31] 

Interacting with Agencies on the Review of Their Certification 
Submissions: 

OPM provides oversight to the certification process by communicating 
and working directly with agencies to help them improve their systems. 
According to OMB's lead official in the certification review process, 
OPM takes the lead on the certification review process and OMB has a 
concurrence role focusing most of its review on specific aspects of 
agencies' certification submissions including aligning executive 
performance expectations with organizational and program goals, 
ensuring executives' goals are sufficiently results-oriented and 
challenging to drive improved performance; measuring organizational 
performance; and linking organizational results to the performance 
rating distribution. Overall, the selected agencies have positive 
working relationships with OPM on executive resources issues. For 
example, officials at five of the selected agencies found that working 
with OPM on the individual performance plans prior to submitting the 
certification package was helpful and that OPM provided useful feedback 
at this step in the certification process. Through its executive 
resources forums, OPM has also communicated directly with agencies on 
SES performance-based pay and the certification process, including 
sharing key results from the SES survey with agencies. 

However, OPM could strengthen its communication with agencies and 
executives on how it uses the SES performance appraisal data and 
correlation coefficient in determining whether agencies are making 
meaningful distinctions based on relative performance as measured 
through the performance and pay differentiation certification criteria. 
Further communication from OPM is important in order for agencies to 
have a better understanding of how they are being held accountable for 
these certification criteria and make the necessary improvements to 
their systems to maintain certification. Officials at four of the 
selected agencies said they are unclear about how OPM uses the SES 
appraisal data to assess whether agencies are meeting these criteria 
and making meaningful distinctions in performance overall. In addition, 
officials at four of the selected agencies said that the communication 
they have received from OPM individually and through broader forums, 
such as OPM's executive resources forums and certification guidance, 
has not provided them with a clear sense of how OPM is using the 
correlation coefficients to determine how agencies are addressing the 
pay differentiation criterion. For the coefficients based on fiscal 
year 2006 appraisal data, OPM provided each agency with their 
coefficients and technical information explaining the concept of a 
correlation coefficient, but did not communicate to agencies how the 
scores were used for certification decisions. In addition, OPM gave 
this information only to PMA-scored agencies. OPM provided all agencies 
with 10 or more senior executives their correlation coefficient based 
on the fiscal year 2007 appraisal data along with some contextual 
information, but OPM does not address the concerns expressed by 
officials at the selected agencies regarding how the correlation 
coefficient is being used in certification decisions. 

With respect to agencies' working relationships with OMB, officials at 
five of the selected agencies had little to no direct contact with OMB 
through past reviews of their certification submissions and did not 
have a clear understanding of OMB's role in the certification review 
process. However, while their interaction with OMB and understanding of 
its role was limited, the selected agency officials were satisfied 
overall with how they received OMB's feedback through OPM. OMB's 
feedback on agencies' systems is most commonly communicated to agencies 
via the letter that it sends to OPM stating its concurrence or 
nonconcurrence with OPM's certification recommendation, according to 
OMB's lead official in the certification review process. For the 2008 
certification review process, OMB is working with agencies to identify 
areas of improvement for their appraisal systems and asking agencies to 
commit to working with OPM and OMB to address these areas. This 
communication and feedback to agencies is taking place prior to 
receiving OPM's and OMB's certification decisions, according to the OMB 
official. 

Future Plans and Potential Refinements for the SES Performance-Based 
Pay System and Certification Process: 

OPM is considering phasing out the distinction between full and 
provisional certification once all agencies have received full 
certification. Provisionally certified agencies receive the same pay 
flexibilities--access to higher basic pay and total compensation--as 
those with fully certified systems. Currently, agencies' systems are 
required to meet the same criteria for both types of certification, 
with the only distinction being fairly subtle differences in the degree 
to which the agencies meet the criteria, according to OPM. When 
certification began in 2004, an agency needed to only meet four of the 
nine criteria and demonstrate that its system in design would meet the 
remaining certification criteria to receive provisional certification. 
We reported that it would be important for OPM to continue to monitor 
the certification process, especially for those agencies' systems with 
provisional certification, to help ensure that provisional 
certifications do not become the norm.[Footnote 32] After that initial 
year, agencies' systems were required to meet all nine certification 
criteria to receive provisional certification, which according to a 
senior OPM official was done in part because agencies would have had 
the opportunity to produce performance data showing that meaningful 
distinctions were made by the second year. OPM expects that all 
applicant SES appraisal systems will meet full certification criteria 
in the future. As of October 28, 2008, 76 percent of the agency 
certification submissions reviewed by OPM and OMB had received full 
certification for calendar year 2008. According to a senior OPM 
official, OPM is currently revisiting the certification regulations to 
determine what revisions should be made. 

Regarding potential refinements to the certification process, officials 
at the selected agencies identified a need for increased efficiency in 
the certification submission process given the large quantity of 
documents required and the time it currently takes to gather the 
certification package. 

* For example, officials at five of the selected agencies are in favor 
of moving to an electronic submission process, if it reduces the amount 
of paper documentation that is required and streamlines the process of 
compiling the necessary documentation for agencies. Selected agency 
officials acknowledged that the certification process is moving in that 
direction with the SES-PAAT, which is an electronic tool. For agencies 
completing the SES-PAAT, OPM requested five copies of the completed 
tool and supporting documentation, such as the sample of performance 
plans and organizational performance assessments. A senior human 
resources official who is not using the SES-PAAT said her agency 
submitted to OPM seven copies of its certification package for 2007. 
While OPM has informally encouraged agencies through its executive 
resources forums or e-mail communications to submit their certification 
submissions electronically, according to a senior OPM official OPM has 
not directly discussed electronic certification submissions in its 
guidance and submission instructions to the agencies. In addition, 
officials at three of the selected agencies said that OPM could lessen 
agencies' reporting burden by only requiring them to submit 
documentation to OPM and OMB on those areas of the system where 
substantive changes have occurred since the previous certification 
submission. While OPM and OMB should receive completed certification 
submissions as part of their oversight responsibilities, moving to 
electronic documentation may help agencies more easily gather and 
submit the certification documents that change minimally from year to 
year. OPM and OMB officials both said they support electronic 
certification submissions and would not rule out a more electronic 
certification process in the future, starting with the SES-PAAT. 

* Overall, selected agency officials supported lengthening the 
certification coverage beyond 2 years once agencies have proved their 
systems are addressing the certification criteria. Currently, full 
certification lasts for 24 months. Specifically, the process of 
gathering the necessary documentation for certification submissions 
would be less burdensome to agencies if they had more time between 
recertification deadlines, according to a senior human resources 
official. Prior to receiving an extended certification coverage period, 
the agency officials acknowledged that their systems would have to be 
operating at the fully certified level. The lead OMB official 
overseeing the certification review process said OMB could potentially 
be supportive of lengthening the certification coverage, once agencies 
have their systems up and running and all agencies have full 
certification. Recognizing extending coverage would require a statutory 
change, OPM is reviewing the recently passed law that changes the 
certification coverage to up to 24 months from a calendar-year-based 
coverage, among other things, and OPM is not taking a position on 
extending certification coverage at this time. 

Conclusions: 

OPM and OMB have taken important steps in overseeing the design and 
implementation of agencies' senior executive performance- based pay 
systems. The selected agencies' experiences with implementing their SES 
systems can help inform other agencies' efforts to hold executives 
accountable for organizational results and link pay for performance. 
USAID has an opportunity to strengthen the link between organizational 
and individual performance by providing rating and reviewing officials 
with specific information on organizational performance. For senior 
executives to lead the way in transforming their organizations to meet 
the complex challenges facing the nation, it will be important for them 
to have confidence that the SES performance-based pay system is 
operating as intended and that they will be rewarded according to their 
performance. In this regard, making meaningful distinctions among 
executives when assessing performance is important to the overall 
credibility of the SES performance-based pay system. Less than a third 
of senior executives governmentwide strongly agreed or agreed that 
bonuses or pay distinctions were meaningfully different among senior 
executives, according to OPM's SES survey. OPM has an opportunity to 
help ensure that agencies are making meaningful distinctions in 
executive performance by strengthening its communication with agencies 
and executives on the importance of using a range of rating levels when 
assessing performance, while avoiding forced distributions. 
Additionally, communicating how OPM uses data and other tools in making 
certification decisions will be important so that agencies can make 
continuous improvements to their systems to support the development of 
a stronger performance culture and the attainment of their missions, 
goals, and objectives. Moving forward, it will also be important for 
OPM and OMB to identify ways to improve the certification process and 
make it more streamlined while ensuring that agencies have the 
guidance, tools, and training they need to implement effective 
performance appraisal and pay systems for their senior executives. 


Recommendations for Executive Action: 

To help ensure consistency and clarity in how organizational 
performance is considered in appraising executive performance, we 
recommend that the Administrator of USAID provide uniform 
organizational performance assessments to PRB members and other 
reviewing officials to help inform their appraisal recommendations for 
senior executives at the end of the performance appraisal cycle. 

To help improve agencies' understanding of certain aspects of the 
certification decisions, we recommend two areas for the Acting Director 
of OPM to take action to strengthen OPM's communication with agencies 
and executives on: 

* the importance of making meaningful distinctions in performance while 
avoiding the use of forced distributions and that a fully successful 
rating is valued and rewarded and: 

* how it uses the SES performance appraisal data and the correlation 
between ratings and performance pay in determining whether agencies are 
making meaningful distinctions based on relative performance as 
measured though the pay and performance differentiation certification 
criteria. 

In addition, to help improve the efficiency of the certification 
submission process for agencies, we recommend that the Acting Director 
of OPM and Director of OMB explore opportunities for streamlining the 
certification process, such as electronic submissions or lengthening 
the full certification coverage beyond 2 years for agencies that 
received full certification. 

Agency Comments and Our Evaluation: 

We provided a copy of the draft report to the Secretaries of Defense, 
Energy, State, and the Treasury; the Commissioners of NRC; the 
Administrator of USAID; the Acting Director of OPM; and the Director of 
OMB for their review and comment. 

DOE had no comments on the draft report. We received written comments 
from DOD and OPM, which are included in appendixes III and IV. NRC, 
OMB, State, Treasury, and USAID provided clarifying and technical 
comments, which we incorporated as appropriate. With respect to making 
meaningful distinctions in performance, Treasury officials provided 
broad comments on the challenge agencies face in using the full range 
of rating levels given that executives are high achievers and had to 
exhibit exceptional performance to enter the SES. The officials 
suggested that meaningful distinctions can not be defined simply by the 
distribution of performance ratings. Our findings and recommendations 
about making meaningful distinctions acknowledged challenges, but also 
the need for additional communication from OPM in this area. 

Regarding our recommendations, USAID, OPM, and OMB expressed general 
agreement. The Acting Director stated that OPM looks forward to working 
with agencies and OMB to find ways to further improve communications 
with the agencies concerning the certification process. OMB generally 
agreed with our assessment and recommendation regarding the 
possibilities of streamlining the certification process to improve 
efficiency and potentially extending full certification coverage beyond 
2 years. OMB stated that it agrees with OPM that careful review of the 
newly passed law and its effect will be necessary before considering 
such an extension. Regarding our discussion of pay compression, OPM 
stated that it is not comfortable with the identification of tiers as a 
means to address SES pay compression. The Acting Director commented 
that using tiers results in salaries clustering around points in the 
salary range rather than only at the top of the range. While we 
recognize OPM's concern about agencies' use of tiers, we are not 
recommending the use of tiers as a way for agencies to address future 
problems with pay compression and have revised the language in the 
report to clarify this point. 

As agreed with your offices, unless you publicly announce the contents 
of this report earlier, we plan no further distribution of it until 30 
days from the date of this letter. At that time, we will send copies of 
this report to the appropriate congressional committees; the 
Secretaries of Defense, Energy, State, and the Treasury; the 
Commissioners of NRC; the Administrator of USAID; the Acting Director 
of OPM; the Director of OMB; and other interested parties. The report 
will also be available at no charge on the GAO Web site at [hyperlink, 
http://www.gao.gov]. 

If you or your staff have any questions regarding this report, please 
contact me at (202) 512-6806 or goldenkoffr@gao.gov. Contact points for 
our Offices of Congressional Relations and Public Affairs may be found 
on the last page of this report. GAO staff who made major contributions 
to this report are listed in appendix V. 

Signed by: 

Robert N. Goldenkoff: 
Director, Strategic Issues: 

[End of section] 

Appendix I: Objectives, Scope, and Methodology: 

This report examines selected agencies' policies and procedures for 
their career Senior Executive Service (SES) performance appraisal and 
pay systems in (1) factoring organizational performance into senior 
executive performance appraisal decisions, (2) making meaningful 
distinctions in assessing and rewarding senior executive performance, 
and (3) building safeguards into senior executive performance appraisal 
and pay systems. In addition, this report examines how the Office of 
Personnel Management (OPM) and the Office of Management and Budget 
(OMB) are providing oversight in the certification of the senior 
executive performance-based pay system through their statutory roles. 

We selected the U.S. Departments of Defense, Energy, State, and the 
Treasury; the U.S. Nuclear Regulatory Commission (NRC); and the United 
States Agency for International Development based on variations in 
agency mission, organizational structure, size of their career SES 
workforces to reflect agencies with a large, average, and small number 
of executives; and results of their SES performance appraisal systems 
in terms of the percentage of SES rated at the highest rating levels 
and the percentage that received performance awards or bonuses from 
fiscal years 2004 to 2006, according to OPM's governmentwide data 
reports. We focused on career members of the SES because they represent 
the majority of all SES appointments governmentwide. 

To address our first and third objectives, we analyzed selected 
agencies' documents on their SES performance appraisal and pay systems 
including performance management policies, directives, guidance, and 
other related information; performance planning and appraisal 
templates; briefings, memos, and other materials used to communicate 
aggregate appraisal results to senior executives; and the submission 
documents to OPM and OMB for certification in 2007. We also interviewed 
cognizant senior-level officials in the human capital offices at the 
selected agencies regarding their agencies' SES systems and senior- 
level OPM and OMB officials on how organizational performance and 
safeguards are integrated into the certification process. 

To address our second objective, we analyzed aggregate SES basic pay, 
performance rating, bonus, and pay adjustment data as provided by the 
agencies for fiscal year 2007. We defined our universe of analysis as 
career senior executives who received ratings. In calculating the 
percentage of eligible senior executives who received bonuses (cash 
awards) or pay adjustments (increases to basic pay) and average (or 
mean) amounts, we excluded executives who received a rating less than 
"fully successful" (level 3), as applicable, from the eligible 
population since those executives are not eligible to receive bonuses, 
according to the selected agencies' policies, or pay increases, 
according to OPM regulation. We also excluded senior executives at NRC, 
Treasury, and State who received Presidential Rank Awards from our 
calculations of percentages of eligible SES members receiving bonuses 
and average amounts because those individuals were not considered for 
bonuses that year, according to the agencies' policies. In order to 
have consistency in our analysis across selected agencies, we included 
senior executives who were rated but left their positions--because of 
retirement, attrition, or assignment to a lower grade--prior to 
performance payouts being made in our analysis. The agencies' policies 
and practices varied in whether or not senior executives who retired 
were eligible for performance payouts. 

In analyzing the basic pay rates, we used the rates of basic pay for 
rated, career senior executives after pay adjustments were made for the 
fiscal year 2007 appraisal cycle. We compared these basic pay amounts 
to the governmentwide SES basic pay range and the Executive Schedule 
(EX) pay levels, specifically EX-level II (the SES pay cap) and EX- 
level III (within the SES pay band) for calendar year 2008. EX pay 
rates are set at the beginning of each calendar year and because 
performance payouts from the completed appraisal cycle are usually paid 
out in January, agencies are able to use the next calendar year SES pay 
rates for their pay ranges. In calculating the percentage of senior 
executives paid at or above EX-level III, we included those SES paid at 
EX-level II in our calculations. For this objective, we also analyzed 
relevant agency policies and guidance regarding performance ratings, 
bonuses, and pay adjustments, and interviewed selected senior-level 
agency officials from the human capital offices. For the governmentwide 
perspective, we reviewed OPM's Report on Senior Executive Pay for 
Performance for Fiscal Year 2007 to identify relevant governmentwide 
performance data as a comparison point for the selected agencies. We 
analyzed OPM memos and guidance on the certification process and 
criteria, and interviewed senior-level OPM and OMB officials who 
oversee the certification review process on how the SES performance 
data are used as part of the certification process. 

We checked the agency data for reasonableness and the presence of any 
obvious or potential errors in accuracy and completeness. We also 
reviewed related agency documentation, interviewed agency officials 
knowledgeable about the data, and brought to the attention of these 
officials any concerns or discrepancies we found with the data for 
correction or updating. The agency officials confirmed the correctness 
of the data or in some cases provided corrections to the data, which we 
used in our analysis. On the basis of these procedures, we believe the 
data are sufficiently reliable for use in the analyses presented in 
this report. 

For our fourth objective, we analyzed OPM's and OMB's guidance, memos, 
and other documents regarding the certification process and criteria; 
interviewed senior-level OPM and OMB officials involved in the 
certification process regarding their oversight of the certification 
process; and interviewed senior-level agency officials from the 
selected agencies' human capital offices to obtain their perspectives 
on OPM and OMB oversight. As part of this objective, we are reporting 
on the status of OPM's progress toward addressing recommendations from 
our past work on the SES performance-based pay system and certification 
process.[Footnote 33] 

We conducted our work from October 2007 through November 2008 in 
accordance with generally accepted government auditing standards. 

[End of section] 

Appendix II: Certification Criteria for the Senior Executive 
Performance-Based Pay System: 

In November 2003, Congress authorized a new performance-based pay 
system for members of the Senior Executive Service.[Footnote 34] 
Agencies are to base pay adjustments for senior executives on 
individual performance and contributions to the agency's performance by 
considering the individual's accomplishments and such things as unique 
skills, qualifications, or competencies of the individual and the 
individual's significance to the agency's mission and performance, as 
well as the individual's current responsibilities. If an agency's 
senior executive performance appraisal system is certified by the 
Office of Personnel Management (OPM) and the Office of Management and 
Budget (OMB) concurs, an agency can raise the pay cap for its senior 
executives to $172,200 for basic pay (Level II of the Executive 
Schedule) and $221,100 for total compensation (the total annual 
compensation payable to the Vice President). 

To qualify for senior executive pay flexibilities, agencies' 
performance appraisal systems are evaluated against nine certification 
criteria and any additional information that OPM and OMB may require to 
make determinations regarding certification. As shown in table 4, the 
certification criteria jointly developed by OPM and OMB are broad 
principles that position agencies to use their pay systems 
strategically to support the development of a stronger performance 
culture and the attainment of their mission, goals, and objectives. 

Table 4: OPM's and OMB's Senior Executive Performance Appraisal System 
Certification Criteria: 

Criterion: Alignment; 
Description: Individual performance expectations must be linked to or 
derived from the agency's mission, strategic goals, program/policy 
objectives, or annual performance plan. 

Criterion: Consultation; 
Description: Individual performance expectations are developed with 
senior employee involvement and must be communicated at the beginning 
of the appraisal cycle. 

Criterion: Results; 
Description: Individual expectations describe performance that is 
measurable, demonstrable, or observable, focusing on organizational 
outputs and outcomes, policy/program objectives, milestones, and so 
forth. 

Criterion: Balance; 
Description: Individual performance expectations must include measures 
of results, employee and customer/stakeholder satisfaction, or 
competencies or behaviors that contribute to outstanding performance. 

Criterion: Assessments and guidelines; 
Description: The agency head or a designee provides assessments of the 
performance of the agency overall, as well as each of its major program 
and functional areas, such as reports of agency's goals and other 
program performance measures and indicators, and evaluation guidelines 
based, in part, upon those assessments to senior employees and 
appropriate senior employee rating and reviewing officials. The 
guidance provided may not take the form of quantitative limitations on 
the number of ratings at any given rating level. 

Criterion: Oversight; 
Description: The agency head or designee must certify that (1) the 
appraisal process makes meaningful distinctions based on relative 
performance; (2) results take into account, as appropriate, the 
agency's performance; and (3) pay adjustments and awards recognize 
individual/organizational performance. 

Criterion: Accountability; 
Description: Senior employee ratings (as well as subordinate employees' 
performance expectations and ratings for those with supervisor 
responsibilities) appropriately reflect employees' performance 
expectations, relevant program performance measures, and other relevant 
factors. 

Criterion: Performance differentiation; 
Description: Among other provisions, the agency must provide for at 
least one rating level above fully successful (must include an 
outstanding level of performance), and in the application of those 
ratings, must make meaningful distinctions among executives based on 
their relative performance. 

Criterion: Pay differentiation; 
Description: The agency should be able to demonstrate that the largest 
pay adjustments, highest pay levels (basic and performance awards), or 
both are provided to its highest performers, and that overall the 
distribution of pay rates in the SES rate range and pay adjustments 
reflects meaningful distinctions among executives based on their 
relative performance. 

[End of table] 

Source: GAO. 

[End of section] 

Appendix III: Comments from the Department of Defense: 

Office Of The Under Secretary Of Defense: 
4000 Defense Pentagon: 
Washington, DC 20301-4000: 

Personnel and Readiness: 

November 24, 2008: 

Mr. Robert N. Goldenkoff: 
Director, Strategic Issues, U.S. Government Accountability Office: 
U.S. Government Accountability Office: 
441 G Street, N.W.: 
Washington, DC 20548: 

Dear Mr. Goldenkoff, 

This is the Department of Defense (DoD) response to the GAO draft 
report, GAO- 09-82, `Results-Oriented Management: Opportunities Exist 
for Refining the Oversight and Implementation of the Senior Executive 
Performance-Based Pay System,' dated October 29, 2008 (GAO Code 
450628). 

The Department appreciates the opportunity to respond to the audit. The 
analysis of selected agency practices and policies will be useful as 
the Department continues to refine its own performance management 
practices. The Department's comments are generally editorial in nature, 
however, they provide additional clarity to help inform the DoD effort. 
They are: 

1. The title of the DoD official is, "Principal Director to the Deputy 
Under Secretary of Defense for Civilian Personnel Policy." You can 
shorten it to Principal Director. 

2. Page 13: Suggest a slight rewrite of the last sentence in the DoD 
bullet. "According to the Principal Director to the Deputy Under 
Secretary of Defense for Civilian Personnel Policy, the DoD Components 
had the flexibility to use both the department wide organizational 
assessment and their own organizational assessments. Component 
organizational assessments were required to be linked to the department 
wide priorities and end of year assessment. Component organizational 
assessments can provide a level of specificity that enables a clearer 
connection or "line of sight" between individual executive performance 
and organizational performance." 

3. Page 18. Suggest an additional sentence to the last paragraph before 
Table 2. "DOD also uses the Tier Structure to differentiate executive 
payouts (basic pay increases and bonuses) to recognize high level 
performance in some positions has more impact than comparable 
performance in others. 

4. Page 21: Suggest a slight rewrite to sentence 5, second paragraph. 
"DoD is communicating the message that the new pay for performance 
management system recalibrates performance assessments as a way to help 
change the culture and make meaningful distinctions in performance. A 
fully successful or equivalent rating is a high bar standard as well as 
a valued and quality rating. Levels above this require extraordinary 
results." Part of this communication... 

5. Page 29: Suggest a slight rewrite to sentence at the top of the 
page. "...The Principal Director to the Deputy Under Secretary of 
Defense for Civilian Personnel Policy said when pay increases are not 
possible given salary cap issues, bonuses are a tool for Components to 
reward their executives' performance for achieving results." Last 
sentence — A comment for your consideration. Tiers are the most common 
way for agencies to control salary costs. However, tiers can exacerbate 
pay compression issues. 

If you have additional questions, please do not hesitate to contact me 
at 703-614-9487. 

Signed by: 

Patricia Bradshaw: 
Deputy Under Secretary of Defense: 
(Civilian Personnel Policy): 

[End of section] 

Appendix IV: Comments from the Office of Personnel Management: 

United States Office Of Personnel Management: 
Washington, DC 20415:  
[hyperlink, http://www.opm.gov]: 
"Our mission is to ensure the Federal Government has an effective 
civilian workforce.": 
[hyperlink, http://www.usajobs.gov]: 

November 18, 2008: 

The Honorable Gene L. Dodaro: 
Acting Comptroller General: 
U.S. Government Accountability Office: 
441 G Street, NW: 
Washington, DC 20548: 

Dear Mr. Dodaro: 

Thank you for the opportunity to provide comments in response to the 
U.S. Government Accountability Office (GAO) draft report entitled 
Results-Oriented Management: Opportunities Exist for Refining the 
Oversight and Implementation of the Senior Executive Performance-Based 
Pay System (GAO-09-82). 

I appreciate GAO's recommendations for improving communications with 
the agencies concerning the executive appraisal system certification 
process. I agree that communication about system certification with the 
agencies is vitally important. The U.S. Office of Personnel Management 
has already taken a number of steps to improve our communications with 
the agencies, particularly through workshops, forums, Chief Human 
Capital Officer Council activities, and briefings for individual 
agencies as well as interagency groups. We have plans to conduct 
additional workshops on system certification using the Senior Executive 
Service Performance Appraisal Assessment Tool (SES-PAAT), which we 
recently introduced to improve the certification process. We look 
forward to working with agencies and the Office of Management and 
Budget to find ways to further improve the process. 

We have one observation about the report's discussion regarding pay 
compression and agencies establishing pay tiers within the broad pay 
range. While we agree that pay compression is a problem, we are not 
comfortable with the identification of tiers as a means to address SES 
pay compression. Tiers are used by agencies for a purpose other than 
relieving pay compression to recognize varying levels of responsibility 
and difficulty among SES positions, with the most demanding jobs 
falling in the top pay tier. This strategy results in a greater spread 
among the salaries of individual SES members, and ensures that SES 
members in the toughest jobs don't receive the same rate of basic pay 
as other Senior Executives. However, using pay tiers does not 
necessarily address the issue of pay compression. Instead, it results 
in salaries clustering around two or three points in the salary range 
rather than only at the top of the range. Also, the growing concern 
about compression is mostly fueled by the fact there is greater upward 
movement each January in the General Schedule (GS) than the Executive 
Schedule, resulting in a smaller and smaller difference each year 
between GS-15 and SES salaries. If anything, tiers would probably make 
the difference between the average SES and GS 15 salary even smaller. 

Again, thank you for allowing me to comment on this report. I am 
providing specific technical corrections to the draft report and would 
ask for your consideration of these changes. 

Enclosure: 

[End of section] 

Appendix V: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Robert N. Goldenkoff, (202) 512-6806, goldenkoffr@gao.gov: 

Staff Acknowledgments: 

In addition to the individual named above, Belva Martin, Assistant 
Director; Amber Edwards; Janice Latimer; Donna Miller; Meredith Moore; 
Mary Robison; Sabrina Streagle; and Greg Wilmoth made major 
contributions. 

[End of section] 

Footnotes: 

[1] National Defense Authorization Act for Fiscal Year 2004, Pub. L. 
No. 108-136, November 24, 2003; 5 U.S.C. § 5382. 

[2] Relative performance means the performance of a senior employee 
with respect to the performance of other senior employees, including 
their contribution to agency performance, where appropriate, as 
determined by the application of a certified appraisal system. 5 C.F.R. 
§ 430.402. 

[3] GAO, Human Capital: Symposium on Designing and Managing Market- 
Based and More Performance-Oriented Pay Systems, [hyperlink, 
http://www.gao.gov/products/GAO-05-832SP] (Washington, D.C.: July 27, 
2005). 

[4] GAO, Human Capital: Senior Executive Performance Management Can Be 
Significantly Strengthened to Achieve Results, [hyperlink, 
http://www.gao.gov/products/GAO-04-614] (Washington, D.C.: May 26, 
2004). 

[5] These three areas fit into the alignment, results, assessments and 
guidelines, performance differentiation, and pay differentiation 
criteria. See app. II for a description of all nine criteria. 

[6] GAO, Human Capital: Selected Agencies Have Implemented Key Features 
of Their Senior Executive Performance-Based Pay Systems, but 
Refinements Are Needed, [hyperlink, http://www.gao.gov/products/GAO-08-
1019T] (Washington, D.C.: July 22, 2008). 

[7] 5 C.F.R. § 430.301(b). 

[8] The Homeland Security Act of 2002, Pub. L. 107-296, § 1322, 
November 25, 2002; 5 U.S.C. § 5307. 

[9] National Defense Authorization Act for Fiscal Year 2004, Pub. L. 
No. 108-136, November 24, 2003; 5 U.S.C. § 5382. 

[10] Senior Professional Performance Act of 2008, Pub. L. No. 110-372, 
§ 3, October 8, 2008. 

[11] Senior Professional Performance Act of 2008, Pub. L. No. 110-372, 
§ 3, October 8, 2008. 

[12] GAO, Office of Personnel Management: Key Lessons Learned to Date 
for Strengthening Capacity to Lead and Implement Human Capital Reforms, 
[hyperlink, http://www.gao.gov/products/GAO-07-90] (Washington, D.C.: 
Jan. 19, 2007). 

[13] GAO, Results-Oriented Cultures: Creating a Clear Linkage between 
Individual Performance and Organizational Success, [hyperlink, 
http://www.gao.gov/products/GAO-03-488] (Washington, D.C.: Mar. 14, 
2003). 

[14] [hyperlink, http://www.gao.gov/products/GAO-04-614]. 

[15] State's senior executive performance appraisal cycle ended on July 
31, 2008. The appraisal cycle is from August 1 to July 31. 

[16] Agencies are permitted to award bonuses from 5 to 20 percent of an 
executive's rate of basic pay from a pool that cannot exceed the 
greater of 10 percent of the aggregate rate of basic pay for the 
agency's career SES appointees for the year preceding, or 20 percent of 
the average annual rates of basic pay to career SES members for the 
year preceding; 5 U.S.C. § 5384. 

[17] DOE uses a four-level appraisal system with no rating level 
between outstanding and meets expectations. 

[18] GAO, Financial Regulators: Agencies Have Implemented Key 
Performance Management Practices, but Opportunities for Improvement 
Exist, [hyperlink, http://www.gao.gov/products/GAO-07-678] (Washington, 
D.C.: June 18, 2007). 

[19] Given that they both have SES and Senior Foreign Service (SFS) 
members, State and USAID discuss issues of executive compensation and 
other executive resources areas in order to maintain comparability for 
SES bonuses and pay adjustments between the agencies and with the SFS, 
according to agency officials. The Foreign Service Act of 1980, as 
amended, limits the percentage of SFS members who can receive bonuses 
to 33 percent; 22 U.S.C. § 3965. According to a senior-level State 
official, the percentage of SES members receiving bonuses has increased 
recently to between 50 to 60 percent at State due to an effort to 
offset regular pay increases SFS members receive through promotions. 

[20] Agencies can nominate senior executives for Presidential Rank 
Awards, which recognize career senior executives who have demonstrated 
exceptional performance over an extended period of time. The OPM 
Director reviews agency nominations and recommends candidates to the 
President. These awards are either 20 percent or 35 percent of the 
recipient's basic pay. 

[21] 5 C.F.R. § 430.404(a)(9). 

[22] At NRC, the majority of senior executives rated at level 4 who did 
not receive pay increases retired before performance payouts were made 
for the fiscal year 2007 appraisal cycle. 

[23] [hyperlink, http://www.gao.gov/products/GAO-05-832SP] and GAO, 
Human Capital: Trends in Executive and Judicial Pay, [hyperlink, 
http://www.gao.gov/products/GAO-06-708] (Washington, D.C.: June 21, 
2006). 

[24] Bonuses awarded to senior executives are cash payments not added 
to the executives' basic pay and therefore do not count toward the 
governmentwide basic pay cap. 

[25] [hyperlink, http://www.gao.gov/products/GAO-05-832SP]. For more 
information, including the complete list of safeguards, see GAO, Human 
Capital: DOD Needs to Improve Implementation of and Address Employee 
Concerns about Its National Security Personnel System, [hyperlink, 
http://www.gao.gov/products/GAO-08-773] (Washington, D.C.: Sept. 10, 
2008). 

[26] 5 U.S.C. § 4312(b)(3). 

[27] Senior Professional Performance Act of 2008, Pub. L. No. 110-372, 
§ 3, October 8, 2008. 

[28] The correlation coefficient ranges from -1 to +1. An agency that 
has a strong relationship between ratings and performance pay will have 
a positive correlation coefficient close to +1, which indicates that 
the executives' summary ratings are the agencies' primary bases for 
determining their performance pay. An agency that has a strong negative 
relationship between ratings and performance pay will have a 
coefficient near -1. A coefficient of zero indicates there is no 
correlation between ratings and performance pay. 

[29] [hyperlink, http://www.gao.gov/products/GAO-07-90]. 

[30] [hyperlink, http://www.gao.gov/products/GAO-07-90]. 

[31] An Executive Order in November 2007 required agency heads to 
appoint performance improvement officers as members of the Performance 
Improvement Council. These officers are responsible for coordinating 
the organization's performance management activities including 
assisting the agency head in developing and using performance measures 
in individual performance appraisals to ensure accountability for 
greater effectiveness. Exec. Order No. 13,450, 72 Fed. Reg. 64,517 
(Nov. 15, 2007). 

[32] GAO, Human Capital: Agencies Need Leadership and the Supporting 
Infrastructure to Take Advantage of New Flexibilities, [hyperlink, 
http://www.gao.gov/products/GAO-05-616T] (Washington, D.C.: Apr. 21, 
2005). 

[33] [hyperlink, http://www.gao.gov/products/GAO-07-90]. 

[34] National Defense Authorization Act for Fiscal Year 2004, Pub. L. 
No. 108-136, November 24, 2003; 5 U.S.C § 5382. 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site, 
[hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional 
information. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: