U.S. Office of Personnel Management - Ensuring the Federal Government has an effective civilian workforce

Archive

Multi-Rater Appraisal

"I'm a firm believer in the value of multi-rater appraisal!"

This endorsement was provided by Russ Bennion, one of the speakers at a popular session that was presented by the Department of Energy at OPM's National Performance Management Conference. "360-Degree Appraisal: A Multi-Rater Approach" was moderated by Tim Dirks, Deputy Assistant Secretary for Human Resources, and included: Ann Ewen, Ph.D., President of TEAMS, Inc., the consulting firm guiding many of Energy's components as they change to a team-based structure; Russ Bennion, Human Resource Advisor at Energy's Idaho National Engineering Laboratory; and Ellen Harris, Personnel Management Specialist at Energy's Morgantown Energy Technology Center.

• Concept
• An Operating Program
• A New Program

Concept. Ann Ewen began the session by explaining the concept behind using multi-rater appraisal why the feedback it provides is more credible and effective than that derived from a more traditional approach. She then spoke about the TEAMS software for computerizing the performance feedback process. TEAMS software is customized by the user organization to reflect its own particular mission, goals and values.

An Operating Program. Following Dr. Ewen, Russ Bennion described the multi-rater appraisal program used by the Idaho National Engineering Laboratory. The Laboratory has been using it as part of a major reorganization for almost 3 years. The reorganization established half the office in a large matrix structure and eliminated one layer of supervision, taking the supervisor-employee ratio from 1:4.5 to 1:17. It was recognized that a traditional performance management system would not work well with that wide a span of control, so a multi-rater program was adopted.

The performance plans for the Laboratory workforce are aligned with organizational values. Every plan contains five performance elements: professionalism, motivation, teamwork, competence, and program/mission accomplishment. The first four elements demonstrate "how" employees do their jobs. The fifth element, program/mission accomplishment, is a narrative standard written by the employee with buy-in from his or her supervisor. This element lists "what" the employee will accomplish and outlines the objectives, the strategies needed to achieve the objectives, the resources, the milestones and target dates for completion, and the measures to determine if the objectives were achieved.

Twice per year employees are asked to establish a rating team comprising 10 to 15 colleagues, customers, supervisors, and subordinates (where applicable), in addition to rating themselves. The supervisor approves or deletes/selects other raters with the agreement of the employee. Then the employee is assessed on the first four elements using an automated multi-rater program.

Once all the data are collected, an "Employee Profile" is developed showing the tabulated results of all the inputs. The employee takes this profile (the how) and the program/mission accomplishment element (the what) and sits down with the supervisor to determine the final rating for the year. The multi-rater results make up 60 percent of the rating and the accomplishment element makes up 40 percent.

Through employee involvement, it was determined that awards should not be dependent upon the end-of-year ratings of record. A new recognition system was developed emphasizing group awards and individual special act awards to be given throughout the year for specific accomplishments as they are completed.

According to Mr. Bennion, "Idaho's performance management process is still evolving even after several years. However, it gets a 70 percent approval rating from employees, compared to only 30 percent who liked the old way of performance management. The end result of the new process is increased employee empowerment and accountability for what they do and how they do it."

A New Program. At Morgantown Energy Technology Center, using multiple raters was part of the Center's larger quality approach to improve the organization. According to Ellen Harris, it was recently instituted after a successful pilot that incorporated the feedback for progress review sessions. Developed by Center employees through a process improvement team using TEAMS software, the survey instrument measures the following four factors: customer service and quality product delivery; personal attributes and behaviors as applied to the job; leadership, team dynamics, and advancement of organizational values; and job measurement, job practice, and applied technical competency. In addition, supervisors are measured on supervisory competency.

Center employees recommend 10 supervisors, customers, peers, and subordinates to form their 8-member rating team. The rating team members must be in an employee's "knowledge network" to provide valid input to the rating process. The supervisor makes the final decision as to who actually serves on the team. Once the rating period is completed, Human Resources prepares the feedback reports and distributes them to employees and their supervisors.

Morgantown will use the multi-rater feedback in the evaluation process as the most significant factor in determining the individual performance rating. A rating of "Excellent," the highest rating possible in Morgantown's three-level program, must be supported by the multi-rater survey results. The survey results will also be used for the calculation of awards and bonuses that are given out to recognize outstanding individual performance.

According to Ms. Harris, "The 360-degree feedback system is a powerful tool to communicate the mission of the organization to each employee. Since each employee has the potential to be a rater, the information in the survey instrument is communicated throughout the organization. The 360-degree feedback information also provides a valuable tool for improving individual performance as well as organizational performance."

Originally published on October 1995.