PLAN FOR RATING APPLICANTS

• Introduction

• Developing the Examining Plan

• Issuing the Announcement

INTRODUCTION

When an agency announces an examination for any occupation, it must have an examining plan. Plans for in service placement actions used by human resources or personnel offices follow the same basic principles as those used for outside hiring. The plan should include the job elements, a crediting plan for rating applicants, announcement information, forms, questionnaires to be used, and other rating information.

Ideally, a panel composed of examiners and agency operating officials who know the job requirements prepare the examining plan. In this way operating officials and other interested parties play a vital role in the examining process which is directly related to local job requirements. The number of panel members and the methods used vary, depending on the nature of the job, or jobs, and local considerations.

In preparing the examining plan, the panel can adapt material from examining guides to meet local requirements. For example, general instructions, announcement paragraphs, application forms, suggested crediting plans, and other appropriate material (not necessarily all of these in each case, however) may be included. 

DEVELOPING THE EXAMINING PLAN

Developing the examining plan usually requires a number of steps:

1. Studying the job

   A thorough understanding of the work and the characteristics (or job elements) needed for success is necessary.

2. Identifying the job elements

   Identifying the proper set of elements for a particular job involves:

   • Deciding which of the five job categories is appropriate;

   • Comparing the job to be filled to those described in the category; and

   • Selecting the set of approved elements most appropriate for the job.

3. Deciding how to obtain information about applicants

   A decision must be made as to the best way to obtain information about the degree to which applicants possess required abilities. Forms or oral interviews designed to elicit this information should:

   • Have questions directly related to the elements for the job;

   • Be limited to relevant data needed to form valid judgments about the applicant's ability to do the job; and

   • Be worded in a manner that will be easily understood and answered by applicants.

   The selection of application forms, checklists, etc., depend on the nature of the job to be filled. Forms may be modified or new forms be developed for local use.

   1. Short-Answer Supplemental Experience Statement Sheets

      Supplemental experience sheets may be used to obtain more information from the applicant if necessary. Such forms should be kept simple, and, to the extent possible, they should not require skill in writing. In developing forms locally, rating guides may be used as a reference for the kinds of information usually needed in connection with job elements.

   2. Reference Checks or Qualification Questionnaires

      While reference checks or qualification questionnaires are not required, they may be useful to support or supplement information received from applicants:

      • When supervisors can be contacted easily;

      • When it is not practicable to get information from applicants; or

      • When there is some doubt concerning the statements made by an applicant.

      Qualification inquiries should be made of sources such as employers and schools when available information indicates the possible existence of an unfavorable record.

      If the desired information is obtained by telephone, the examiner should ask essentially the same questions that are asked on the application form, and a brief written summary of the information should be kept with the application.

      As in other examining procedures, present employers should not be contacted unless permission is given by applicants, and all information received should be considered confidential.

   3. Tests

      In deciding whether tests will be used, the value of the information to be obtained should be considered in relation to the availability of tests, their cost, and time factors.

      If tests are used as one indication of ability, the examining plan should provide complete information as to the type of test, its use in measuring an element, and a method of converting test results to point values.

4. Preparing the crediting plan

   Preparation of the crediting plan is one of the most important steps in examining for a job. The plan contains descriptions of specific kinds of acceptable experience, training, or other information for each element. Information illustrating degrees of potential or ability for an element are grouped as follows:

   • Demonstrated superior ability (4 points)

   • Demonstrated satisfactory ability (3 points)

   • Barely acceptable or potentially satisfactory ability (2 points)

   • Information which is of some value, but indicates less than barely acceptable ability (1 point)

   Point value descriptions are best prepared with the assistance of subject-matter experts who can describe the shop practices, materials, equipment, etc., using terminology of the occupation. They are usually developed after review of the applications received. In this manner, all of the possible kinds of information that will be needed in rating the applications are included. A good understanding of requirements is increased in group discussion; and a group interpretation tends to be more generally applicable than one adopted by an individual rater without consultation.

   After the crediting plan is prepared, it should be tested to be sure that it will be a useful tool in assessing the potential of applicants for success on the job, e.g., that the highly qualified applicants will be given the most credit, etc.

   1. a. Rating Guides

      Rating guides developed by the Office of Personnel Management are helpful in preparing local crediting plans.

      Each rating guide contains a definition of the particular job element; levels of ability within the element; work examples where appropriate; and typical questions that may be asked of competitors. These examples and descriptions are broad and general because the rating guides cover a variety of jobs in which the element is important.

      A rating guide has two major purposes:

      (1) It enables the panel to select the example that most nearly matches the level of ability required in the job to be filled. The example selected serves as a general description of demonstrated satisfactory ability (3-point value) and represents the mid-point of the eligibility range.

      (2) It provides a basis for developing point value descriptions in the local crediting plan. The general description of demonstrated satisfactory ability (3-point value) can be expanded to fit the local job requirement.

      Using the 3-point value description as a reference, often one of the higher levels of ability in the rating guide can be selected as a guide for the 4-point value description. Similarly, lower levels usually serve as guides in preparing 2 and 1-point value descriptions.

   2. b. Point Value Descriptions

      Point value descriptions should be developed for the particular job only in terms of actual local conditions, not on assumed requirements. However, the panel should avoid over specificity of requirements; for example, it would not be proper to require experience on the particular make and model of lathe used locally. A suggested method is described below:

      (1) 3-point value description:

      Each selected level of ability (3-point value from rating guide) is expanded to a more detailed and specific description. For example, the rating guide for the level "Ability to perform the common tasks associated with the journeyman duties in a line of work on employee's own initiative, subject to occasional inspection" for Element No. 1, could be expanded to read as follows for a WG-9 Carpenter job:

      Able to plan and complete work involving the construction of building additions and modifications (put up frameworks, rafters, bearings and trusses, frame windows and doors; install dry walls; complete trim work, etc.) obtaining exact trueness, precise fits and joints, and a finished appearance with structural soundness. Uses any of the accepted methods, techniques, and devices of the trade, with little or no advice from supervisor, except on unusual problems. Uses a variety of shop woodworking equipment and precision layout and measuring devices such as levels, plumb lines, and framing squares, as well as common hand and power tools. Able to read, interpret and apply building plans, specifications, sketches and material guides, and do arithmetic and simple geometry. Overall work is checked to see that it meets trade standards.

      (2) 4, 2, and 1-point value descriptions:

      After the 3-point value is described, it is used as a reference point to develop the 4, 2, and 1-point descriptions.

      A 4-point value description encompasses the 3-point description, but describes superior performance by ability to do common tasks with complete independence, or to do complex or unusual tasks and solve unusual problems.

      A 2-point value description covers barely acceptable or potential ability to perform at the level described for 3 points by ability to do common tasks under close supervision; by intermittent, occasional performance on an independent basis; or by other comparable indicators of ability.

      A 1-point value description covers ability which is of some value in the rating, but is not evidence of barely acceptable or potential ability to perform at the 3-point level.

      • Information that illustrates lower than the 1-point value need not be described.

   3. c. Supplemental Point Value Information

      It may be desirable and useful to supplement point value descriptions with examples of work experience and training, test scores, or other evidence of ability which may be obtained from applications, vouchers, discussions with panel members and other sources. Examples of this for a journeyman in a trade are as follows:

      • Four points might be illustrated by an applicant's statement that he served as a leader or first-line supervisor and has demonstrated ability as a satisfactory journeyman. Four points might also be shown by demonstrated superior ability of a journeyman as recognized by awards, supervisory appraisals, or other testimonials.

      • Two points might be shown by an example of work experience where common tasks associated with the journeyman job were performed under close supervision. Similar examples could be shown for each of the other point values.

   4. d. Adjusting the Crediting Plan

      During the rating of applications, if it becomes evident that an insufficient number of eligibles will be obtained, the 2-point criteria for each element should be checked to insure that requirements have not been overstated. (However, caution must be used to avoid lowering the criteria below the barely acceptable or potential ability level. Any significant reduction of qualification requirements may well dictate a reevaluation of the job to insure proper grade relationship.)

      If the crediting plan is correct, shortage recruiting procedures may be used.

   5. e. Testing the Crediting Plan

      Before the first group of applicants is notified of their ratings, the completed crediting plan should be tested to insure uniform interpretation and realistic rankings. Testing may be done as follows:

      (1) Testing with Employees:

      Supervisors select five or more employees who are considered to be excellent in job performance and five or more who are considered satisfactory. These employees should provide information similar to that required of applicants. They are then rated on the elements.

      If an excellent employee receives a low total score, or if a satisfactory employee receives a high total score, the crediting plan should be reviewed to see whether the proper amount of credit is being given for the various kinds of experience. Adjustments should be made accordingly.

      (2) Testing with Applicants:

      The panel selects a set of 10 to 15 applications which can be ranked fairly obviously from the poorest to the best on the basis of an overall judgment of each applicant's ability. Some papers should appear to be excellent and others poor, including probable ineligibles if possible. These applications are then rated and ranked on the elements.

      If the results of the two methods of ranking do not agree, a review of the total process should be made to determine whether the ranking on the basis of overall judgment was inaccurate, the crediting plan was incorrectly applied, or one or more of the point value descriptions in the crediting plan should be changed. Adjustments are made where needed.

      If the panel disagrees by more than one point on any single element, they should determine whether the difference resulted from their application of the crediting plan or from inconsistencies in the crediting plan and take appropriate action.

5. Devising the conversion table

   The conversion table is used for converting raw scores to a rating of from 70 to 100. It may be adjusted to fit the type of ranking needed for different situations. For instance, a total raw score of 22 with 6 elements converts to a rating of 95.0; a raw score of 17 with 5 elements converts to 91.0, etc. Veteran preference points, if any, are added to the converted rating to form the final rating.

6. Documenting the crediting plan

   All decisions, modifications, or interpretations made prior to or during the rating process are recorded in the crediting plan. This is especially important for kinds of experience and training that are considered to be non-qualifying for the screen-out element, whether described in the announcement or decided in the course of rating applications.

ISSUING THE ANNOUNCEMENT

The announcement may be issued at any time after decisions are made on the job elements, tests (if appropriate), and application forms that are to be used.

Page Updated 11 July 2000