Agencies must consider many factors when selecting or developing an assessment tool and overall selection strategy. The appropriate assessment(s) depends on variables such as the number and level of vacant positions, the applicant pool, and the anticipated number of applicants based on recent experience when announcing similar positions. Other key factors that should be considered in selecting or developing an assessment strategy include:

• Critical competencies or Knowledges, Skills, and Abilities (KSAs) to be assessed
  
• Number of positions to be filled in relation to the number of applicants anticipated
  
• Reliability and validity evidence associated with the assessment tool
  
• Potential for adverse impact against race, gender, national origin or other subgroups in the applicant population
  
• Cost and expertise needed to develop the assessment tool
  
• Cost and feasibility of implementing the tool
  
• Costs or consequences of a bad hiring decision
  
• Acceptability of assessment process and tools to applicants

From a return-on-investment perspective, the validity of the assessment tool is one of the most important factors to consider when selecting an assessment strategy.

Validity is the degree to which an assessment is measuring a job-related competency or characteristic and how well it is measuring that characteristic. Validity also refers to the relationship between performance on an assessment and performance on the job. It is important to demonstrate evidence of validity for using an assessment tool for a particular purpose. There are several different types of validity evidence. For example, if you demonstrate that your assessment tool (for example, a structured interview) is measuring competencies that are critical to successfully performing a given job, you have content validity evidence. If you demonstrate that your assessment tool (for example, a written test) predicts actual performance on the job, you have evidence of the predictive validity of the assessment tool. An assessment tool that predicts performance on the job (or some other important outcome) better than another assessment tool is said to have higher predictive validity.

One or more assessment options may be used to determine an applicant's ability to successfully perform a job. Agencies must decide the number of competencies to measure, which competencies to measure, how to measure the given competencies, when in the selection process to measure given competencies, and how to combine scores with other components of the assessment process. It is important to remember that there are various advantages and disadvantages associated with every assessment option. Identifying the most appropriate assessment tool for a given situation requires careful consideration of the costs and benefits associated with each option in relation to the needs and priorities of the user.

Sources of Guidance

The resources described below provide additional information and guidance. Adherence to the guidelines described will ensure consistency with current legal and professional standards.

Testing and Assessment: An Employer's Guide to Good Practices. The U.S. Department of Labor (DOL) developed this document to assist managers and human resource professionals in making assessment-related decisions. This resource is available in hard copy from the Government Printing Office (GPO Order Desk 202/512-1800, Stock Number 029-014-00259-3) or by searching the Department of Labor web site http://www.doleta.gov/ .

Delegated Examining Operations Handbook: A Guide for Federal Agency Examining Offices. OPM's Delegated Examining Operations Handbook is a comprehensive document by which OPM delegates competitive examining authority to agencies under the provisions of 5 U.S.C. 1104, as amended by Public Law 104-52 of 1995. Chapter 2, in particular, provides further guidance for selecting assessment instruments. This Handbook is located at www.opm.gov/deu.

Uniform Guidelines on Employee Selection. The Guidelines provide a set of principles for determining proper test use and selection procedures, covering topics such as test fairness and adverse impact. You can find the Guidelines at www.uniformguidelines.com.

Standards for Educational and Psychological Testing. The Standards provide information on test development, use, and evaluation in employment and other assessment settings. More information can be found by searching the American Psychological Association's website at www.apa.org/ .

Principles for the Validation and Use of Personnel Selection Procedures. The Principles discuss procedures for conducting validation research and personnel selection, specifying principles of good practice in the choice, development, evaluation, and use of personnel selection procedures. This resource is available through www.siop.org.

Getting Expert Help

The issues to consider when selecting or developing an assessment strategy or specific assessment tool are complex. Measurement specialists and trained human resource specialists can assist in selecting or developing valid, fair, and effective assessment strategies that meet specific hiring needs. There are many vendors of assessments. It is the agency's responsibility to ensure that assessments meet all regulatory and legal criteria. Agencies should ensure that they receive appropriate documentation to support the selection process. OPM psychologists can be contracted for assistance and assessment development on a reimbursable basis.