Section VI: Annotated References (Part III)

Work Samples and Simulations References:

Campion, J. E. (1972). Work sampling for personnel selection. Journal of Applied Psychology, 56(1), 40-44.

The author developed a work sample checklist from supervisors' ratings of important tasks and behaviors expected of maintenance mechanics. The test was given to 34 employees by an outside consulting firm, and checklist evaluations were compared with (a) supervisors' evaluation of each employee on three factors; and (b) employees' responses on the test of mechanical comprehension, the Wonderlic Personnel Test, and the short employment tests. Results indicated performance on the work sample was significantly related to supervisory evaluations of job success, but none of the validity coefficients for the paper-and-pencil tests were significant.

Gilliland, S. W. (1995). Fairness from the applicants' perspective: Reactions to employee selection procedures. International Journal of Selection and Assessment, 3(1), 11-19.

Applicants' reactions to selection procedures were examined in terms of the satisfaction and/or violation of ten procedural justice rules. Critical incidents (n = 237) of fair and unfair treatment during selection were collected from 31 individuals who had recently experienced job search and hiring processes. Incidents were categorized into ten procedural justice rules and the distribution of these incidents was examined for different hiring outcomes and different selection procedures. Dominant procedural concerns reflected selection procedure job relatedness and interpersonal treatment applicants received. Accepted applicants were primarily concerned about consistency of treatment, while rejected applicants were more concerned with timely feedback and blatant bias. Ease of faking was the primary procedural concern of applicants taking honesty and personality tests, while job relatedness was the primary concern with ability and work sample tests. Research issues were discussed and a number of practical suggestions were offered in terms of minimizing applicants' negative reactions to the selection process.

Lance, C. E., Johnson, C. D., Douthitt, S. S., Bennett, W., & Harville, D. L. (2000). Good news: Work sample administrators' global performance judgments are (about) as valid as we've suspected. Human Performance, 13(3), 253-277.

Data obtained on over 1,500 first-term U.S. Air Force enlisted personnel indicated work sample administrators' global ratings of work sample performance substantially reflect actual ratee behavior in the work sample, and not potentially biasing factors (e.g., race, gender, amount of recent experience), supporting the "folk wisdom" these global performance judgments are, in fact, valid and unbiased measures of performance.

Robertson, I. T. & Kandola, R. S. (1982). Work sample tests: Validity, adverse impact and applicant reaction. Journal of Occupational Psychology, 55(3), 171-183.

Work sample tests are assigned to one of four categories: (1) psychomotor; (2) individual, situational decision making; (3) job-related information; and (4) group discussion/decision making. Validity data drawn from over 60 studies are presented and show psychomotor work sample tests and group discussions predict job performance relatively well when compared with more conventional forms of psychological testing, such as intelligence or personality tests. Data showing other validity relationships are presented, and the importance of point-to-point correspondence between predictors and criteria is discussed. Research on the adverse impact of work sample tests and applicant reaction to such tests is reviewed and suggests the tests may help to reduce adverse impact and produce positive reactions from candidates.

Schmidt, F. L. & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124(2), 262-274.

This article summarizes the practical and theoretical implications of 85 years of research in personnel selection. On the basis of meta-analytic findings, this article presents the validity of 19 selection procedures for predicting job performance and training performance and the validity of paired combinations of general mental ability (GMA) with the 18 other selection procedures. Overall, the three combinations with the highest multivariate validity and utility for job performance were GMA plus a work sample test (mean validity of .63), GMA plus an integrity test (mean validity of .65), and GMA plus a structured interview (mean validity of .63). A further advantage of the latter two combinations is they can be used for both entry level selection and selection of experienced employees. The practical utility implications of these summary findings are substantial. The implications of these research findings for the development of theories of job performance are discussed.

Background Evaluation/Investigation References:

Hilliard, P. A. (2001). Comparison of the predictive validity of a written test, an integrity test, a conscientiousness questionnaire, a structured behavioral interview and a personality inventory in the assessment of job applicants' background investigations, and subsequent task or contextual performance. Dissertation Abstracts International: Section B: The Sciences & Engineering, 62(6-B), 2981.

This study was designed to compare the validity of several personnel selection instruments in the prediction of the results of applicants' background investigations, and incumbents' subsequent contextual and task work performance. The selection instruments used were a written test, an integrity test, a conscientiousness questionnaire, a structured behavioral interview, and a personality inventory. Out of a total of 168 applicants who were interviewed, 23 were subsequently hired, and job performance evaluations were collected for 18 employees. Although there were some statistically significant findings (e.g., interviews and conscientiousness predicted background investigation results and task performance), the primary hypotheses were not supported. An additional findings was many potential applicants screened themselves out of the process early in the proceedings once they realized an extensive background investigation would be conducted.

McDaniel, M. A. (1989). Biographical constructs for predicting employee suitability. Journal of Applied Psychology, 74(6), 964-970.

The use of background investigation data in personnel screening is reviewed. Background investigations are routinely conducted on persons seeking positions of trust in law enforcement, the nuclear power industry, and military and civilian occupations requiring government-issued security clearances. The application of background investigation information in personnel screening differs in many important ways from biodata applications developed by psychologists. In this article, these differences are reviewed, and the construct and criterion-related validity of a survey-based measure are examined. This measure taps content areas typically explored in background investigations. Seven background factors were identified. The background scales showed acceptable reliability, informative interscale relationships, and useful levels of criterion-related validity.

McFadden, K. L. (1997). Policy improvements for prevention of alcohol misuse by airline pilots. Human Factors, 39(1), 1-8.

Analyzes two strategies for reducing pilot-error aviation accidents: conducting background checks on pilots for driving-while-intoxicated (DWI) convictions, and random preflight alcohol testing of airline pilots. The results of this study are based on analysis of data obtained from the Federal Aviation Administration on the flying performance of 70,164 airline pilots. DWI convictions were found to be associated with a significantly greater risk of a pilot-error accident. In contrast, no evidence was found to validate the assumption a random alcohol testing program could have prevented accidents. The results provide support for improving the existing DWI background check program and for reducing the sampling rate of random alcohol testing for airline pilots. This twofold strategy could result in greater improvements in aviation safety and reduced overall costs.

Job-Fit Measures References:

Arthur, W., Jr., Bell, S. T., Villado, A. J., & Doverspike, D. (2006). The use of person-organization fit in employment decision making: An assessment of its criterion-related validity. Journal of Applied Psychology, 91, 786-801.

This article cautions against the use of P-O measures as selection tools. Using meta-analytic analyses, the criterion-related validity of P-O fit as a predictor of job performance was only .15 with a small effect size. Another consideration is if using P-O as a criterion-related selection tool, it needs to abide by the same professional standards and practices (e.g., psychometric and legal) as other selection tests as outlined by the Civil Rights Act (CRA; 1964, 1991) and Equal Employment Opportunity Commission (EEOC; 1978). At this time, the authors suggest it is best to limit P-O measures for after the candidate has been hired, such as for placement, and not selection.

Cable, D. M., & Judge, T. A. (1997). Interviewers' perceptions of person-organization fit and organizational selection decisions. Journal of Applied Psychology, 82, 546-561.

A model of person-organization fit and organizational hiring decisions is developed and tested, using data from 38 interviewers making hiring decisions about 93 applicants. Results suggest interviewers can assess applicant-organization values congruence with significant levels of accuracy and interviewers compare their perceptions of applicants' values with their organizations' values to assess person-organization fit. Results also suggested interviewers' subjective person-organization fit assessments have large effects on their hiring recommendations relative to competing applicant characteristics, and interviewers' hiring recommendations directly affect organizations' hiring decisions (e.g., job offers).

Dineen, B. R., Ash, S. R., & Raymond, N. A. (2002). A web of applicant attraction: Person-organization fit in the context of web-based recruitment. Journal of Applied Psychology, 87(4), 723-734.

Applicant attraction was examined in the context of Web-based recruitment. Specifically, the provision of feedback to individuals regarding their potential Person-Organization (P-O) fit with an organization related to attraction was studied. Objective and subjective P-O fit, agreement with fit feedback, and self esteem also were examined in relation to attraction. Results of an experiment that manipulated fit feedback level after a self-assessment provided by a fictitious company Web site found both feedback level and objective P-O fit were related to attraction. In addition, attraction was related to the interaction of objective fit, feedback, and agreement and objective fit, feedback, and self esteem. Implications and future Web-based recruitment research directions are discussed.

Judge, T. A., & Cable, D. M. (1997). Applicant personality, organizational culture, and organizational attraction. Personnel Psychology, 50, 359-394.

Examined the dispositional basis of job seekers' organizational culture preferences and how these preferences interact with recruiting organizations' cultures in their relation to organization attraction. Data were collected from 182 business, engineering, and industrial relations students who were seeking positions at the time of the study. Results obtained from multiple sources suggested the Big Five personality traits generally were related to hypothesized dimensions of culture preferences. Results also suggested both objective person-organization fit and subjective fit (applicant's direct perception of fit) were related to organization attraction.

Kristof, A. L. (1996). Person-organization fit: An integrative review of its conceptualizations, measurement, and implications. Personnel Psychology, 49, 1-49.
Presents a comprehensive definition and conceptual model of person-organization fit that incorporates supplementary as well as complementary perspectives on fit. To increase the precision of the construct's definition, it is also distinguished from other forms of environmental compatibility, such as person-group and person-vocation fit. Measurement as it relates to supplementary and complementary fit is discussed and recommendations are offered regarding the necessity of its use. A distinction is made between the direct measurement of perceived fit and the indirect measurement of actual person-organization fit, and the debate regarding differences scores is reviewed.

Martinez, M. N. (2000). Get job seekers to come to you. HR Magazine, 45, 45-52.

This article discusses ways organizations can use their website to attract top talent. Interactive self-assessments of job fit are promoted as one effective practice. The author states applicants are desperate for tools that help them determine their "fit" in an organization and the key is to give them results.

Physical Ability Tests References:

Arvey, R. D., Maxwell, S. E., & Salas, E. (1992). Development of physical ability tests for police officers: A construct validation approach. Journal of Applied Psychology, 77,996-1009.

A construct validation approach was followed and results indicated eight physical ability test events were significantly related to two important constructs underlying the job performance of police officers: strength and endurance. In addition, the data were examined for potential gender differences and bias. Considerable differences were shown between men and women on both test and performance variables.

Arvey, R. D., Nutting, S. M., & Landon, T. E. (1992). Validation strategies for physical ability testing in police and fire settings. Public Personnel Management, 21, 301-312.

Discusses two issues within the context of selection and staffing for police and fire fighting positions: (1) the increasing litigation challenging the validity of physical ability tests in screening applicants for these positions, and (2) the lack of published literature concerning the use and validation of physical ability tests in these settings. The authors discuss issues associated with court challenges of traditional content validity procedures and suggest there may be an over-reliance on this particular validation strategy. They suggest construct validation procedures may be an alternative procedure to provide evidence concerning the validity of physical ability tests within these contexts. Construct validation is described and illustrated via hypothetical examples.

Campbell, W. J., & Fox, H. R. (2002). Testing individuals with disabilities in the employment context: An overview of issues and practices. In R. B. Ekstrom & D. K. Smith (Eds.) Assessing Individuals with Disabilities in Educational, Employment, and Counseling Settings (1^st ed, p. 198). Washington, DC: American Psychological Association.

This chapter discusses the possible issues that can occur when testing individuals with disabilities. It noted while the Americans with Disabilities Act (ADA) permits the use of physical ability tests, the tests cannot include (or involve) physiological or biological measures. They also warned of the possibility of applicants hurting themselves while performing a physical ability test. Depending on the extensiveness of the physical ability test, employers may request applicants to provide a certificate from their physicians indicating their ability to safely perform a physical ability test.

Campion, M. A. (1983). Personnel selection for physically demanding jobs: Review and recommendations. Personnel Psychology, 36, 527-550.

Central issue of this paper is improvement in personnel selection systems for physically demanding jobs is needed due to equal employment opportunity (EEO) considerations, concern for worker physical well-being, and the lack of alternative procedures. After addressing the special EEO sensitivities of physical abilities selection, the literature is reviewed from a variety of disciplines on (1) the physiological background underlying the selection strategies, (2) the assessment of human physical abilities, (3) the measurement of physical requirements of jobs, and (4) the physical abilities personnel selection studies reported in the literature.

Hogan, J. (1991). The structure of physical performance in occupational tasks. Journal of Applied Psychology, 76, 495-507.

Two lines of research concerning the dimensionality of physical performance in occupational tasks are described. In the first, the physical requirements of tasks are analyzed as reflected in job analyses. In the second, the structure of physical abilities tests used to predict performance in physically demanding jobs is evaluated. Results of the data analysis suggest the structure of physical abilities has three major components: strength, endurance, and movement quality. This structure appears to be independent of job type or level of incumbents' performance.

Realistic Job Previews References:

McEvoy, G. M., & Cascio, W. F. (1985). Strategies for reducing employee turnover: A meta-analysis. Journal of Applied Psychology, 70(2), 342-353.

In looking at the relative effectiveness of the Realistic Job Preview (RFP) as a turnover-reduction strategy, this meta-analysis found RJPs are about half as effective as job enrichment strategies in reducing turnover. However, it should be noted the job enrichment measures included in this meta-analysis were used with existing workers, whereas the RJPs were used during the application process. Given the low correlation between RJPs and reduction in turnover, the authors suggest managers use other turnover reduction strategies prior to hiring.

Pitt, L. F., & Ramaseshan, B. (1995). Realistic job information and salesforce turnover: An investigative study. Journal of Managerial Psychology, 10(5), 29-36.

Salespeople were asked to provide ratings on the volume, personal relevance, depth, and accuracy of information they received when they applied for their current jobs. They were also asked about their intention to quit their current job. The results indicated those applicants (employees) who felt they received more accurate and realistic job information were significantly less likely to indicate (or consider) quitting their current jobs.

Saks, A. M, Wiesner, W. H., & Summers, R. (1996). Effects of job previews and compensation policy on applicant attraction and job choice. Journal of Vocational Behavior, 49, 68-85.

Students were asked to rate the attractiveness of jobs after reviewing a Realistic Job Preview (RJP) and a Traditional Job Preview (TJP). Jobs with RJPs were rated as more attractive only when those positions had a higher compensation than the TJP position. Results of this study indicate many other factors (with compensation being studied here) may contribute to an applicant's decision to accept (or reject) a job when coupled with an RJP.

Wanous, J. P. (1989). Installing a realistic job preview: Ten tough choices. Personnel Psychology, 42(1), 117-133.

The author reviews the ten different choices one must make when constructing a Realistic Job Preview (RJP). Examples of the choices include whether the content should be descriptive or judgmental, contain high or medium negativity, or whether the results should be shared or kept proprietary. For each choice, the author includes the pros and cons of each. The author concludes the article with personal recommendations for each of the ten choices.