The Validity and Utility of Selection Methods in Personnel
Copyright 1998 by the American Psychological Association, Inc. 0033-2909/98/S3.00
The Validity and Utility of Selection Methods in Personnel Psychology: Practical and Theoretical Implications of 85 Years of Research Findings
Frank L. Schmidt University of Iowa
John E. Hunter Michigan State University
This article summarizes the practical and theoretical implications of 85 years of research in personnel selection. On the basis of meta-analytic findings, this article presents the validity of 19 selection procedures for predicting job performance and training performance and the validity of paired combinations of general mental ability (GMA) and Ihe 18 other selection procedures. Overall, the 3 combinations with the highest multivariate validity and utility for job performance were GMA plus a work sample test (mean validity of .63), GMA plus an integrity test (mean validity of .65), and GMA plus a structured interview (mean validity of .63). A further advantage of the latter 2 combinations is that they can be used for both entry level selection and selection of experienced employees. The practical utility implications of these summary findings are substantial. The implications of these research findings for the development of theories of job performance are discussed.
From the point of view of practical value, the most important property of a personnel assessment method is predictive validity: the ability to predict future job performance, job-related learning (such as amount of learning in training and development programs), and other criteria. The predictive validity coefficient is directly proportional to the practical economic value (utility) of the assessment method (Brogden, 1949; Schmidt, Hunter, McKenzie, & Muldrow, 1979). Use of hiring methods with increased predictive validity leads to substantial increases in employee performance as measured in percentage increases in output, increased
References: Baker, T. G. (1996). Practice network. The Industrial-Organizational Psychologist, 34, 44-53. Bar-Hillel, M, & Ben-Shakhar, G. (1986). The a priori case against graphology: Methodological and conceptual issues. In B. Nevo (Ed.), Scientific aspects of graphology (pp. 263-279). Springfield, IL: Charles C Thomas. Ben-Shakhar, G. (1989). Nonconventional methods in personnel selection. In P. Herriot (Ed.), Handbook of assessment in organizations: Methods and practice for recruitment and appraisal (pp. 469-485). Chichester, England: Wiley. Ben-Shakhar, G., Bar-Hillel, M., Bilu, Y, Ben-Abba, E., & Hug, A. (1986). Can graphology predict occupational success? Two empirical studies and some methodological ruminations. Journal of Applied Psychology, 71, 645-653. Ben-Shakhar, G., Bar-Hillel, M., & Rug, A. (1986). A validation study of graphological evaluations in personnel selection. In B. Nevo (Ed.), Scientific aspects of graphology (pp. 175-191). Springfield, IL: Charles C Thomas. Borman, W. C., White, L. A., Pulakos, E. D., & Oppler, S. H. (1991). Models evaluating the effects of ratee ability, knowledge, proficiency, temperament, awards, and problem behavior on supervisory ratings. Journal of Applied Psychology, 76, 863-872. Boudreau, J. W. (1983a). Economic considerations in estimating the utility of human resource productivity improvement programs. Personnel Psychology, 36, 551-576. Boudreau, J. W. (1983b). Effects of employee flows or utility analysis of human resources productivity improvement programs. Journal of Applied Psychology, 68, 396-407. Boudreau, J. W. (1984). Decision theory contributions to human resource management research and practice. Industrial Relations, 23, 198-217. Brogden, H. E. (1949). When testing pays off. Personnel Psychology, 2, 171-183. Carlson, K. D., Scullen, S. E., Schmidt, F. L., Rothstein, H. R., & Erwin, F. W. (1998). Generalizable biographical data: Is multi-organizational development and keying necessary? Manuscript in preparation. Carroll, J. B. (1993). Human cognitive abilities: A survey of factor analytic studies. New Tfork: Cambridge University Press. Cascio, W. F, & Silbey, V. (1979). Utility of the assessment center as a selection device. Journal of Applied Psychology, 64, 107-118. Collins, J. (1998). Prediction of overall assessment center evaluations from ability, personality, and motivation measures: A meta-analysis. Unpublished manuscript, Texas A & M University, College Station, TX. Cronshaw, S. F, & Alexander, R. A. (1985). One answer to the demand for accountability: Selection utility as an investment decision. Organizational Behavior and Human Performance, 35, 102-118. Dye, D. A., Reck, M., & McDaniel, M. A. (1993). The validity of job knowledge measures. International Journal of Selection and Assessment, I, 153-157. Gaugler, B. B., Rosenthal, D. B., Thornton, G. C., & Benson, C. (1987). Meta-analysis of assessment center validity. Journal of Applied Psychology, 72, 493-511. Holland, J. (1986). New directions for interest testing. In B. S. Plake & J. C. Witt (Eds.), The future of testing (pp. 245-267). Hillsdale, NJ: Erlbaum. Hollander, E. P. (1956). The friendship factor in peer nominations. Personnel Psychology, 9, 435-447. Huffcutt, A. I., Roth, P. L., & McDaniel, M. A. (1996). A meta-analytic investigation of cognitive ability in employment interview evaluations: Moderating characteristics and implications for incremental validity. Journal of Applied Psychology, 81, 459-473. Hunter, J. E. (1980). Validity generalization for 12,000 jobs: An application of synthetic validity and validity generalization to the General Aptitude Test Battery (GATE). Washington, DC: U.S. Department of Labor, Employment Service. Hunter, J. E. (1986). Cognitive ability, cognitive aptitudes, job knowledge, and job performance. Journal of Vocational Behavior, 29, 340362. Hunter, I. E., & Hunter, R. F. (1984). Validity and utility of alternative predictors of job performance. Psychological Bulletin, 96, 72-98. Hunter, J. E., & Schmidt, F. L. (1982a). Fitting people to jobs: Implications of personnel selection for national productivity. In E. A. Fleishman & M. D. Dunnette (Eds.), Human performance and productivity. Volume I: Human capability assessment (pp. 233-284). Hillsdale, NJ: Erlbaum. Hunter, J. E., & Schmidt, F. L. (1982b). Quantifying the effects of psychological interventions on employee job performance and work force productivity. American Psychologist, 38, 473-478. Hunter, J. E., & Schmidt, F. L. (1990). Methods of meta-analysis: Correcting error and bias in research findings. Beverly Hills, CA: Sage. Hunter, J. E., & Schmidt, F. L. (1996). Intelligence and job performance: Economic and social implications. Psychology, Public Policy, and Law, 2, 447-472. Hunter, J. E., Schmidt, F. L., & Coggin, T. D. (1988). Problems and pitfalls in using capital budgeting and financial accounting techniques in assessing the utility of personnel programs. Journal of Applied Psychology, 73, 522-528. Hunter, S. E., Schmidt, F. L., & Jackson, G. B. (1982). Meta-analysis: Cumulating research findings across studies. Beverly Hills, CA: Sage. Hunter, J. E., Schmidt, F. L., & Judiesch, M. K. (1990). Individual differences in output variability as a function of job complexity. Journal of Applied Psychology, 75, 28-42. 274 SCHMIDT AND HUNTER Schmidt, F. L., & Hunter, J. E. (1992). Development of causal models of processes determining job performance. Current Directions in Psychological Science, 1, 89-92. Schmidt, F.L., Hunter, I.E., McKenzie, R. C., & Muldrow, T.W. (1979). The impact of valid selection procedures on work-force productivity. Journal of Applied Psychology, 64, 609-626. Schmidt, F. L., Hunter, J. E., & Outerbridge, A. N. (1986). The impact of job experience and ability on job knowledge, work sample performance, and supervisory ratings of job performance. Journal of Applied Psychology, 71, 432-439. Schmidt, F. L., Hunter, J. E., Outerbridge, A. N., & Goff, S. (1988). The joint relation of experience and ability with job performance: A test of three hypotheses. Journal of Applied Psychology, 73, 46-57. Schmidt, F. L., Hunter, J. E., Outerbridge, A. M., & Tratrner, M. H. (1986). The economic impact of job selection methods on the size, productivity, and payroll costs of the federal work-force: An empirical demonstration. Personnel Psychology, 39, 1-29. Schmidt, F. L., Hunter, J. E., & Pearlman, K. (1980). Task difference and validity of aptitude tests in selection: A red herring. Journal of Applied Psychology, 66, 166-185. Schmidt, F. L., Hunter, J. E., & Pearlman, K. (1982). Assessing the economic impact*of personnel programs on workforce productivity. Personnel Psychology, 35, 333-347. Schmidt, F. L., Hunter, J. E., Pearlman, K., & Shane, G. S. (1979). Further tests of the Schmidt-Hunter Bayesian Validity Generalization Model. Personnel Psychology, 32, 257-281. Schmidt, F. L., Law, K., Hunter, J. E., Rothstein, H. R., Pearlman, K., & McDaniel, M. (1993). Refinements in validity generalization methods: Implications for the situational specificity hypothesis. Journal of Applied Psychology, 78, 3-13. Schmidt, F. L., Mack, M. J., & Hunter, J. E. (1984). Selection utility in the occupation of U.S. Park Ranger for three modes of test use. Journal of Applied Psychology, 69, 490-497. Schmidt, F. L., Ones, D. S., & Hunter, J. E. (1992). Personnel selection. Annual Review of Psychology, 43, 627-670. Schmidt, F. L., Ones, D. S., & Viswesvaran, C. (1994, June 30-July 3). The personality characteristic of integrity predicts job training success. Presented at the 6th Annual Convention of the American Psychological Society, Washington, DC. Schmidt, F. L., & Rothstein, H. R. (1994). Application of validity generalization methods of meta-analysis to biographical data scores in employment selection. In G. S. Stokes, M. D. Mumford, & W. A. Owens (Eds.), The biodata handbook: Theory, research, and applications (pp. 237-260). Palo Alto, CA: Consulting Psychologists Press. Steiner, D. D. (1997). International forum. The Industrial-Organizational Psychologist, 34, 51-53. Steiner, D. D., & Gilliland, S. W. (1996). Fairness reactions to personnel selection techniques in France and the United States. Journal of Applied Psychology, 81, 134-141. Viswesvaran, C., Ones, D. S., & Schmidt, F. L. (1996). Comparative analysis of the reliability of job performance ratings. Journal of Applied Psychology, 81, 557-560. Waters, L. K., & Waters, C. W. (1970). Peer nominations as predictors of short-term role performance. Journal of Applied Psychology, 54, 42-44. Wigdor, A. K, & Garner, W. R. (Eds.). (1982). Ability testing: Uses, consequences, and controversies (Report of the National Research Council Committee on Ability Testing). Washington, DC: National Academy of Sciences Press. Jansen, A. (1973). Validation of graphological judgments: An experimental study. The Hague, the Netherlands: Monton. Jensen, A. R. (1998). The g factor: The science of mental ability. Westport, CT: Praeger. Levy, L. (1979). Handwriting and hiring. Dun 's Review, 113, 72-79. McDaniel, M. A., Schmidt, F.L., & Hunter, J. E. (1988a). A metaanalysis of the validity of methods for rating training and experience in personnel selection. Personnel Psychology, 41, 283-314. McDaniel, M. A., Schmidt, F. L., & Hunter, J. E. (1988b). Job experience correlates of job performance. Journal of Applied Psychology, 73, 327-330. McDaniel, M. A., Whetzel, D. L., Schmidt, F. L., & Mauer, S. D. (1994). The validity of employment interviews: A comprehensive review and meta-analysis. Journal of Applied Psychology, 79, 599-616. Mount, M. K., & Barrick, M. R. (1995). The Big Five personality dimensions: Implications for research and practice in human resources management. In G. R. Ferris (Ed.), Research in personnel and human resources management (Vol. 13, pp. 153-200). JAI Press. Neter, E., & Ben-Shakhar, O. (1989). The predictive validity of graphological inferences: A meta-analytic approach. Personality and Individual Differences, 10, 737-745. Ones, D. S. (1993). The construct validity of integrity tests. Unpublished doctoral dissertation, University of Iowa, Iowa City. Ones, D. S., Viswesvaran, C., & Schmidt, F. L. (1993). Comprehensive meta-analysis of integrity test validities: Findings and implications for personnel selection and theories of job performance. Journal of Applied Psychology Monograph, 78, 679-703. Pearlman, K., Schmidt, F. L., & Hunter, J. E. (1980). Validity generalization results for tests used to predict job proficiency and training criteria in clerical occupations. Journal of Applied Psychology, 65, 373-407. Rafaeli, A., & Klimoski, R. J. (1983). Predicting sales success through handwriting analysis: An evaluation of the effects of training and handwriting sample context. Journal of Applied Psychology, 68, 212- 217. Ree, M. J., & Earles, J. A. (1992). Intelligence is the best predictor of job performance. Current Directions in Psychological Science, 1,8689. Rothstein, H. R., Schmidt, F. L., Erwin, F. W., Owens, W. A., & Sparks, C. P. (1990). Biographical data in employment selection: Can validities be made generalizable? Journal of Applied Psychology, 75, 175-184. Schmidt, F. L. (1988). The problem of group differences in ability scores in employment selection. Journal of Vocational Behavior, 33, 272-292. Schmidt, F. L. (1992). What do data really mean? Research findings, meta analysis, and cumulative knowledge in psychology. American Psychologist, 47, 1173-1181. Schmidt, F. L. (1993). Personnel psychology at the cutting edge. In N. Schmitt & W. Borman (Eds.), Personnel selection (pp. 497-515). San Francisco: Jossey Bass. Schmidt, F. L., Caplan, J. R., Bemis, S. E., Decuir, R., Dinn, L., & Antone, L. (1979). Development and evaluation of behavioral consistency method of unassembled examining (Tech. Rep. No. 79-21). U.S. Civil Service Commission, Personnel Research and Development Center. Schmidt, F. L., & Hunter, J. E. (1977). Development of a general solution to the problem of validity generalization. Journal of Applied Psychology, 62, 529-540. Schmidt, F. L., & Hunter, J. E. (1981). Employment testing: Old theories and new research findings. American Psychologist, 36, 1128-1137. Schmidt, F. L., & Hunter, J. E. (1983). Individual differences in productivity: An empirical test of estimates derived from studies of selection procedure utility. Journal of Applied Psychology, 68, 407-415. Received April 8, 1997 Revision received February 3, 1998 Accepted April 2, 1998 •