Journal of Applied Psychology 2024

Are automated video interviews smart enough? Behavioral modes, reliability, validity, and bias of machine learning cognitive ability assessments.

Louis Hickman¹; Louis Tay²; Sang Eun Woo²

¹ Virginia Tech · ² Department of Psychological Sciences, Purdue University

Abstract

(measurement and predictive). Overall, the general mental ability, verbal ability, and intellect AVI models captured similar behavioral manifestations of cognitive ability. Validity evidence results were mixed: For example, AVIs trained on observer-rated intellect exhibited superior convergent and criterion relationships (compared to the observer ratings they were trained to model) but had limited discriminant validity evidence. Our findings illustrate the importance of examining psychometric properties beyond convergence with the test that ML algorithms are trained to model. We provide recommendations for enhancing discriminant validity evidence in future AVIs. (PsycInfo Database Record

DOI

10.1037/apl0001236

Sources

openalex

Cite

APA 7 MLA 9 Chicago (author–date) Harvard Elsevier Harvard (JFE, JBF…) The Accounting Review Academy of Management American Marketing Association INFORMS (Management Science) Copy citation Copied ✓