Personality tests are a popular and frequently used selection tool. However, personality is often based on self-report measures from job candidates, which can be lengthy and susceptible to faking. To remedy this, researchers (Fan et al., 2023) aimed to assess the validity of a quick and efficient measure of personality. Their novel approach used a person’s conversation with an artificial intelligence (AI) chatbot to extract personality-related information.
PERSONALITY FROM AN AI CHATBOT
The researchers conducted a study with over 1,500 undergraduate students to examine the properties of the AI personality scores. To do this, participants engaged in a two-way conversation with an AI chatbot for 20-30 minutes. They also completed a self-report personality assessment to compare with the AI personality scores; GPA and peer-rated college adjustment were used to measure outcomes of personality. Finally, the researchers used participants’ chat responses to train AI to predict the self-reported personality scores.
The researchers found that the AI personality scores were able to satisfy many of the standards that help researchers evaluate the quality of a psychological test. For example, the AI scores were consistent and generally similar to the self-report personality measures. Further, in some analyses, AI scores had predictive power that extended beyond self-reported personality scores – for example, when measuring the ability of extraversion to predict GPA and college adjustment.
PRACTICAL APPLICATIONS
Their study provides promising support for the future of machine learning techniques in personality assessment. As these techniques become more common in employee selection, organizations should be aware that this might help them save time and still accurately measure personality. However, the authors caution that more research is needed before fully implementing this technique. For example, we need to learn more about whether it is fair to protected minority groups, is susceptible to faking, as well as its legal and ethical implications.
Fan, J., Sun, T., Liu, J., Zhao, T., Zhang, B., Chen, Z., Glorioso, M., & Hack, E. (2023). How well can an AI chatbot infer personality? Examining psychometric properties of machine-inferred personality scores. Journal of Applied Psychology, 108(8), 1277–1299.
Image credit: istockphoto/PhonlamaiPhoto