How AI models successfully detect personality traits from written text

A research team at the University of Barcelona (UB) has shown how artificial intelligence (AI) models can detect personality traits from written texts, and for the first time has managed to analyze in detail how these systems make decisions. These results, published in the journal PLOS One, open up new perspectives for understanding how personality manifests itself in natural language and also how more transparent and reliable automatic detection tools can be built.

The paper is co-authored by three UB experts: David Saeteros and David Gallardo-Pujol, researcher and director, respectively, of the Individual Differences Lab Research Group (IDLab) of the Faculty of Psychology and the Institute of Neurosciences (UBneuro), and Daniel Ortiz Martínez, researcher at the Faculty of Mathematics and Computer Science.

Opening the ‘black box’ of algorithms

The study has analyzed how two advanced AI models, BERT and RoBERTa, process text data to detect personality characteristics following two main psychological frameworks: the Big Five personality trait system (openness to experience, responsibility, extraversion, agreeableness and emotional stability) and the Myers-Briggs Type Indicator (MBTI), an instrument that classifies people along the dimensions of extrovert-introvert, sensory-intuitive and thinking-feeling.

“In psychology, there is a prevalent model of personality and other less validated models, which we use to understand and measure individual differences in behavior, emotions and thinking,” the researchers explain.

The texts analyzed in the study were obtained from two databases fed with questionnaires from both models (Big Five and MBTI), which had previously been classified according to the presence of indicators of the different personality traits and types that comprise them. Subsequently, researchers have used explainable AI techniques to observe the AI models and see which language patterns influence the identification of personality traits in these writings.

“Explainability techniques allow us to ‘open the black box’ of algorithms, which ensures that predictions are based on psychologically relevant signals and not on artifacts in the data,” note the authors.

Specifically, they used a technique called integrated gradients, which allows them to identify exactly which words or phrases contribute to the prediction of a specific personality trait. “This methodology has allowed us to visualize and quantify the importance of various linguistic elements in the model’s predictions,” they say.

For example, they have observed that words such as hate, which would traditionally be associated with negative traits, can appear in contexts that actually reflect kindness (“I hate to see others suffer”). “Without understanding how the model interprets these words in context, we may draw the wrong conclusions,” they stress.

This approach guarantees the scientific validity of the performance of AI models, as it allows “verifying whether the models align with established psychological theories and also provides a solid basis for continuous improvement by ensuring that they are based on linguistic patterns that are genuinely related to the psychological constructs they are intended to measure,” they add.

The limitations of the MBTI model

The study also highlighted the limitations of the MBTI model compared to the Big Five one, which shows a stronger basis for both automated personality analysis and classical psychometric analysis.

“Despite being widely used in computer science and some applied fields of psychology, the MBTI model has serious limitations for automatic personality assessment, as our results indicate that the models tend to rely more on artifacts than on real patterns,” they note.

Applications of automatic personality detection

The use of automatic personality detection techniques with AI models can have a major impact on the field of personality psychology. “With these methods, psychologists will identify linguistic patterns associated with different personality traits that, with traditional methods, might go unnoticed. This can lead to more natural and less intrusive assessment methods, especially valuable for the study of large populations,” the researchers note.

In the clinical field, the authors point out that they can help in “initial assessment and follow-up of patients by focusing attention on changes in language or verbal expression as indicators of important psychological elements for therapy.” They also point out that they can play an important role in other areas: in personnel selection, in educational personalization, in social research—it would facilitate the analysis of large volumes of textual data—or in the development of virtual assistants and conversational agents, as it would help to create more natural and adapted interactions.

“It is important to stress that all such applications should be based on scientifically sound models and incorporate the explainability techniques we have explored, to ensure ethical and transparent use,” they add.

Despite the potential, researchers believe that these models will not replace traditional personality tests in the short term, but will complement them and offer an additional and deeper perspective. “We see an evolution towards a multimodal approach, where traditional assessments are combined with natural language analysis, digital behavior and other data sources to get a more complete picture of personality,” they note.

This integrative approach will, according to the researchers, build on the strengths of each methodology, providing a “richer and more nuanced view of the human personality.” In this sense, AI models can be “especially useful in contexts where traditional data collection is difficult or when large volumes of information need to be analyzed efficiently,” they add.

Validating research in other contexts

The next steps in this study include extending the analysis to other text types, platforms, languages and cultures to confirm whether the patterns identified are consistent across different contexts. The researchers want to explore the application of these techniques to other psychological constructs beyond personality, such as emotional states or attitudes.

Researchers are also working to integrate multimodal data into these analyses—combining text with other forms of expression, such as voice or non-verbal behavior, and using technologies such as automatic audio transcription (Whisper.ai)—as well as their application in real-life contexts. The team wants to “collaborate with clinicians and human resources professionals to evaluate the effectiveness of these tools in real-world settings, ensuring that they have a positive and ethical impact.”