Large Language Models Predict Human Social Behavior via Interpretable Mechanisms

The advancement of large language models (LLMs) presents new avenues for forecasting human behavior in various situations. Despite this potential, previous research has primarily focused on replicating behavior, with comparatively little exploration of the transparent or interpretable models that underpin human decision-making. To address this gap, researchers have developed MindEvolve, an autonomous system aimed at predicting behavior in social interactions by creating models that are both interpretable and symbolic.

MindEvolve underwent rigorous testing, evaluating the modeling abilities of numerous LLMs within a series of socioeconomic games that span four fundamental aspects of social cognition. These areas include economic motivations, social motivations, social reasoning (often referred to as theory of mind), and recursive planning. The symbolic models generated by the LLMs were then subjected to scrutiny by human experts, who assessed their interpretability and theoretical consistency.

The findings reveal that while many LLMs are adept at emulating economic and social preferences, especially in straightforward scenarios, they often struggle to extend their models to more intricate psychological processes. Nevertheless, a select group of high-performance models exhibit remarkable aptitude in capturing more advanced cognitive functions, such as theory of mind and recursive planning.

These results underscore the promising prospect that LLMs are not just capable of mimicking human behavior but also of developing interpretable, mechanistic explanations of human social decision-making.

Overall, these discoveries offer a blueprint for the development of LLM-based cognitive modeling, aiming to achieve a level of theory construction comparable to that of human experts.