Language models identify psychiatric symptoms nearly as accurately as early-career clinicians

A recent study conducted by scientists at the Central Institute of Mental Health (CIMH) has revealed that large language models can identify psychiatric symptoms in transcripts of psychiatric interviews with nearly the same accuracy as early-career clinicians. The research, published in the journal npj Digital Medicine, examined the performance of 10 language models compared to 108 practicing clinicians from three psychiatric clinics.

The study focused on the complex task of psychopathological assessment, which involves evaluating patients' mental states through interviews.

The language models were tasked with assessing 100 features of the AMDP system, a widely used method in German-speaking psychiatry for describing psychiatric symptoms. In contrast, the clinicians were allowed to review complete video and audio recordings of the simulated interviews. The results showed that the two highest-performing language models, GPT-5.1 and Gemini-3-Pro-Preview, achieved an average accuracy of 72% in identifying individual psychopathological characteristics.

This accuracy was comparable to the 68% achieved by the clinical comparison group, which consisted primarily of young clinicians.

Interestingly, the study found that clinicians were more likely to infer the presence or absence of a characteristic based on incomplete information, while language models tended to classify such characteristics as "unassessable." This discrepancy was particularly noticeable for symptoms that rely on nonverbal or visual cues. The researchers suggest that human clinical experience and machine-based evaluation could complement each other in psychiatric assessment.

The study's lead author, Dr. Esra Lenz, emphasized that the findings provide initial evidence that AI could support the process of psychopathological assessment in everyday psychiatric practice. However, she stressed that the AI systems do not replace clinical experience or direct assessment by specialists. The researchers also investigated whether a language model could serve as an additional decision-making aid in cases of conflicting clinical assessments, finding that its evaluation improved accuracy compared to a random choice between two clinical assessments.

The study's limitations include the use of simulated interviews involving only three different psychiatric disorders, with no real patients involved. Therefore, the results cannot be generalized to real-world psychiatric interviews or specific disorders. Future studies must address these limitations by examining the findings in a larger number of diverse interviews, involving real patients, and conducted under real-world clinical conditions.

The study's authors emphasize the exploratory nature of the research and stress the need for careful examination of AI's strengths and limitations before it can be used in psychiatric diagnosis.