Urgent.News

What's breaking now, across thousands of outlets.

AI

"You’re right"—LLMs will get climate info wrong if you ask

Some models fare better, but all fail in similar ways.

"You’re right"—LLMs will get climate info wrong if you ask

Studies have examined how individuals engage with climate science information presented through various sources, such as search engines, social media, and traditional media outlets. With the emergence of large language models (LLMs) as part of the public information landscape, it is essential to evaluate their performance in handling climate change topics.

However, the ephemeral nature of LLM outputs, limited to individual chat sessions, poses a challenge for researchers who lack a direct method to analyze the information users have encountered.

To address this issue, a team of researchers from University College London and UNESCO conducted a comprehensive study to test LLMs with a variety of prompts and assess their responses. The researchers employed both straightforward factual inquiries and more intricate scenarios, including instances where users believed their responses were accurate and subsequently challenged the LLM's answers.

The study utilized 33 different language models, encompassing various iterations of major frontier and open-source models. The leading frontier models examined in the research included Claude Fable 5, Claude Opus 5, Gemini 3.1 Pro Preview, Gemini 3.6 Flash, and GPT-5.6 Sol. A complete list of the models employed in the study can be found in the report.

It is worth noting that prior research on Large Language Models (LLMs) has faced criticism for focusing on specific models rather than assessing a broader range of emerging models after the experiments were conducted. In this particular study, researchers aimed to conduct a thorough examination of 33 diverse models to provide a more comprehensive understanding of LLMs' performance in addressing climate change-related inquiries.

Written by urgent.news from Ars Technica Science's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

This story

This is one outlet's version. Read the fullest account.

Every outlet that reported this, and when →

Read the original at arstechnica.com →

More in AI

More from Thursday 19 November →