By Tenzin Tamang.
AI sees climate change as "hotter" than experts do. Our research explores this overestimation bias in LLMs.
Ever asked an AI chatbot about climate change? Many of us turn to these powerful tools for quick information, but our new research reveals a surprising quirk: AI models tend to "see hotter", frequently overestimating the impacts of climate change compared to the consensus of expert scientists. This tendency to exaggerate gets even stronger when we ask the AI to respond as if it were a climate scientist.
What we did
We took statements about climate change impacts directly from the highly respected Intergovernmental Panel on Climate Change (IPCC) 2023 Synthesis Report. For each statement, the IPCC provides a specific confidence level (e.g., low, medium, or high).
We then presented these same scenarios to popular GPT-family models and asked them to rate their confidence in two conditions: (1) directly, and (2) acting as a climate scientist.
- AI overestimates: AI models generally rated the climate change impacts with higher confidence than the IPCC experts did. They showed an "overestimation bias."
- The "scientist" role makes it stronger: When the AI was asked to adopt the role of a climate scientist, this overestimation became even more pronounced. It seemed to double down on the certainty of severe impacts.
Tenzin Tamang (@TBlonnee) is a doctoral researcher affiliated with the Web Mining Lab, a Computational Social Science (CSS) research lab housed within the Department of Media and Communication at City University of Hong Kong.