Guest post: How artificial intelligence is fast becoming a key tool for climate science

§ Earth system models

Climate models have seen continuous improvement over recent decades. The most recent developments have seen the incorporation of biogeochemical cycles – the transfer of chemicals between living things and their environment – and how they interact with the climate system. Like their predecessors, these “Earth system models” (ESMs) are used to project future climate according to different scenarios of greenhouse gas emissions.

But while adding in new processes and greater detail have resulted in more sophisticated simulations of the Earth’s climate, it comes at the cost of increasingly large and complex models.

ESMs are built on equations that represent the processes and interactions that drive the Earth’s climate. Some of these processes can be described by fundamental laws – such as the Navier-Stokes equations of fluid motion, which capture the speed, pressure, temperature and density of the gases in the atmosphere and the water in the ocean. However, others – such as physiological processes governing the vegetation that covers vast parts of the land surface – cannot and instead require approximations based on observations. 

These approximations – as well as other limitations that stem from the sheer complexity of the Earth system – introduce uncertainties into the model’s representation of the climate. 

As a result, despite the tremendous success of ESMs, some limitations remain – such as how well models capture the severity and frequency of extreme events, and abrupt changes and “tipping points”.

§ Machine learning for climate research

Machine learning is an overarching term for many different tools, of which artificial neural networks – a set of interconnected algorithms modelled loosely on the human brain – are a widely known and used example.

Through “supervised learning” – which uses datasets to “train” algorithms – these tools can be used to uncover patterns and complex relationships between variables, allowing them to perform specific tasks like classifying or analysing data.

For instance, a neural network can be trained to identify and classify patterns in satellite imagery – such as cloud structures, ocean eddies or crop quality – as well as to perform a weather forecast based on past records, model outputs and physical balance equations.

In contrast to ESMs, machine learning does not require prior knowledge about the governing laws and relations within a problem. The respective relations are derived entirely from the data used during an automated learning process. This flexible and powerful concept can be expanded to almost any level of complexity. 

The availability of observed climate data and model simulations in combination with ready-to-use machine learning tools – such as TensorFlow and Keras – have led to an explosion of machine learning studies in Earth and climate sciences. These have explored how machine learning can be applied to enhance or even replace classical ESM tasks.

Despite wordings like “learning” and “artificial intelligence”, today’s machine learning applications in this field are far from intelligent and lack actual process knowledge. More accurately, they are highly specialised algorithms that are trained to solve very specific problems solely based on the problem-related presented data.

Consequently, machine learning is often considered a black box that makes it hard to gather insights from. Similarly, it is often very difficult to validate machine learning in terms of physical consistency, even if their generated outputs may seem plausible.

Many of today’s machine learning applications for climate sciences are proof-of-concept studies that work in a simplified environment – for example, with a spatial resolution much lower than in state-of-the-art ESMs or with a reduced number of physical variables. Thus, it remains to be seen how well machine learning can be scaled up to operational and reliable usage.

§ Towards neural Earth system modelling

Initially, machine learning in climate research was primarily used for automated analysis of patterns and relations in Earth observations. However, more recently, it has been increasingly targeted towards ESMs – for example, by taking over or correcting specific model components or by accelerating computationally demanding numerical simulations.

This development has led to the concept of “hybrids” of ESMs and machine learning, which aim to combine their respective methodological advantages while minimising their limitations. For instance, the hybrid concept has been explored for analysing continental hydrology.

Continuing this line of research will increasingly blend the so-far still strict line between process-based models and machine learning approaches. 

In a similar way to how “coupled” climate models link different parts of the Earth system – such as the ocean and atmosphere – new interfaces will establish a dynamic exchange of information between ESMs and machine learning tools. This two-way exchange allows process-based models to improve their physical consistency and, in turn, feedback knowledge to the machine learning component, allowing the whole hybrid system to continuously evolve.

The possible variants of such hybrid systems are summarised under the term – which we have coined – of “neural Earth system modelling”. This is illustrated in the figure below, which shows the stages of how ESMs (blue shading) and machine learning (yellow) can come together.

We have also outlined a set of characteristics and goals for neural Earth system modelling:

1. That hybrids ensure physical consistency – for example, conservation laws and other balance equations;
2. That hybrids can better reproduce and predict out-of-distribution samples, such as extreme events;
3. That hybrids can correct, improve and validate themselves;
4. That neural Earth system modelling allows replicability and interpretability.

§ Explainable and interpretable

When we are trying to improve ESMs, there is a danger that everything starts looking like a nail to be hit by the new machine learning hammer. Here, methods called “interpretable AI” (IAI) and “explainable AI” (XAI) are coming to play. 

Within machine learning, making a correct prediction for the wrong reasons can be termed taking a “shortcut”, or having a system description that is “underdetermined”. Taking shortcuts is increasingly likely within climate science because the data available to us from the observational record is short and biased towards recent decades.

As a community we’re still only dipping our toes into the potential benefits of machine learning, as well as learning about the inevitable pitfalls. What is abundantly clear already is that machine learning can give us answers that seem correct for entirely wrong reasons.

IAI focuses on building models that are intuitively interpretable, while XAI is used to retrospectively assess why a neural network made the predictions it did. Both IAI and XAI lend themselves to giving any machine learning application within an ESM a “sanity-check” to see if it learned anything physical. This is important, as – by definition – any application to assess future climate operates in an out-of-sample space, where the core physical laws would not change, but the way the system expresses them likely would.

Methods within machine learning are evolving very quickly, IAI and XAI being no exception. Other methods that seem to have potential are adversarial learning (aiming to fool machine learning systems with misleading inputs) and “few-shot” methods (aiming to effectively train machine learning systems with only a small amount of data) and overall methods that are targeted more at learning underlying equations.

While the community surrounding ESMs explores the new machine learning tools and the powerful potential within them, it will undoubtedly accelerate progress if the different tools are recognised as such – that each may be appropriate for different tasks. However, ensuring that the machine learning predictions adhere to known physics or mechanisms is tantamount to building the needed tools that ESMs aspire to be.