Can the public make sense of uncertainty in weather and climate prediction?
Despite it being a good idea to carry sunglasses, an umbrella and snow provisions at all times in the UK to be truly prepared, people generally want to be told something certain about the weather. But forecasting how earth’s climate system will evolve in the next few hours, days, weeks and months is far from an exact science.
Following our blog last week from the Royal Society meeting on uncertainty in weather and climate prediction, we take a look at what uncertainty in forecasting means to the general public.
In forecasts we see online, climate scientists express some of the uncertainty by giving probabilities rather than a precise prediction. For example, the Met Office predict a 80 per cent chance of rain in London this afternoon. But when the Met Office started to do this last year, complaints in some parts of the media revealed some of the issues scientists face.
Trust issues
As Liz Stephens from the University of Bristol explained to the Royal Society meeting on Friday, a Daily Mail piece complained that the extra information was confusing and that it could be used as “a way to deflect blame” when people found themselves caught in unexpected downpours.
But people have a fundamental understanding of uncertainty and use it everyday to make decisions. As Professor Peter Webster from the Georgia Institute of Technology explained:
“Ask people how they cross the busiest road in their area, they assess risk all the time”.
We tend to accept uncertainty when doctors talk about medical treatments in terms of the chance of success. So why do people trust climate scientists less when they talk about uncertainty in weather and climate forecasts?
Weighing up the odds
Scientists use their expert knowledge and thousands of daily observations from across the globe to design computer models that mimic the climate system. They run slightly different versions of the model to determine the most and least likely outcomes for how the weather could change over time. This is called an ensemble forecast.
Even the best models are approximations of reality because the real climate system has levels of complexity that we cannot fully capture this way, or even yet comprehend. As Professor Tim Palmer, organiser of the meeting, explained in an interview with Carbon Brief:
“Inevitably there are errors and uncertainties in these observations and the observations don’t cover the entire globe so the initial state will be slightly uncertain.”
Image - Michael Fish _forecast 1987 (note)
Michael Fish’s infamous BBC forecast of a storm that hit south east England in 1987. The models predicted England would escape most of the damage, with the storm passing over the channel into Europe.
Ensemble forecasting is also the way scientists work out seasonal and long-term forecasts. But uncertainties amplify over time, which means that scientists’ ability to accurately mimicking the real climate system decreases for longer forecasts. Palmer continues:
“The famous Michael Fish storm of 1987 was a classic case where tiny uncertainties in the position of a low pressure system over the Atlantic made an enormous impact on whether it amplified or not.”
Communicating uncertainty
From a scientific perspective, quantifying uncertainty with probabilities is a good thing that leads to more reliable forecasts. But it also presents a challenge. As the Met Office’s Ken Mylne explained to the meeting:
“We can’t get away from the uncertainty of the climate system, so we have to find ways to communicate it”
Many techniques exist for presenting weather forecasts, ranging from a simple ‘deterministic’ prediction of what the weather will be like – for example, ‘overcast’, ‘sunny intervals’ or ‘light rain’ – to a complex array of probability information and graphics. For example, the UK Met Office website offers users a forecast which shows the maximum and minimum forecasted temperatures as well as the most likely temperature.
Image - Met Office _Screen Shot (note)
One way the Met Office show uncertainty in weather forecasts is by showing the full range of forecast temperatures, not just the most likely.
To explore whether some ways of displaying uncertainty are more effective than others, the Met Office launched a six-week public survey last year called “The Weather Game”.
Preliminary analysis of the results from the 8,000 participants suggests that adding uncertainty information to a forecast did not confuse people. Interestingly, adding an image such as an umbrella or a shaded bar to the likelihood of rain, expressed as a percentage, did not improve people’s ability to interpret the forecast. The fact that cold hard numbers are particularly effective is perhaps a surprising result, but as Mylne explained in an interview with Carbon Brief today:
“People might think that they are confused by a number but it’s unambiguous, whereas people might interpret images or words like ‘medium risk’ in different ways”.
It’s important to realise that people will have different preferences and that different methods might work for different media, however. Stephens, the Weather Game project’s leader, told Carbon Brief yesterday:
“It might be that if the probability of precipitation was presented in a different setting, like on the television, we’d get different results”.
The Met Office isn’t the only organisation experimenting with uncertainty information. The BBC commissioned some research into the best ways to communicate probability, the results of which should be released soon.
Where do we go from here?
The message from the research seems to be that, despite concern in the media, complexity is not a bad thing when it comes helping people understand uncertainty in weather forecasting.
For longer-term climate forecasts, scientists do not have the luxury of comparing forecasts to the actual weather every few weeks to continually improve their models. Instead, the uncertainty needs to be gradually reduced through more long-term observations and improving scientists’ knowledge of the climate system through research.
As Mylne told us, the key to improving understanding of short term weather forecasts, extreme weather forecasts and longer term climate forecasts is familiarity.
“We need to start getting these [probability forecasts] out there more…then it will be so much easier for people to understand things like seasonal forecasts and long-range forecasts that they don’t see everyday”.
But he agrees that this shift is likely to be more of a marathon than a sprint. If just a small proportion of people understand today why uncertainty in weather and climate prediction is not a dirty word, more of the next generation will, and so on until probability forecasting becomes the norm.