Businesses not allowed to be ‘unconcerned’ about green charges in npower poll
Are three quarters of major energy users worried about about what green policy measures might means for their businesses, as the Sunday Telegraph claims in an article citing polling by Npower? They might be, but npower’s survey doesn’t prove it.
Businesses are protesting over the rising cost of green energy reforms, according to the front page of the Sunday Telegraph business section.
The story is based on an online survey created by energy company npower. The survey aims to give businesses the chance to express their views about the government’s Electricity Market Reform (EMR) package – a set of policies aimed at shifting the country away from fossil fuels and toward nuclear and renewable power.
But the Telegraph’s doesn’t get its reporting quite right. And a closer look reveals some problems with the poll, too.
Not all big energy users
Three quarters of “major energy users” said they were worried about the impact of the government’s green overhaul of the energy system on their businesses, according to the Sunday Telegraph.
This statement is an over-interpretation of the poll – which found that three quarters of businesses are concerned, but didn’t specify who the businesses were.
Asked about the proportion of respondents who came from big energy consumers, a spokesperson tells us:
“There will definitely be some major energy users that will have taken part.”
Npower says 75 per cent of respondents identified themselves as coming from businesses that consume energy, and 25 per cent as energy consultants or brokers.
The company doesn’t appear to have taken any measures to control who filled the survey in – anyone still can – but some of the publicity was focused on its big energy consumers. So a more accurate interpretation might be that three quarters of respondents – some of whom could be big energy consumers – are concerned about green charges.
‘Puny’ sample size, big margin of error
Meanwhile, npower’s poll has bigger problems – perhaps the most striking of which is sample size. Despite the potentially large pool of contributors, the survey only attracted 65 respondents – a small number in comparison to the number of businesses in the UK.
Polling expert Leo Barasi tells Carbon Brief:
“…. a sample of 65 people is puny. If the universe of people in the UK that could potentially enter the poll was very small, say 80 people, 65 wouldn’t be too bad. But they’re asking for energy consumers or energy brokers in UK businesses, which seems like it could apply to essentially every business in the country.”
As a result, the margin of error on Npower’s poll is probably around 12 points, Barasi says. That means every data point could be 12 percentage points higher, or 12 percentage points lower.
By comparison, a standard political poll has a margin of error of about two points.
Skewed towards concerned
Barasi also criticises the phrasing of the question which asks the respondent to agree or disagree with the statement “I am concerned about the impact Electricity Market Reform will have on my business”:
Image - Screen Shot 2013-09-03 At 09.15.07 (note)
Polling questions that present people with statements and ask them to agree or disagree “produce results skewed towards agreement”, says Barasi. A better way to get a realistic idea of the respondents’ views could be to present two opposing statements and ask which they agree with more – or ask a straightforward questions like “how concerned are you about the impact EMR will have on your business?”.
But Barasi adds:
“Even that isn’t especially useful though, as it costs nothing for respondents to say they’re concerned. What we really want to know is how it ranks relative to other concerns, so we can see whether it really matters. The poll doesn’t attempt that.”
Businesses can’t be unconcerned
Npower’s press release – and the Sunday Telegraph article – also highlight that the vast majority of respondents (nine out of 10) are worried about the impacts of government’s reforms on UK competitiveness.
This appears to be based on the following question:
Image - Screen Shot 2013-09-03 At 12.37.07 (note)
Notice something unusual? Barasi points it out:
“The … question series only gives ‘concerned’ options, with no ‘unconcerned’ options. The results from them, which apparently show high ‘concerned’ scores (!) should be discounted. The quote in the article, that UK businesses are worried about competitiveness, can’t be drawn from the poll.”
So it’s perhaps not that surprising that nearly all the respondents said they were concerned. Overall, npower might be right that businesses are worried – but if you don’t give people the opportunity not to be worried, it’s perhaps not that surprising.
Leo Barasi blogs about public opinion at Noise of the Crowd and helped Carbon Brief with our regular polling.