In Defence of Liberty

Driven by data; ridden with liberty.

Misreporting Polls and Other Matters

Misreporting polls

A rather curious incident occurred with an ORB International poll, sponsored by the Daily Telegraph.

The article’s initial version began:

Support for Brexit has hit a five month high, with 55 per cent of the population now backing Britain’s exit from the European Union, a new poll has found.

A new survey from Orb International shows a four per cent boost for Theresa May in days after she triggered the start of Brexit talks at the end of last month.

This statement — that “support for Brexit” had attained a new high — was repeated in the original headline.

However, this is not what the question was. The actual question wording was:

Do you approve or disapprove of the way in which the Government is handling the Brexit negotiations?

This is, plainly, a question about perceived government competence in a policy area, and not about support for leaving the European Union itself.

The article was then amended to say “Theresa May’s handling of Brexit”, which is closer to the question wording, though it does not mention the Prime Minister by name.

Zero Hours Contracts, Again

I occasionally return to the matter of Dr Eoin Clarke and his repeated errors with regards to zero-hours contracts figures.

Let’s examine this one post on Twitter by Dr Clarke:

Firstly, Dr Clarke makes an error with the central estimate of the report he is citing.

The central estimate for the number of employees reporting they are on zero-hours contract was 905,000 during October to December 2016, representing 2.8% of all people in employment.

Secondly, there is an important caveat, regarding the large step change between 2012 and 2013. The ONS report states:

Comparisons with 2012 and earlier years are complicated by a large increase between 2012 and 2013 that appeared to be mainly due to increased recognition of “zero-hours contracts”. This change was covered in a previous ONS report published on 30 April 2014.

Thirdly, Dr Clarke duplicates effort by reproducing the line graph the ONS has already made in the cited report.


Fourthly, Dr Clarke’s graph title is slightly misleading: the figure is based on self-reporting of Labour Force Survey respondents. The Office for National Statistics highlights this definition in its report, where respondents classed by the LFS as being on “zero-hours contracts” will be those who:

  • are employed (have done at least one hour of paid work in the week before they were interviewed or reported that they were temporarily away from their job);
  • report that their working arrangements in their main employment include some form of flexibility;
  • recognise that the flexibility of their working arrangements is a result of being on a “zero-hours contract”.

The recognition that it is a self-reporting figure is important, because it helps explain the large surge between 2012 and 2013, due to media exposure and political focus on zero-hours contracts.

‘Polls Are Wrong’

With a general election looming, anti-polling sentiment festers. There is the instantaneous dismissal that any poll must be wrong, because of previous polling misses.

It is worth remembering that the average mean absolute error for large parties in final polls has been 2.3 percentage points.

By this measure, British polling has not been particularly worse.

Moreover, there must be a recognition of the size of these errors. Polling errors have not been especially large in the UK, but are often clustered around an incorrect result (as they were in the 2015 General Election), giving false certainty.

Ultimately, there is only one way we will find out if present polling for the 2017 General Election is accurate. To all the anti-polling people: let’s find out, together.



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


This entry was posted on April 29, 2017 by in National Politics and tagged , , , .
%d bloggers like this: