In Defence of Liberty

Driven by data; ridden with liberty.

How to Read and Report Polls

Reporting on polling is often a difficult task, but people who report on polls have a responsibility to their readers and to their profession. There are critical pieces of information that readers must have to understand any specific measure of public opinion.

These pieces of information are:

The polling company and sponsor

Every major polling company (often called market or social research companies, or pollsters) is a member of the British Polling Council. The British Polling Council is a voluntary association with the purpose to:

  • Ensure standards of disclosure of polling results;
  • Advance understanding among the public and politicians of how polls are conducted and should be interpreted;
  • Provide advice and spread best practice of how to conduct and report polls.

This company is the organisation that actually undertakes the poll of public opinion. There are currently 16 members of the British Polling Council: BMG Research, ComRes, Forefront Market Research, GfK UK, Harris Interactive, ICM, Ipsos MORI, Kantar Public UK, LucidTalk, Mindmetre, Opinium, ORB, Panelbase, Populus, Survation and YouGov.

There are other polling companies that may be quoted as pollsters, but are not presently members of the BPC. These includes SurveyMonkey and Sky Data. Lord Ashcroft Polls is more accurately described as a client of polling companies.

NatCen (and ScotCen) undertake the British (and Scottish) Scottish Attitudes survey, and are headed by Professor John Curtice, who is also the chairman of the British Polling Council.

The sponsor is the person or organisation that actually pays for the polls to be conducted. Some polls do not have a sponsor, and are self-funded by the polling company. Often, a newspaper may be the one sponsoring the poll. However, the polling sponsor does not influence the survey results. Preferably, sponsors should be quoted as thanks for their service.

The Sample Size

Readers should know how many people were included in the sample, and what population these respondents were drawn from.

Typically, polls of British public opinion are focused on Great British adults.

Methodology and Fieldwork Dates

It is important for readers to know when a poll was conducted.

Also, there should be references to what type of poll it was. There are three prevalent survey modes in polls of Great British public opinion: face-to-face (where the interviewer speaks to the chosen respondent in person), telephone (where the respondent is contacted through their landline or mobile phone), and internet panels (where the respondent is contacted through their participation in an internal panel).

By stating the polling methodology, it should be clear that self-selecting polls should not be reported.

Full Question Wording

It is very helpful for readers to know exactly how each question was worded. This is available through the computer tables (otherwise known as the data tables, cross-tabs, or tables), where each question is given, in order, with a summary of the poll results.

Ideally, online versions of newspaper articles should link to these polling computer tables, as these tables are required to be produced by membership of the British Polling Council.

This is an example of a computer table from YouGov’s internet panel survey of 1,042 London adults, between 24th and 28th March 2017. This poll was sponsored by the Queen Mary University of London, as the first in a polling series for the Mile End Institute.

yougov-computer-tables.PNG

For instance, one question is:

In hindsight, do you think Britain was right or wrong to vote to leave the European Union?

Next, we can read across the tables to see the total percentages of respondents that said Britain was ‘right to leave’, ‘wrong to leave’, or ‘don’t know’. These were the coded responses.

Reporting the Shares

It should be a principal part of poll reporting to state the shares (to the nearest percentage) of each poll. It has become noticeable that poll reports tend to focus on the lead, that is, the difference between the largest party and the second party.

However, that lead may fluctuate more widely than the two party shares, due to any anti-correlation and random polling errors.

Reading the Tables

In the YouGov/Queen Mary University of London survey example, the total shares for ‘right to leave’ and ‘wrong to leave’ should be stated (34% and 53%, respectively).

That difference can be mostly explained by the EU referendum vote in London, as the percentages of Remain voters who said Britain was ‘wrong to leave’ and Leaver voters who said Britain was ‘right to leave’ are very similar (84% and 85%, respectively).

Typically, polling companies will show the vote shares in each question by the following segments:

Total: the total response for the whole sample;

Gender: self-reported gender of the respondents, with ‘male’ and ‘female’ shown;

Age: age of the respondents, placed into particular buckets, e.g. 18-24, 25-49, 50-64 and 65+;

Social Grade: this is a classification of the type of work that the respondent does, usually shown where A, B and C1 are placed in one group (“ABC1”) and C2, D and E respondents are placed in another (“C2DE”);

nrs-social-grades.PNG

Vote Recall: respondents are normally asked to recall how they voted in a previous general election or previous referendum;

Vote Intention: respondents are normally asked how they intend to vote in a future general election or future referendum.

Other segments may be given, depending on the poll.

In Summary

It is vital for journalists to impart the polling company, their sponsor, the sample size, the polling methodology, fieldwork dates, the full question wording and shares.

This good work can be supplemented by linking to the polling computer tables, which show the questions in order, with each response broken down by various segments.

 

Advertisements

Information

This entry was posted on April 8, 2017 by in Statistics and tagged , , .
%d bloggers like this: