In Defence of Liberty

Driven by data; ridden with liberty.

General Election 2017: Seat Predictions and MRP Models

Whilst opinion polls can estimate party vote shares across Great Britain (or whatever part of the country is being surveyed), this is not a precise indicator of the number of MPs in the House of Commons.

Various organisations created seat predictions, based on opinion polls and other data.

All but one of the prominent predictions were for a Conservative majority of varying sizes. Only YouGov’s MRP model for the 2017 General Election indicated there would be a hung parliament as its central estimate.

general-election-2017-seat-predictions.PNG

I really need to buy Excel.

 

What is an MRP model?

MRP (sometimes called ‘Mr. P’) stands for Multi-level Regression and Post-Stratification.

AceAttorneyDualDestinies-PW-3DThinkingSpeak

The original methodology was found in a 1965 MIT Press paper by Pool, Abelson and Popkin. The purpose of this statistical technique is take national polling data and make projections onto sub-national units.

For UK elections, we are interested in House of Commons constituencies. In the US, this technique has been typically used to estimate state-level opinion.

There are multiple steps to the MRP modelling process, which is explained in a Princeton primer.

Firstly, gather national opinion polls and combine the survey results into a single data set. These polls should include demographic data (that is also in the national census), and some geographic variable. If multiple polls are used, group variables can be used, which can help control for different wordings, house effects, and changes over time.

Secondly, collect census data to enable post-stratification. It is not enough to know headline statistics for how may men and women live in each constituency. In this model, opinion is treated as a function of demographic and geographic variables, such as gender, age, race, education. We need to know, for instance, how many 18-24 year-old black university graduates there are in each constituency.

Thirdly, fit a regression model for a survey response (like voting intention) given the respondents’ demographics and geography. This model treats every individual as if their opinion is an outcome of their demography and geography. We need to know what percentage of each different combination of gender, age, race, education, and constituency (as an example) would vote Labour, Conservative, a different party, or not vote at all, based on the survey data.

Finally, post-stratify the demographic-geographic types. The previous steps have given us a modeled probability that any adult will give a particular survey response (such as voting Conservative, or voting Labour). The final step is post-stratification, by computing a weighted average of these probabilities, to estimate the proportion of adults giving particular responses in each constituency.

Are MRP models better than polling?

Not necessarily.

An MRP model is a method of using national survey data to produce estimates of vote intention (or some another survey response) in constituencies (or another sub-national unit).

MRP models are projections, based on survey responses, and so are prone to the same sampling errors that polls are. Moreover, if the MRP model does not include an important demographic variable in relation to the opinion that is seeking estimation, it may fare poorly.

To demonstrate this, the YouGov 2015 ‘Nowcast’ suggested that, as its central estimate, Labour and Conservatives would both get 276 seats.

YouGov 2015 Nowcast May 6-01.png

This was incorrect.

Furthermore, the Ashcroft Model was also an MRP model for this election, and suggested (as a central estimate) there would be a Conservative majority of around 60.

This year, the YouGov MRP model was remarkably accurate, calling 93% of constituencies correctly, with a disproportionate number of the incorrect estimations occurring in Scotland in close contests.

general-election-2017-yg-est.png

This image is, of course, courtesy of YouGov.

If there is a sampling error, for whatever reason, this error will flow through the MRP model and produce an erroneous estimate.

We may see more polling organisations (like YouGov) and clients (such as Lord Ashcroft) produce more MRP models in future. The large samples required for the MRP modelling mean that is an expensive process, in terms of both money and statistical expertise. However, those large samples also mean that there is smaller sampling variation than in standard polls.

Expressing the probabilities of outcomes (like a hung parliament or a majority of a specified size) will remain important.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Information

This entry was posted on June 11, 2017 by in National Politics and tagged , , .
%d bloggers like this: