Neil Turner's Blog

Blogging about technology and randomness since 2002

The problem with election polls

The results of the British general election on Thursday were unexpected, not at least by me. For months, the polls had the Conservatives and Labour neck-and-neck, with neither party winning a majority of seats. As such, it was anticipated that one of these parties would have to govern as a coalition, or form a minority government. That didn’t happen and the Conservatives won a narrow majority.

This situation was unthinkable right up until the polls closed, when the first exit polls were released. The BBC’s poll suggested that the Conservatives would win 316 seats – ten short of a majority – and this was completely at odds with the polls and predictions. As it was, the Conservatives did even better and won more than the 326 required for a majority.

So why did the polls get it so wrong? Well, I’m not a professional pollster, but, armed with an A-level in Mathematics & Statistics from 2002, a university degree and a job working in data analysis, I’ll try to go through some of the reasons why things didn’t work out.

Popular vote vs Number of seats

There are two ways of measuring the election results. One is a simple popular vote, or ‘vote share’: how many people nationally voted for each party, across all constituencies. The second is based on the number of seats won by each party. I’ll explain what this means.

The House of Commons, where Members of Parliament are elected to sit (on benches rather than physical seats), has 650 seats, with each seat representing a ‘constituency’ – a geographical area of the UK. The constituency boundaries are set to be roughly equal by population; in sparsely-populated rural areas, a constituency may cover a large area of land, whereas in a major city it may only be a few square miles at most. The voters, or constituents, of each constituency, elect someone to represent them in their seat in the House of Commons, and generally the political party (or parties) with the most number of seats forms a government.

So the number of seats is the measure that actually matters. And the two don’t always tally; the BBC shows both results so that you can compare. Whilst the Conservatives and Labour came first and second respectively in both, the SNP (Scottish Nationalist Party) won the third highest number of seats despite having the fifth highest share of the vote. Their candidates only stood in Scotland, but were very successful with a net gain of 50 seats from the other parties, even with only 4.8% of the national popular vote. UKIP (UK Independence Party) came third in the popular vote, with over 12% of people nationally voting for them, but only won one seat. The Green Party also only retained one seat, with 3.8% of the popular vote.

Sample size

Opinion polls can’t ask every voter how they will vote – the companies running the polls simply don’t have the resources, and I think most people would get pretty annoyed if they were asked week-in, week-out who they will vote for. So the pollsters take a ‘representative sample’ of voters, and then extrapolate this to a national picture. They will pick, say, 1000 people across the country and ask them who they will vote for.

And that’s fine for calculating the popular vote. But it’s rather useless when calculating how many seats this will translate into. Remember, there’s 650 seats – if you only ask 1000 people, then even if you can guarantee an even spread across all constituencies, you will only get the opinions of either 1 or 2 people per constituency. That’s not representative.

To get a more accurate picture, you will need to get a representative sample in each constituency. You’re looking at 100 people in each of the 650 seats – so you’ll need to poll 65,000 people each time. That takes a lot more resources.

Sample bias

The method of polling people also affects the sample. YouGov is one such pollster and the one that I’m most familiar with, as I take surveys with them regularly. Assuming that its polls rely on its online panel members, then you already have a couple of problems with a biased sample.

Firstly, the panel that you are asking is self-selecting; these are people who have signed up to YouGov to fill out surveys in return for cash (typically 50p each time). And to be able to fill out YouGov’s surveys, you need a computer or mobile device with internet access, and whilst this may be a surprise for people like you reading blogs like this, but not everyone in the UK has internet access. Similarly, any pollster that relies on phone surveys will not be able to contact anyone who doesn’t have a landline telephone (unless they call mobiles).

To get a truly accurate poll, you will therefore need to make sure that your sample reflects each individual constituency, and that you don’t introduce inadvertent bias with your data collection methods. It’s a hard task, and may go to explaining why the polls did not accurately reflect the ultimate result. I say ‘may’ – I don’t know the full details behind their models, but I’ve heard that there’s an increase focus on trends.

I’m sure the pollsters will be looking carefully at what happened last week. They’ll almost certainly need to change their methods, so that the public and politicians will still have faith in their data in future.

Comments are closed.