Taps Coogan – October 28th, 2020

Enjoy The Sounding Line? Click here to subscribe.

Here’s the thing about polls. Despite our strong desire to believe that we can predict the outcome of an election by looking at the polls, history suggests otherwise. A recent study by Berkeley looked at 1,400 polls from 11 presidential election cycles and found that they only predicted the correct outcome 60% of the time despite relying on 95% statistical confidence intervals.

It’s not that the margin of error statements that come with polls are lies. Most elections simply end up being close enough to fall within the margin of error. This election is not an exception. Furthermore, the reported polling margin of error represents just one source of error in polling: sampling error. It doesn’t include errors in the turnout models that polls use to approximate the demographics of likely voters nor the possibility of future changes in sentiment.

There is another problem. Back in the 1990s around 30% of people responded to telephone polls. By 2018, that number had fallen to 7%. A glance at polls from this election cycle suggests that the number has fallen well below 5%. When 95%+ of people don’t respond to polls, one has to wonder how representative the remaining 5% are.

There is some evidence that Republicans are less likely to respond to polls and twice as likely to mislead pollsters. One admittedly Republican leaning polling group says conservatives are four to five times less likely than Democrats to complete a poll in 2020. Is all that true? Who knows… You can’t really poll people on whether they’re honest with pollsters. A large study of political polls over the last 75 years did not see a decline in polling accuracy but that has as much to do with polls not being particularly accurate 75 years ago as anything else.

What Do the Polls Say?

State polls are what count when it comes to picking the president and the observed accuracy of state polls is in the ballpark of 5%. In other words, if two candidates are within 5% in state polls, it’s best to assume that you can’t predict the outcome based on the polls. At the time of writing, 181 electoral college votes are in states where the polling margin between the candidates is within 5%, more than enough for either candidate to win.

Despite pollsters’ and pundits’ compulsion to create Rube Goldberg-like models that turn statically inconclusive polling results into statements like ‘Biden has an 88 out 100 chance of winning,’ the reality is that this election remains within a standard polling margin of error. That’s really all the polls have to say with any real confidence. In fact, that’s almost always what the polls have to say about any given election. It’s not exactly the stuff that builds a high-flying punditry career, hence the tendency of the media to over-interpret and overstate polling time and time again.

As a final note, if there was ever an election where the polls should be taken with an extra grain of salt, it would be one with unpredictable changes in voter turnout and voting methodology, where the pollsters are believed to be partisan by roughly half of the people that they’re polling, and where people’s lives are being ruined for expressing their voting intention in public.

Would you like to be notified when we publish a new article on The Sounding Line? Click here to subscribe for free.