Jumping the Electoral Shark

For months after the 2016 election, there seemed to be no shortage of articles and think pieces about how the national polling apparatus failed to predict the Trump win and how this failure was a result of inaccurate polling methods. And now, just over two years since the failure of these polls, the media is gearing up for 2020 will a host of predictive polls that read more like fanfiction than hard data (but hey, at least we know that Joe Biden has a seven point hypothetical lead on Trump). So, before the storm of polls begin to dominate the 24/7 newsroom, I figured I would try and figure out how polls are conducted, how conclusions are drawn from them, and if we can even rely on polls at all. I’m sure I’ll answer all of these in a two page blog post.

The primary method used by pollsters and publications alike to conduct polling is described by Nate Cohn in his article “Why Did We Do the Poll the Way We Did“. This method, called “random digit dialing”, is exactly what it sounds like. Random phone numbers are dialed until someone picks up and participates in the poll. This is, of course, not the only way to conduct a poll. Cohn explains that for the titular poll in question, the team over at the New York Times utilized registered voter profiles to gather their phone numbers. Already we can see the problems inherent in conducting polling, not just political polls either. All polls depend on there being a willing population of pollees for the pollsters’ analysis to of any use. Even if a poll satisfies all the basic rules of modelling in statistics (Random, 10%, Success/Failure) there is no guarantee that it’s accurate, all data gathered from polls is various shades of inaccurate.

Analysis is a whole different issue. Was the population composed of all possible phone numbers or just registered voters? What was your line of questioning? Were your questions leading? Might the pollees have been giving you “polite” answers? And once you have your data, whose to say that the reality of the situation hasn’t already changed? It is, in short, an incredibly difficult. I think this fact is really put on display when you see hardcore analysts just as uncertain at this point as anyone else.

In a March 20th podcast, the folks over at FiveThirtyEight.com discussed Trump’s re-election. Some, like Geoffrey Skelley makes the claim that, at present, “econometrics models” place Trump as a possible favorite for re-election. Others, Nate Silver specifically, are not so certain Trump’s economy can make up for the fact that his approval rating is sitting around 41% or 42%. The discussion goes around and around and never quite lends the listener a real sense of certainty.

Don’t get me wrong, there is still more than a year before anyone heads to the ballot box to cast a vote, so I don’t fault FiveThirtyEight or anyone else for not having some bold assertion about the result of the 2020 race. However, both politicians and voters rely on polls to make decisions, the former using them to determine if they should run to begin with and the latter deciding if their vote will be any good. But if decisions are being made based on faulty info, decisions can be made prematurely.

In the realm of polling, I find myself inclined to believe E. J. Dionne and Thomas E. Mann in saying that “Polling is a tool, not a principle“. I think that polls obviously have a necessary and permanent place in our political discourse. To suggest that polling should be done away with completely is far too naive (even for my tastes). What is important however, is that both politicians and citizens understand the deficiency in the polling apparatus.

Here’s Batman agreeing with me