Punking the Polls

Polls do not predict things. They measure the answers to specific questions. about certain topics, usually how someone feels about someone or something (candidate, policy, breakfast cereal) and what their intentions are (vote, not vote, buy, not buy). A pollster quantifies the answers are quantified. There is no soothsaying in what is quantified. Polls are nothing more than data.

Some polls are very good at measuring public opinion. Others are not. Whether a poll is reliable or not comes down to methodology.  How the questions are worded and who is being questioned influences the outcome of polls, which is why when you read a poll, you first look at the methodology. The methodology will tell you the story behind the poll. A poll without a backstory or methodology is not worth reading.

The polls we read about in the press are one of two types: There are external polls, ones that collect information used to inform the general public. Then there are internal polls, which are not released to the general public (though they can be leaked). Internal polls are used to inform whomever financed the poll, such as a soft drink maker looking to test the market for a product or a politician wanting to know what issues plays best among a certain demographic.

Easy to see the usefulness of internal polls: A political campaign or snack food maker can use polling information for strategy and marketing. But what about external polls, those polls we see released by news networks, newspapers, universities, etc.? Who finds use in those polls? If we are talking politics, the answer is politicians, policy makers, campaign strategists, advertisers, journalists, and pundits. (Not the public is not in that list.

Journalists tend to report poll data by turning the numbers into readable prose. Good journalists contextualize the data by telling you about the methodology. Really good journalists will reference other polls and give a historical twist to the reporting. Pundits tend to report on polls and interpret them. Some use the polls to make predictions.

There are pundits who are very good at writing on polls, Nate Silver is one of the best. Silver tends to interpret and explain polls, while commenting on a poll’s reliability. He also uses polls to calculate the odds of something happening by measuring probability. For years, Silver did this with sports. Later he turned to politics. In sports, there are many people like Silver, people who use data to calculate odds. It is something that has been going on for decades and, in the sports gambling world and fantasy sports, there is an industry around data analysis.

I’d say that 99.99% of the people who play with sports numbers, as well as those that consume what folks like Silver report on sports, realize that odds are not predictions. Odds measure probability. If you are to use measures of probability to make a bet, well, that decision is on you. You don’t blame the numbers or the person reporting the numbers. You understand that no one can predict whether or not the sun gets into an outfielder’s eyes preventing him from catching what should be an easy fly ball, which would have prevented the tying run to score from third, which led to extra innings in which the outfielder’s team lost and so did you because you had fifty bucks riding on the game. You accept that you were screwed by chance.

Sports gamblers also understand that every contest has odds and that odds do not guarantee an outcome. Often gamblers will bet against the odds and pick an underdog because the payoff of a team beating the odds is better than going with the odds. So, when a team wins in an upset, making good on the 25% chance that statisticians and odds makers gave the underdog of winning, those who bet against the odds are pleased. They also accept that there is a 75% chance of what they want to happen not happening. But, because humans and life is unpredictable, they are willing to take a chance and back what most people agree is a loser.

Unfortunately, the public doesn’t approach polls or reporting on polls the way gamblers or sports fans do sports. So, when someone like Nate Silver says, in 2016, that based on the polls, Clinton has a 75-85% chance of winning we ignore the 15-25% chance that she might lose. When the underdog (Trump) beats the odds by falling within the 15-25% percentile, we say that the polls are wrong or that Silver called it wrong or both. Truth is the polls presented numbers and Silver interpreted them as he would have a Yankees v Red Sox game. We read certainty into the numbers and used them as predictions. The public screwed up in its interpretation of an interpretation. So, did some of the pundit class.

This is very important to keep in mind going forward. Right now, the polls show Biden with a substantial lead over Trump, not just nationally but in key states. Odds are that the numbers reported now mean a Biden victory if everyone who says that they will vote votes the way they say they will and there is not interference or shenanigans. I do not expect anyone, especially those in the Biden campaign, to take polling data as a prediction of what will happen. I think we are good there. However, I do fear that the narrative from 2016 that “the polls were wrong” will be used to justify Trump’s claim of victory.

Here is a scenario I can see playing itself out:  Election night, polls close, Trump is ahead because all the counting isn’t done. Trump declares victory. When challenged, he points to the polls and says, “See, once again, the polls are wrong. You can’t trust the polls. Polls are fake news. I won.” Those who believe the fallacy that the “polls are wrong” based on the ignorant belief that polls predict outcomes could be swayed to agree with Trump. “Well, I guess the polls were wrong again, so we get four more years of Trump. Sigh.”

Knowledgeable people will defend the polls or try to explain how polls work, but it will be too late. Once the election results start getting reported, the narrative that the “polls are wrong because they predicted wrong” is stuck. Pushing back against the narrative after it has been established does more to reinforce it than break it – especially in this black/white/right/wrong world we are in. In this case, the only people that benefit from nuance are the Trump people, for once again, they can point at the “under” and state that, once again, Trump beat the odds.

Here’s the thing, something we must stress over and over: Vote counting does not end on election day. States have approximately one month before they need to complete the count to know who to choose to be delegates to the Electoral College. Ideally, every vote is counted by then. There are times when it takes longer, as it did with Florida in the 2000 presidential election.

However long the vote count takes, it has always taken longer than the few minutes, few hours, or even few days past the closing of the polls. It can take longer than a week after election day or two weeks or a month. It takes as long as it takes to count the vote, and that is what we must not only keep in mind, but repeat out load as a mantra. It takes as long as it takes. It takes as long as it takes. It takes as long as it takes.

And throw a “No!” in front of that mantra. “No! It takes as long as it takes.” If you need more emphasis, consult Samuel L. Jackson. Point is, when Trump and his cult start with the “I won” and point to the polls as being wrong once again, dodge that bullshit and counter with the truth, “The votes are still being counted. It takes as long as it takes.” Push back against any claim that “We need to know now,” “The country can’t stand waiting,” or “This uncertainty is dangerous.” More bullshit that you can easily counter with a simple Wiki read of past elections.

You have read enough of my stuff to know that I like to use polls, but you also know that I put them in context and never use them as predictors. Polls are useful for me in my work. However, right now, this close to the election, I don’t have a need for them and I doubt you do, too. Now is a good time to break the habit of poll watching and start prepping for defending your vote, which means demanding that every vote be count. Any attempt to call the election immediately and stop the count is an attack on democracy and your freedom. There is no over/under on that statement. It is 100% true.