Commentary November 19, 2020

The 2020 Election Was a Perfect Example of the Weaknesses — and Strengths — of Political Polls

Daniel A. Cox

Business Insider

In the wake of the 2020 presidential election, there have rightly been a lot of questions raised about the reliability and accuracy of public opinion polling.

Although the votes are still being tallied, it’s clear that in a number of states — such as Wisconsin, Iowa and Florida — state polls missed the mark by a wide margin. But this miss wasn’t a uniform phenomenon. In other states — such as Colorado, Georgia and Nevada — the polling did fairly well.

Certainly, the misses are attracting more attention and leading early critics to publish overwrought headlines (See: The Polling Catastrophe) that probably overshot the severity of the problem.

In aggregate, the election polls, which included a wide variety of different methodologies, did okay. And when you dig beyond the surface, it is clear that some polls and pollsters did better than others.

The 2020 election showed a problem with “election” polls, not all polling

Polling serves a critical function in American society beyond alerting us to the ebbs and flows of political campaigns. While many Americans do only pay attention to polls around election time, survey research is critical for our understanding of the country we live in and what our fellow citizens believe.

At this point, it seems unlikely that the problems that beset political polls would compromise our ability to measure religious identity accurately, or understand consumer preferences. In these instances, we are not attempting to predict future behavior of the electorate — which is at its core an unknown subset of the overall population.

Polling is not well-equipped to predict human behavior

Close election-watchers will know that many pollsters and election modelers talked a lot about uncertainty. It turns out that human beings can be somewhat unpredictable. Who knew?

Surveys allow us to get a sense of human behavior, experience, attitudes and feelings in a particular context at a particular moment in time. Election forecasting requires extrapolating out from that and making big and small assumptions about what is and is not likely to change in the run up to Election Day.

Few research methods are subject to the same level of scrutiny as surveys. And that’s okay. Surveys have become incredibly common and we rely on them to make informed decisions about public investments and private enterprise.

But it’s important to remember that polling is a methodological approach to understanding the world, just like ethnographic fieldwork, qualitative interviewing and experiments. They each have limitations. That does not lessen the value.

The problem is that the yardstick we use to judge polling is its ability to predict human behavior, something that no methodology is equipped to do well. We don’t apply the same tests of predictive accuracy to other methodologies. Even still, polling does a decent job at predicting limited outcomes under certain conditions.

There’s a lot we would not know without polling

Without polling we would know little about the shifting religious landscape in the US.

We would know little about decline of Christian identity and the rise of the religiously unaffiliated. We would know far less about the rise of political polarization or its proximate causes. We would be caught off guard, by changes in cultural norms and political attitudes — such as the rise in support for same-sex marriage and marijuana legalization as well as the increasing acceptance of premarital sex. We would be much less well-equipped to respond to public health challenges, such as the opioid epidemic.

Surveys offer a rare opportunity for us to acquire knowledge about the views and feelings of others outside our personal experience.

Given how socially segregated Americans are along lines of race, class and increasingly politics polling serves a valuable function. If everyone I know is an evangelical Christian it might be very difficult to acknowledge or accept the fact that the country has a rapidly diversifying religious population because everyone I know identifies just like me.

The polling industry faces real challenges. Declining social and institutional trust means that fewer people are willing to take surveys. Those who are willing to complete them are different from those who are not in important ways. This is a problem that has likely been exacerbated during the Trump era as he has regularly sought to discredit sources of information that are critical of him.

Additionally, new methodologies are being introduced that are still untested, while the “gold standard” live-interviewer telephone polling has experienced plummeting response rates. But the answer of course is to acknowledge the limitations of polling, experiment with new and emerging technologies and approaches and most of all be clear about what we’re doing and why. Public trust is built on a foundation of transparency.

Looking ahead it’s important to acknowledge the challenges, while working to address them.

There are structural problems with election polling, but the answer is not to banish all polls from our elections. For one, if reputable pollsters bow out of horse-race polling it leaves the space open to bad actors and opportunists who may conduct surveys for ends other than informing the public.

There has probably been too much coverage of horse-race polls, but rather than ignoring them political reporters should contextualize them more using additional information such as historical voting patterns, registered party identification when available, and personal knowledge of communities and people developed through on-the-ground reporting.

Finally, try to imagine another metric that would allow you to gauge the status of an election in real time. Fundraising? Rally size? Political endorsements? There are biases built into these metrics that would only be identifiable post-hoc. For those of us interested in understanding the world, polling offers an incredibly useful and cost-effective tool. It’s critical that we get it right.