Why the pollsters were so wrong

Gary Welton

The election-night coverage of the 2016 vote began with images of the Clinton campaign team gathering in the Javits Center—under the symbolic glass ceiling. Hillary Clinton supporters were enthusiastic, upbeat and expectant. The polls gave them every reason to expect that America was electing her as the country’s first female president.

There’s just this one little detail, however: the polls were wrong, and the party never happened.

Not since 1948, with the non-election of Thomas Dewey, have the polls been so wrong. Just as in 1948, however, the problem does not rest in the field of statistics, but rather in the field of research methodology.

Republican political consultant Mike Murphy suggested, “Tonight, data died.” He pointed to a very real failure of election forecasting, but his quote suggests a problem with statistics. In fact, however, the problem was in the failure to obtain representative random samples.

The error in 1948 was because many voters did not have telephones. A random sample restricted to homes with telephones might totally misrepresent the political tendencies of homes without telephones. The homes without phones generally had fewer economic resources and different voting priorities. The failure to use random representative polls of all potential voters resulted in the faulty prediction.

In 2016 we use our telephones differently than we did historically. In the past, when my phone rang, I answered it. After years of robo-calls, however, I no longer answer my phone until and unless I decide whether or not I want to talk to the caller.

As a consequence, it is progressively more difficult for pollsters to complete a representative random sample of voter behavior. Our polls are more likely to predict how people who do not use caller ID will behave. It is possible that people who seek more control over their telephones might tend to be the same sort of people who seek more control over their government and other areas of their private lives.

The 2016 poll numbers might have been rigged, but not through any intentional work by the media. Rather, they were unintentionally rigged by the realistic hurdles faced by behavioral scientists, as they seek to explain and understand human behavior.

Even if we are able to fix the research methodology, it is important to remember that statistics always include a margin of error. The data only provide insights into the world. Like the weather forecast, they are not intended to provide any guarantees.

Dr. Gary Welton is assistant dean for institutional assessment, professor of psychology at Grove City College, and a contributor to its Center for Vision & Values.