If you ask Americans whether they trust the polls, many seem unable to let go of what happened in 2016. Polls taken since then have generally found that a majority of Americans have at least some doubts about what polls say. But as FiveThirtyEight wrote in the run-up to the 2016 election, Donald Trump was always a normal polling error behind Hillary Clinton.
And that’s essentially what happened in 2016: Trump beat his polls by just a few points in just a few states. The presidential polls were, simply, not that off. State-level polling was less accurate, although as editor-in-chief Nate Silver wrote after the election, it was “still within the ‘normal’ range of accuracy.”
That doesn’t mean there weren’t plenty of polling lessons to be gleaned from 2016, though. The importance of education in predicting a person’s political preferences was a big one. And so to better understand those takeaways, we contacted 21 well-known pollsters to find out how they adjusted their methodologies, if at all, and what concerns them most about polling in 2020. In the end, 15 got back to us — a 71 percent response rate that pollsters only dream of in this day and age. Here’s what they had to say.
More pollsters are weighting by education and using new ways to reach respondents
Nearly every pollster we talked to has made some kind of modification since the last general election. Some changes were precipitated by what happened in 2016, while others were driven by the challenges facing the polling industry, such as low response rates to phone calls and the greater cost of high-quality polling.
But one thing came up again and again in our interviews: Pollsters told us they were now weighting their samples by education, because one key takeaway from 2016 was just how important someone’s level of educational attainment was in predicting their vote. “In mid-2016, we changed our weights by education, moving the percentage of high school or less respondents up while dropping the college-plus down,” said Jeff Horwitt, a senior vice president at Hart Research, one of the pollsters for the NBC News/Wall Street Journal poll. It was the middle of the election cycle, but already Horwitt and his team were concerned that they might be underestimating the share of the electorate who didn’t have a four-year college degree, and therefore, missing some of Trump’s support. They were right to be concerned, too. A real problem for the polling industry writ large was the underrepresentation of voters with little or no college education.
Some pollsters such as Ipsos and the Pew Research Center have taken weighting by education a step further by weighting for educational attainment within racial groups. This change could be especially important in state-level polling in 2020, as Trump primarily outperformed polls in states that had large populations of white Americans who didn’t have a bachelor’s degree. “This year we are ensuring that we include the combination of education and race/ethnicity in our sampling,” said Cliff Young, president of U.S. public affairs at Ipsos.
Monmouth University Polling Institute director Patrick Murray cautioned, however, that weighting by education isn’t a silver bullet. He noted that weighting by education in a postmortem analysis of their own polling in 2016 had only “a small impact on accuracy and on its own [did] not explain the supposed polling miss in 2016.” Still, weighting by education was, by far, the most common methodology change pollsters reported.
As for other changes since 2016, Marist College Institute for Public Opinion director Lee Miringoff told us they’re paying closer attention to where their respondents live — that is, are they mostly concentrated in a city or outside…