Pollsters fear they’re blowing it again in 2022

Pollsters know they have a problem. But they aren’t sure they’ve fixed it in time for the November election.

Since Donald Trump’s unexpected 2016 victory, pre-election polls have consistently understated support for Republican candidates, compared to the votes ultimately cast.

Once again, polls over the past two months are showing Democrats running stronger than once expected in a number of critical midterm races. It’s left some wondering whether the rosy results are setting the stage for another potential polling failure that dashes Democratic hopes of retaining control of Congress— and vindicates the GOP’s assertion that the polls are unfairly biased against them.

It’s not that pollsters haven’t tried to fix the issues that plagued them in recent elections. Whether they’re public firms conducting surveys for the media and academic instructions or private campaign consultants, they have spent the past two years tweaking their methods to avoid a 2020 repeat.

But most of the changes they have made are small. Some pollsters are hoping that since Trump isn’t running in the midterms, the problems of underestimating Republicans’ vote share will disappear with him. But others worry that Trump’s ongoing dominance of the news cycle — from the FBI seizure of classified documents at Mar-a-Lago to litigation against his businesses in New York — effectively is making him the central political figure going into Election Day.

“There’s no question that the polling errors in [20]16 and [20]20 worry the polling profession, worry me as a pollster,” said Charles Franklin, the director of the Marquette Law School Poll in Milwaukee and a longtime survey-taker in the battleground state of Wisconsin. “The troubling part is how much of that is unique to when Donald Trump is on the ballot, versus midterms when he is not on the ballot.”

After 2016, pollsters said the problem was their samples included too few voters without college degrees. The polls were better for the 2018 midterms, though they were still too Democratic on balance.

Then came 2020 — which was worse than 2016, and for which pollsters have yet to settle on a definitive explanation of what precisely went wrong. As a result, an easy fix has proven elusive. But pollsters have mostly agreed that, particularly in 2020, the surveys missed a chunk of Trump’s voters who refused to participate in polls.

The current 2022 polling is wildly favorable for Democrats. FiveThirtyEight’s “lite” prediction model, which is based solely on the latest polling data, says Democrats have a 79 percent chance to retain control of the Senate. That probability clashes with the expectations of both parties and most independent handicappers, who consider the battle for the chamber to be closer to a coin flip.

And the New York Times noted that some of Democrats’ strongest numbers are coming in the states that have seen the greatest polling misses over the past few elections.

Celinda Lake, a prominent Democratic pollster, told POLITICO that her firm, Lake Research Partners, is working hard to get the right balance of voters in its samples — but that a certain segment of Trump voters is increasingly elusive, especially as the former president’s exploits have preoccupied the headlines lately.

“It was less [of an issue] for a long time,” Lake said. “It looks to us like it is getting to be more of a problem recently, with the Mar-a-Lago thing, with his candidates winning a lot of these primaries, with the Jan. 6 committee.”

Her firm isn’t alone in working to mitigate these issues. Quinnipiac University in Connecticut, a longtime player in political polling, released polls last week showing Sen. Raphael Warnock (D-Ga.) leading GOP nominee Herschel Walker by 6 percentage points, despite other surveys showing a tied race or even a narrow Walker advantage, and Democrats with big leads in both major statewide races in Connecticut.

Doug Schwartz, director of the Quinnipiac University poll, said his interviewers have changed the way they ask respondents about their vote choice, taking care to separate those who say they are undecided from those who refuse to answer the question outright.

“In the end, we’re hoping to reduce that percentage of people that don’t give us a response on the horse-race,” Schwartz said, pointing out that while they accurately reflected Biden’s share of the vote in their 2020 pre-election polls, the large number of refusals led them to underestimate Trump’s. Quinnipiac had now-President Joe Biden leading Trump in two states Trump would carry, Florida and Ohio, on the eve of the election.

Another Northeastern academic pollster, Marist College, released two polls last week: one showing Warnock ahead of Walker by 5 points, and the other showing Ohio GOP Senate nominee J.D. Vance running neck-and-neck with Democratic Rep. Tim Ryan.

Marist also had some 2020 misses, showing Biden leading by 6 points in North Carolina (which he lost) and by 5 points in Pennsylvania (which he won by 1 point) in polls conducted for NBC News. Since 2020, Marist’s Lee Miringoff said the school has diversified its sampling by contacting voters not just via telephone calls, but also text and online interviews.

Miringoff told POLITICO he isn’t as worried about the same non-response bias — the segment of Trump voters who won’t participate in polls, systematically skewing the results toward Democrats — showing up this year. That’s because, he said, Trump himself isn’t on the ballot, and Democrats have mostly erased the GOP’s enthusiasm advantage this summer.

“I’m pretty comfortable that what may have been the case in previous elections may not be the case this time in terms of the misses,” said Miringoff.

In the last midterm election, the most prolific pollster was Siena College in Upstate New York, thanks to an ambitious, roughly $2 million “live polling” project with the New York Times to survey dozens of congressional districts. In all, the Times and Siena conducted just shy of 100 polls that accurately portrayed Democrats’ momentum in their successful bid to flip the House majority.

This year, Siena is doing swing-state polling both with and without the Times, including two new polls last week in Wisconsin and Texas conducted for Spectrum News, cable company Charter Communications’ local news outlets.

Don Levy, director of the Siena College Research Institute, said he is being “as careful as careful can be” to increase the share of Trump voters, both in their sampling (who gets called to participate) and weighting (making them count for more after the interviews are conducted in order to fix their underrepresentation). It’s not enough, Levy said, to just call more Republicans, since it’s a specific kind of Republican whom they are struggling to reach.

“It’s not partisan nonresponse. It’s hardened Trump-backer nonresponse,” said Levy. “A small majority of those are self-identified Republicans, but a significant number of them are self-identified independents or Democrats. You can’t correct that by saying, ‘Let’s weight up the Republicans.’ That doesn’t work.”

Monmouth University, in New Jersey, is trying a different tack — eschewing horse-race polling for surveys that measure each candidate’s level of support without pitting them against one another. Patrick Murray, the director of the school’s polling institute, said his analysis of the 2020 results didn’t reveal a “silver bullet” for fixing their poll, which also failed to predict the closer-than-expected New Jersey governor’s race last year.

Murray cautioned that pollsters who haven’t given up the horse-race that the dynamics of this year’s midterms are different than in the last election — and are likely to be different from the next one in two years. “If the 2022 polls are good,” he said, “it does not necessarily mean we fixed what went wrong in 2020.”

Franklin, the Wisconsin pollster, said he’s made “moderate or marginal adjustments” to the Marquette Law School poll’s methodology, including increasing the percentage of respondents contacted by cell phone. He’s also paying close attention to the response rate for his polls in Wisconsin counties that went more heavily for Trump in the last election — but thus far, voters there aren’t participating in lower numbers.

Another significant polling miss “will continue to be damaging to the reputation of polling,” said Franklin. “I think that’s just obvious and undeniable.”

It may also be inevitable. Partisan campaign pollsters in both parties suggested Trump voters are again difficult to capture in the run-up to this election.

“There is a good chance that a lot of the publicly released surveys are overstating Democratic strength,” said Glen Bolger, a Republican pollster at the firm Public Opinion Strategies.

Amanda Iovino, a Republican pollster at WPA Intelligence who worked on now-Virginia Gov. Glenn Youngkin’s race last fall, added, “It’s still easier to get college-educated voters on the phone” than voters who didn’t graduate from college.

Lake, the Democratic pollster, said she sees the measures that her colleagues are implementing to get the right mix of voters. But she isn’t sure that they will be enough to avert another 2020-style polling miss.

“I’m confident that they’re the right things,” said Lake. “I’m not confident that they’re sufficient.”

But some Democrats are daring not just to believe in the polls — but hoping that the party may actually overperform in November, pointing to two special congressional election wins last month in Alaska and New York, where polls showed Republicans ahead going into Election Day.

“You just saw the polls underestimate the victories in both Alaska and in Upstate New York,” Rep. Sean Patrick Maloney (D-N.Y.), chair of the Democratic Congressional Campaign Committee, said in an interview at a POLITICO Pro Premium Roundtable event earlier this month. “So, if anything, the polls may be showing a conservative bias right now.”