Pollsters worry Trump problem is ‘back with a vengeance’

PHILADELPHIA — Pollsters are again staring down a familiar nemesis: Donald Trump.

The polling industry whiffed every year Trump has been on the ballot. In 2016, Trump upset Hillary Clinton to win the presidency. And after spending four years trying to fix what went wrong, the polls were even worse in 2020. Trump ran far more competitively with now-President Joe Biden than the preelection surveys suggested.

Pollsters are breathing a sigh of relief after largely nailing last year's midterm elections. But presidential years have been a different story in the Trump era.

And now, with Trump expanding his lead over his GOP primary rivals, pollsters are fretting about a bloc of the electorate that has made his support nearly impossible to measure accurately.

“It’s looking a lot like Trump is going to be on the ballot” next November, said Democratic pollster Andrew Baumann. “So that is all back with a vengeance.”

It's not that Trump is some mystical force. The problems are practical. In 2020, he drew out significant numbers of people who had rarely — if ever — voted and who either weren't included in polls or refused to participate in them. Trump trashed the polls that found him consistently trailing Biden. This created a feedback loop that made his supporters even less likely to respond, making the polls even more wrong.

Baumann was among the attendees and presenters at this week's American Association for Public Opinion Research's annual conference, a yearly gathering of pollsters from the academic, media and campaign worlds.

That organization has been grappling with the future of political polling for decades. Just looking at the polls over the past two federal election cycles will give you whiplash. By most measures, last year's midterms represented the pinnacle of election polling. FiveThirtyEight's post-election evaluation found that polls were more accurate than any year since 1998.

But that came two years after the preceding presidential race, when national polls were further off than they had been in 40 years, and the state polls were the worst in recorded history.

The specter of yet another polling failure is looming over the industry's continuing efforts to overhaul its methods.

Among public pollsters, CNN made some of the most dramatic changes to its methodology. Mostly abandoning its longstanding process of random-digit phone sampling, the network and its polling vendor SSRS randomly selected street addresses for its national surveys and mailed out solicitations to complete a poll online or by dialing a number.

For much of their state polling — which requires faster turnaround times to poll statewide races like those for governor or Senate — some voters were surveyed off a file of registered voters and contacted either by email or telephone, depending on the best contact information available. Others were added from SSRS’ existing panel of respondents who said they were registered to vote.

The results were remarkable. CNN’s polls correctly identified the winner in eight of the nine major statewide races they surveyed — missing only the Nevada Senate race — and half of the candidate vote shares were accurate within a single percentage point.

“We were within the error margin on just about every [poll] we did,” said Jennifer Agiesta, the director of polling and election analytics at CNN. “So I feel pretty good about how these turned out. I would say that does give me some confidence between now and 2024.”

But Agiesta said it’s too soon to tell if the same problems that plagued pollsters in 2020 will resurface.

“I don’t think that [Trump's] comments on polling and the way that he presented his views on polling to his supporters were helpful in terms of response rate in 2020,” added Agiesta, who also began a one-year term as president of the pollsters’ organization at this week’s conference. “But I don’t know if that’s going to be the same in future elections.”

The Democratic polling firm Global Strategy Group is trying a significant methodological change on the back end. According to research the firm presented at the AAPOR conference, their 2022 polls were made more accurate by using voters’ self-reported 2020 general-election presidential vote as a variable — a practice numerous others have also adopted, though it’s still far from universal.

That, in addition to other adjustments like trying to include voters who aren’t as politically active, was an important discovery, because Global Strategy Group — like pollsters across the public and private campaign worlds — significantly underestimated Republicans in its 2020 polling.

“We think that the stuff that we’ve done to correct for that — between accounting for past vote history in the 2020 election and looking at how important politics is to a person’s identity — are going to be able to capture that and correct for these biases that really bit us in the ass in 2020,” said Baumann, who is a partner at Global Strategy Group.

The phenomenon that led a segment of Trump voters to boycott the polls is similar to other trends, including those picked up in new polling this week from YouGov, which found that Republicans trust almost all media outlets less than Democrats, with the exception of conservative media.

"If anybody wants to really be honest, it is going to be an enormous challenge if it indeed is going to be Trump [against] Biden in '24," said Don Levy, the director of the Siena College Research Institute, which conducted polling in 2022 for the New York Times and the local cable news outlet chain Spectrum News. “Because we know that that voter is disinclined to speak to us.”

But that’s only part of the problems that pollsters have identified since 2020. It’s not just that voters closely aligned with Trump are harder to reach — less-engaged voters of all stripes are less likely to participate, too.

And, the evidence suggests, the best way to reach those voters — who very well may cast ballots in high-turnout presidential elections like 2020 but are less likely to participate in midterms — remains the traditional and expensive form of phone surveys. Some pollsters have replaced or eschewed that method entirely to save money.

Not all pollsters see a sharp dividing line between 2022 and 2024. Anna Greenberg, a Democratic pollster whose 2022 clients included Sen. Mark Kelly (D-Ariz.), said that while Trump wasn’t a candidate for office last year, he was still a major issue in the midterm campaign.

“Trump was on the ballot,” said Greenberg. “Between Mar-a-Lago documents and the Jan. 6 commission, and then certainly at the statewide level, the so-called MAGA candidates: Blake Masters, [Mehmet] Oz, [Doug] Mastriano … there was so much coverage of these being his candidates.”

Overall, the mood at this week’s conference was mostly positive, riding high from 2022 — even with the prospects of another Trump-sparked miss looming in next year’s presidential election.

“I’m still worried” about 2024, said Baumann. “I don’t think that any pollster should be out there feeling superconfident that we have everything fixed, because we were confident that we did after 2016. And we didn’t.”