Nobody Knows Anything, 2023 Polling Edition

I haven’t been paying much attention to the 2023 election cycle, other than getting annoyed at all the silly polls for next year. The talking heads ran with that exciting polling data showing Trump mopping the floor with Biden in swing states 12 months in advance of the actual election.

Hence, their surprise by the electoral results yesterday.1

Historical data portending to show the outcome a year out — like the recent 2024 POTUS polls — have a poor track record.

Here is Pew in 2007:

“It’s not just the case that polling in the nominating contests is perilous. Polls that test hypothetical general election matchups at this stage in the cycle are mostly wrong about who will win the White House. Early polling does provide a benchmark for charting trends in voter sentiment, but it probably won’t be very predictive of the eventual outcome in 2008.”

I spend (too much) time reviewing research that relies on polls and surveys; for the most part, they are as bad as Wall Street forecasting. The difference is they have the imprimatur of professional statistical analysis.

They shouldn’t.

Polls within the final week of an election tend to be about 60% accurate, e.g.,60% chance of the result falling within the margin of error. More than 10 weeks out, it is a coin toss — about a 50% accuracy rate. Research shows polls 300+ days before an election have no predictive value – they are no better than random guessing.

Here is my firm belief:

If you discuss a Presidential Poll on TV a year out, and fail to mention that its predictive value is zero, you are committing Journalistic Malpractice.

Debates about media bias tend to get framed in a left/right context; it would be accurate and useful to frame media bias in terms of sensationalism and the affinity for a lazy clickbait approach. The media knows how to do horse races, but complex policy analysis and nuance simply fail to generate views.

I complain about how bad the media covers fund managers and stocks; I guess I shouldn’t be surprised that media coverage of an election that won’t happen until 1 year from today is just as terrible.

See also:How Reliable Are the Early Presidential Polls? (Pew, February 14, 2007)

Pollsters: ‘Impossible’ to say why 2020 polls were wrong.(Politico, 07/18/2021)

What 2020’s Election Poll Errors Tell Us About the Accuracy of Issue Polling. (March 2, 2021)

Election polls are 95% confident but only 60% accurate, Berkeley Haas study finds (Haas Berkeley, October 26, 2020)

Previously:Nobody Knows Nuthin’ (May 5, 2016)

Nobody Knows Anything, John Wick edition (September 6, 2023)

Nobody Knows Anything, Calvin & Hobbes Edition (August 14, 2023)

Nobody Knows Anything (Collection)

___________

X. I thought it was obvious since Roe v Wade was overturned that those personal rights were going to be a big deal, but what do I know…