Juggling for High Stakes, Nate Silver Expertly Keeps All the Balls in the Air
Intelligent thoughts from Nate Silver on the riddle wrapped inside a mystery inside an enigma of the 2024 presidential-election polls; Selzer vs. Siena vs. the herd: who is...
Intelligent thoughts from Nate Silver on the riddle wrapped inside a mystery inside an enigma of the 2024 presidential-election polls; Selzer vs. Siena vs. the herd: who is right?; guessing at the reasons behind all of the polling anomalies—statistical noise and model errors, and if so what kind, or signal? & a signal of what?…
Tonight Nate Silver brings the refreshments BIGTIME, apropos of this on Iowa from Selzer & Co.:
He expertly uses his skills not just as a data analyst but as an expert poker player to attempt to understand what is going on with the polls of next Tuesday’s presidential election, why they are saying what their originators say they are saying, and what it means for what is likely to happen next Tuesday—and after. There are three balls in the air here: the polls, what they say about the likely electorate, and what they say about the pollsters. Nate juggles them expertly. So, once again, my advice: read Nate, and then don’t bother to read much of anybody else. Unless they are bringing brand-new information about turnout, the demographic shape of the electorate, or the competence of get-out-the-vote efforts, they add not signal but noise.
Nate Silver:
Nate Silver: A shocking Iowa poll means somebody is going to be wrong: ‘Either Ann Selzer and the New York Times, or the rest of the polling industry…. Selzer — like NYT/Siena — has a long history of bucking the conventional wisdom and being right…. Selzer’s new poll… shows… Harris leading in Iowa 47-44.
Releasing this poll took an incredible amount of guts because… Selzer will probably be wrong. Harris’s chances of… Iowa… in our model… 17 percent…. [Selzer has] 808 likely voters…. In theory, in 95 out of 100 cases, the “real” number should be somewhere between Trump +3.4 and Harris +9.6…. [Other] sources of polling error — especially… that the overwhelming majority of people who pollsters attempt to contact never complete the survey. And… [as] good Bayesians… even… [with] a very good night… winning a state you lost last time by 8 points is a big ask…
Outside of the Midwestern swing states that get surveyed a lot — but which are also therefore more subject to herding — Harris has had some bright spots in Midwestern polls. Ohio… Harris down by only 3%-]points…. [In] Nebraska’s 2nd Congressional District… Harris has been doing great… ahead by an average of 9.6 points, exceeding Biden’s 6.6-point margin from 2020…. Even… a poll… showing a relatively tight race in Kansas….
[Perhaps] pollsters are herding very heavily in high-profile Wisconsin, Michigan, and Pennsylvania but then showing their true colors in the more obscure Midwestern states…. The most important update you should probably make from the Selzer poll is that Harris might overperform her polls in the Blue Wall….
You… know I’ve become increasingly concerned about the possibility of another systematic polling error — although I don’t know whether it will favor Trump, [or] favor Harris….
A lot of things are hard to reconcile in the polls… likely voter models help[ing] Harris in national… but hurt[ing] her in… swing states… high-quality pollsters like Selzer and NYT/Siena… so different than the others… demographic shifts… extremely pronounced in the crosstabs… few polls in states like Wisconsin — less than 10 percent! — dare to show anything outside the range of Harris +2 to Trump +2, even though this is incredibly statistically unlikely… a palpable gap between Harris and Trump’s favorability ratings, but she doesn’t consistently lead him in the head-to-heads… national polls have swung quite a bit toward Trump over the past several weeks but battleground state polls have moved only slightly….
This does not necessarily mean that the polls are underrating Harris. It could be… that pollsters are terrified to publish, say, Trump +5 numbers in Arizona or Georgia or Nevada… <natesilver.net/p/a-shoc…>
Mind you, Silver is not certain his interpretation of what is going on is right—it is just a theory—it is something he is thinking about, not something he is (yet) willing to lay odds on—but it is not implausible.
Nate Silver is focusing on the divergent conclusions from Selzer & Co., Siena, and the pollster herd. He is trying to keep three balls in the air—the polls themselves, the shifting electorate, and pollsters’ methodologies, fears, and biases. Are Selzer’s data showing a surprising lead for Harris in Iowa statistical and model-error noise, or signal? And if a signal rather than Selzer’s own model error, is it a signal of post-Dobbs electoral reshuffle coupled with too-cautious approaches by pollsters unwilling to publish results that stray from the expected norm? And how in this context to understand that Siena poll outlier giving Trump an extra 3%-point lead in Arizona?
To spell it out at greater length, the theory that Silver is playing with and thinking about is that:
pollsters are frantically putting their thumbs on the scale to keep swing-state polls essentially tied out of a fear that any other call will have bad consequences for them in the post-Tuesday world;
are thus herding to a remarkable degree and generating a statistically impossible extremely tightly grouped ensemble of tied races in all the swing states;
but that is not what rational expectations of the likely outcome of Tuesday are as of now;
herding by pollsters putting their thumbs on the scale for a tossup means that the ensemble of polls on which the Silver Bulletin relies are uninformative about what is likely to happen.
What is the likely outcome?
Well, Selzer has Iowa somewhere between Trump by 3.4%-points and Harris by 9.6%-points. That goes with Harris having an average lead of 5.5%-points in the Blue Wall states of Wisconsin, Michigan, and Pennsylvania that are thought to be 6%-points more Democratic than Iowa (compared to the current polling average lead of Harris in the three of 0.6% points).
On the other hand, Siena has Gallego leading Lake by 7%-points while Harris trails Trump by 5%-points in Arizona. But Pennsylvania, Michigan, and Wisconsin are all thought to be less than 3.5%-points more Democratic than Arizona.
If both Selzer on Iowa and Siena on Arizona are correct, then the most likely scenario is that there has been a very large resorting of the American electorate since 2016 and 2020—a resorting that it would be tempting to attribute to Dobbs and immigration that leaves Harris with much better than 50-50 odds.
If Selzer is right on Iowa and if Siena on Arizona has a combination of sampling error and some wrong modeling decisions, then it is very hard indeed to see a Trump win come Tuesday.
If Siena is right on Arizona and it is Selzer who has a combination of sampling error and some wrong modeling decisions, then it is similarly difficult to see a Harris won come Tuesday.
Or they could both have some combination of sampling error and wrong modeling decisions—in which case it could really be a tossup.
The Silver Bulletin forecast: <https://www.natesilver.net/p/nate-silver-2024-president-election-polls-model>
If reading this gets you Value Above Replacement, then become a free subscriber to this newsletter. And forward it! And if your VAR from this newsletter is in the three digits or more each year, please become a paid subscriber! I am trying to make you readers—and myself—smarter. Please tell me if I succeed, or how I fail…
It's not a common form of visualization, but a graph showing the Harris win probability (and/or the width of our posteriors as of current information) as a function of different combinations of pollster informational characteristics would be interesting. At least in this sort of analysis our inference about voting outcomes "goes thru" (in the Bayesian network sense) assumptions about pollster informational characteristics, so that's the set of relevant priors to study sensibility to.
(I think this generalizes to the problem of getting information online on a topic you aren't well-informed on; rather than evaluating the probability of statements, you should focus on evaluating the reliability of experts and sources, although it's usually not as quantitative as in the mixture-of-mixtures-of-polls case.)
"Siena has Gallego leading Lake by 7%-points while Harris trails Trump by 5%-points in Arizona."
I really have trouble believing that Gallego will beat Harris' margin by _twelve points_. Five or six, sure. But something is wonky here. There aren't that many ticket splitters anymore.