CROSSPOST: Patrick Marren on Nate Silver: One & a Half Cheers, Perhaps?
Nate Silver's poll-aggregation work remains very valuable, even though we are trapped by the cult of prediction: polls paralyzing politics by not predicting the future but simply chaining us to...
Nate Silver's poll-aggregation work remains very valuable, even though we are trapped by the cult of prediction: polls paralyzing politics by not predicting the future but simply chaining us to uncertainty…
Yet one more intelligent word on Nate Silver, from the extremely sharp Patrick Marren. The big problem Patrick sees is that most of Silver’s audience DEMANDS certainty that the Rev. Thomas Bayes tells us that we can never have. And a secondary problem is that most of the time we do not care much about the odds—we care about scenario planning, about how best to learn more about the situation, and about what to do when there are other players in the game with interests not aligned with ours and who know more than we do. The frame of mind that comes out of gambling—where we do care about the odds, where there is little we can do to gain more information, where there is nothing to do in response to learning how the dice have fallen, and where the alignment of other players and much about their information sets are known—is not very helpful in figuring out how to navigate in the world…
Patrick Marren: ‘See my Summa Contra Argentum series on my blog, Brad; I was thinking of you as I wrote it. <https://patrickmarren.com/2024/09/summa-contra-argentum-librum-primum>. While I agree that Nate is as good as it gets with this poll-aggregating mishegoss, I lament the svengali-like hold polling approaches have on even the smartest people around. I have even written an entire book on the topic (Fatal Certainty), which is going unpublished so far, because Nate is selling pure crack cocaine, while I am vending spinach, I guess (spinach can be very appetizing if prepared correctly!). We know it's close. We are pretty sure it WILL be close till the end. That's all we need to know. But no one wants to discuss plausible outcomes from each of these two winning; everyone is fixated on the polls, the polls, the polls. Help me get the word out about the Cult of Prediction, Brad... the sphere of things that can be (a) reliably, and (ii) usefully predicted is pretty small, and pretty unstrategic… <https://braddelong.substack.com/p/the-deceptive-poll-punditry-of-statistical/comment/72215611>
SCA 7: A Steelman Defense of Nate Silver
A lot of what I have said in the past few weeks may give you the idea that I think Nate Silver is a fraud, or at best a non-serious person. Neither of those things is true.
I recently have had, let us say, some of my fellow-traveling partisans tell me they find Nate to be those things, and I have had to defend him.
Nate Silver is a very smart and honest guy. A very impressive guy, really.
Two recent statements of friends I normally agree with have proved Nate’s “coupling” hypothesis about “Village liberals” sort of correct:
“[…H]e’s no longer a reliable pollster and is no longer at 538. He does non-professional-pollsterish stuff like discounting convention bounces qualitatively, according to people who know about his methodology. His polls are no longer included in legit aggregated polls….”
“I’ve seen him interviewed about this— a serious pollster would explain where things went wrong instead of taking up a side gig about gambling. Maybe he’ll make more money with this book but to me, this just makes me take him less seriously.”
Several points here:
First of all, Silver is not a “pollster” at all. He is an aggregator of the polling of others. He has created a model that (among other elements, such as demographics) incorporates the measured accuracy of all the polls he uses, weighting them according to their previous validity (or lack thereof). (In addition, he created 538, then left it in 2023, ten years after selling it, presumably for a large amount, to ABC/ESPN.)
Second, Nate’s “qualitative” treatment of things like “convention bounces” (which, again to be fair, is not really represented entirely accurately here) is part of the whole Bayesian approach. With Bayes, you start with certain prior assumptions, and you alter them as events come in. There’s a qualitative element to that to start with – I think in this case, he openly stated that he was not including the normal Democratic Convention bounce in his model, ironically because it tended to make Kamala Harris’ odds look worse (the bounce was far smaller than previous ones, because the electorate is so polarized that little movement can be expected from even a very successful convention anymore).
Third, his life course is pretty much the opposite of what the second commenter was saying. He started off doing baseball statistical analysis, as I recall, and (following Bill James) created a lot of the field of “Sabermetrics,” modeling likely baseball player performance. Then he got into on-line poker, made a bunch of money initially, and then, as he said in The Signal and the Noise, lost it, and got into modeling political races (again, NOT polling—he rates other people’s polls and incorporates them into his predictive models).
Fourth, as a predictor, he is about as good as it gets. Who else had Trump with anything like as high as a 29% chance in 2016? Almost no one, certainly no one seen as serious. 29% chances come in all the time.
So Nate Silver did NOT fail in 2016; prediction itself failed. It was useless and paralyzing.
I think the fact that he is now back into playing poker—and winning at it, ranked in the top 100 in the world recently—makes him, if anything, more impressive. He’s still handicapping the presidential and congressional races, though you have to pay to see what his real latest odds are.
I think he’s been “explaining where things went wrong” for eight years now, and he’s TECHNICALLY right that the failure was overwhelmingly on the part of those consuming his polls.
HIS failure was that he thought he was in some Bayesian statistical salon discussing the arcana of probability theory with well-educated peers who were well aware that the essence of probabilistic forecasting is UNcertainty.
Instead, he was, almost certainly unwittingly, chumming the waters with hope for non-mathematicians, purveying an addictive drug to voracious certainty-addicts.
Once we knew that Trump COULD win back in 2016, we all should have logged off of Nate’s models and started planning for the apocalypse (either way—the one thing both sides agreed upon was that the election of the other candidate might be the end of the world).
We didn’t, of course… so Nate got a lot of unexpected bitterness in response.
P cubed: Polls Paralyze People. <https://patrickmarren.com/2024/09/sca-7-a-steelman-defense-of-nate-silver>
If reading this gets you Value Above Replacement, then become a free subscriber to this newsletter. And forward it! And if your VAR from this newsletter is in the three digits or more each year, please become a paid subscriber! I am trying to make you readers—and myself—smarter. Please tell me if I succeed, or how I fail...
I think we would all be better off if journalists accurately explained the issues the candidates were pushing rather than wasting time writing about the horse race. The wasted airtime and column inches on a race could be better spent informing the viewer/reader - something journalists in the USA seem reluctant to do well. Journalists also still fail to add context to numbers which is a disservice often due, I think, to journalistic laziness.
Many thanks for posting this, Herr Doktor Professor.
I would quibble with your summary of my attitude toward Nate. I'm a little harder on him in general than you imply. Not because he is in any way dishonest - he's not; but because he and so many others think Bayesian statistical modeling is the ONLY serious way to deal with future uncertainty. When his predictions result in a lot of people getting mad, the solution is not to write another book explaining how the only problem is that all these muddle-headed "Village" liberals that dominate academia, government and the media are simply incapable of thinking in a rigorous manner, while Nate and his fellow "Riverians" are possessed of minds that burn always with this hard, gem-like Walter Pater-meets-Heisenberg flame.
In fact, if we accept that the Trump presidency, which was supported or retrospectively defended by many of those whom Silver identifies as "Riverians" (e.g. Peter Thiel, Elon Musk), was a gigantic America-damaging self-inflicted wound, one must ask whether the liberals were the ones who were muddle-headed.
I would submit, first, that conservatives and Trump supporters - including those very real, very wealthy elites that have supported him - are arguably hugely more muddle-headed and irrational than liberal elites in academia, government, and media. (After all, it is not Kamala Harris who plays The Village People at her rallies.) The average Trump voter demonstrably believes in a lot that is blatantly untrue, and is far more "coupled" in his/her views than any Democrat. And I believe this even applies to Peter Thiel, and even more so to Elon Musk.
Second, even if many liberals demonstrated a lack of understanding of the arcana of Bayesian forecasting in 2016, a fuller embrace of Bayesian forecasting was not the answer in 2020, nor is it in 2024. Better forecasting would very likely be a terrible misinvestment of time and resources by the opponents of Trump. The answer was and is to point out loudly to America that Trump was/is a criminal dictator-wannabe who is a direct threat to our basic republican rule of law, as well as a profoundly ignorant, angry man who should not be given the nuclear launch codes. Constantly recalibrating the odds of victory in a race that is utterly unpredictable and well within the margin of error is not a rational pastime for anyone (other than, perhaps, campaign flacks betting on ad spends across states).
Thirdly, and maybe most importantly, academia, government, and journalism have already gone far down the "River" Nate so loves, and the results have been terrible. Academia, aside from the hard sciences, is not a predictive discipline, because it deals with fundamentally unpredictable things. (I would love to get Brad's opinion on whether economics should be considered primarily a predictive discipline. I think a cursory examination of the BLS job forecasts as well as other forecasts of GDP, inflation, etc. etc., is sufficient to prove that if it is, it really objectively sucks at its job. IMAGINING alternative economic scenarios, positive, negative, and simply very different from today, and then explaining how they might come to pass, would be a more productive use of the very real genius of today's generation of economists, and create some serious intellectual capital.)
Government is as good as it gets in prediction - NOAA and the NWS come to mind - but what are probably the most critical functions of government deal with the fundamentally unpredictable (terrorism, international politics, war, economic growth, technological advancement). Multiple scenarios of the future are a better use of our time in these areas - or would be, if they were seriously tried. (I say this as one who wrote a scenario of a coronavirus coming out of China and destroying world trade, killing 0.2% of those infected, including hundreds of thousands of Americans, making teleconferencing a reality, and causing anti-Asian bigotry... in 2003.)
And journalism has become far more focused on numbers and the horse race, "decoupling" these estimates of electoral outcomes in exactly the way Nate admires - and I would say this has stripped our political journalism, and our politics, of any moral or even practical sense. If this presidential race is "decoupled," as it was in 2016 and even to an extent in 2020, from the objective fact that one side is a rather normal, moderate, generic politician, while the other side is openly promising to do its level worst to destroy our democracy and rule of law, then the average voter is going to think it's just another sportsball contest, a choice with no more real importance than being a Yankee fan or being a Dodger fan. This kind of "journimalism" inevitably and rather subtly elevates what used to be considered bizarre, dangerous, and unacceptable to an estate equal to what is completely ordinary and objectively somewhat successful, productive, and useful.
Nate demands we "decouple" our opinion of Donald Trump from our judgment of whether he can win this election. And that is fine.
But the real effect of his "decoupling" demand is to make people sneer at passionate supporters of one side or another - and most especially, the side that is passionately seeking to defend our government and our political system from the dangerous, the lawless, and the dictatorial. He THINKS he's just being a rigorous "Riverian," but he's legitimizing the annoyance of the tech-bros of Silicon Valley at being slightly inconvenienced in their pursuit of unbridled wealth (wealth which originates from our taxpayer-funded research and development via NASA, the Internet, etc.); and de-legitimizing the righteous anger of citizens at the hostile takeover of our government (and academia, and journalism), by amoral Riverian demi-trillionaires whose hard gemlike flames always somehow end up lighting the way toward dictatorship and a worsening standard of living for the mean American household - and more hard cash for themselves.
So, frankly, as I say in another blog piece, I see Nate's "River" as a cancer on our politics, our academics, our government, our journalism, and our society. It's an organism that has mutated and metastasized throughout our body politic, economic, cultural. It has us all seeing everything as a bet. It has a hugely growing proportion of our people losing their shirts in casinos or in online poker - or on Wall Street, crypto, or other techbro obsessions - which recent studies show has had a negative effect on our economy at large.
So - one and a half cheers? No. No cheers. Just a lot of deep concern over the effect this quantification mania is having on our entire society, and some righteous anger about this rapidly multiplying tumor of mindless calculation and lack of imagination. Until we have an AI app for rigorous use of imagination (the possibility of that's a topic for another day - and even if we get one of those), "The River" is leading us nowhere good.
Not that I feel strongly about this...