Nate Silver's poll-aggregation work remains very valuable, even though we are trapped by the cult of prediction: polls paralyzing politics by not predicting the future but simply chaining us to...
I think we would all be better off if journalists accurately explained the issues the candidates were pushing rather than wasting time writing about the horse race. The wasted airtime and column inches on a race could be better spent informing the viewer/reader - something journalists in the USA seem reluctant to do well. Journalists also still fail to add context to numbers which is a disservice often due, I think, to journalistic laziness.
Many thanks for posting this, Herr Doktor Professor.
I would quibble with your summary of my attitude toward Nate. I'm a little harder on him in general than you imply. Not because he is in any way dishonest - he's not; but because he and so many others think Bayesian statistical modeling is the ONLY serious way to deal with future uncertainty. When his predictions result in a lot of people getting mad, the solution is not to write another book explaining how the only problem is that all these muddle-headed "Village" liberals that dominate academia, government and the media are simply incapable of thinking in a rigorous manner, while Nate and his fellow "Riverians" are possessed of minds that burn always with this hard, gem-like Walter Pater-meets-Heisenberg flame.
In fact, if we accept that the Trump presidency, which was supported or retrospectively defended by many of those whom Silver identifies as "Riverians" (e.g. Peter Thiel, Elon Musk), was a gigantic America-damaging self-inflicted wound, one must ask whether the liberals were the ones who were muddle-headed.
I would submit, first, that conservatives and Trump supporters - including those very real, very wealthy elites that have supported him - are arguably hugely more muddle-headed and irrational than liberal elites in academia, government, and media. (After all, it is not Kamala Harris who plays The Village People at her rallies.) The average Trump voter demonstrably believes in a lot that is blatantly untrue, and is far more "coupled" in his/her views than any Democrat. And I believe this even applies to Peter Thiel, and even more so to Elon Musk.
Second, even if many liberals demonstrated a lack of understanding of the arcana of Bayesian forecasting in 2016, a fuller embrace of Bayesian forecasting was not the answer in 2020, nor is it in 2024. Better forecasting would very likely be a terrible misinvestment of time and resources by the opponents of Trump. The answer was and is to point out loudly to America that Trump was/is a criminal dictator-wannabe who is a direct threat to our basic republican rule of law, as well as a profoundly ignorant, angry man who should not be given the nuclear launch codes. Constantly recalibrating the odds of victory in a race that is utterly unpredictable and well within the margin of error is not a rational pastime for anyone (other than, perhaps, campaign flacks betting on ad spends across states).
Thirdly, and maybe most importantly, academia, government, and journalism have already gone far down the "River" Nate so loves, and the results have been terrible. Academia, aside from the hard sciences, is not a predictive discipline, because it deals with fundamentally unpredictable things. (I would love to get Brad's opinion on whether economics should be considered primarily a predictive discipline. I think a cursory examination of the BLS job forecasts as well as other forecasts of GDP, inflation, etc. etc., is sufficient to prove that if it is, it really objectively sucks at its job. IMAGINING alternative economic scenarios, positive, negative, and simply very different from today, and then explaining how they might come to pass, would be a more productive use of the very real genius of today's generation of economists, and create some serious intellectual capital.)
Government is as good as it gets in prediction - NOAA and the NWS come to mind - but what are probably the most critical functions of government deal with the fundamentally unpredictable (terrorism, international politics, war, economic growth, technological advancement). Multiple scenarios of the future are a better use of our time in these areas - or would be, if they were seriously tried. (I say this as one who wrote a scenario of a coronavirus coming out of China and destroying world trade, killing 0.2% of those infected, including hundreds of thousands of Americans, making teleconferencing a reality, and causing anti-Asian bigotry... in 2003.)
And journalism has become far more focused on numbers and the horse race, "decoupling" these estimates of electoral outcomes in exactly the way Nate admires - and I would say this has stripped our political journalism, and our politics, of any moral or even practical sense. If this presidential race is "decoupled," as it was in 2016 and even to an extent in 2020, from the objective fact that one side is a rather normal, moderate, generic politician, while the other side is openly promising to do its level worst to destroy our democracy and rule of law, then the average voter is going to think it's just another sportsball contest, a choice with no more real importance than being a Yankee fan or being a Dodger fan. This kind of "journimalism" inevitably and rather subtly elevates what used to be considered bizarre, dangerous, and unacceptable to an estate equal to what is completely ordinary and objectively somewhat successful, productive, and useful.
Nate demands we "decouple" our opinion of Donald Trump from our judgment of whether he can win this election. And that is fine.
But the real effect of his "decoupling" demand is to make people sneer at passionate supporters of one side or another - and most especially, the side that is passionately seeking to defend our government and our political system from the dangerous, the lawless, and the dictatorial. He THINKS he's just being a rigorous "Riverian," but he's legitimizing the annoyance of the tech-bros of Silicon Valley at being slightly inconvenienced in their pursuit of unbridled wealth (wealth which originates from our taxpayer-funded research and development via NASA, the Internet, etc.); and de-legitimizing the righteous anger of citizens at the hostile takeover of our government (and academia, and journalism), by amoral Riverian demi-trillionaires whose hard gemlike flames always somehow end up lighting the way toward dictatorship and a worsening standard of living for the mean American household - and more hard cash for themselves.
So, frankly, as I say in another blog piece, I see Nate's "River" as a cancer on our politics, our academics, our government, our journalism, and our society. It's an organism that has mutated and metastasized throughout our body politic, economic, cultural. It has us all seeing everything as a bet. It has a hugely growing proportion of our people losing their shirts in casinos or in online poker - or on Wall Street, crypto, or other techbro obsessions - which recent studies show has had a negative effect on our economy at large.
So - one and a half cheers? No. No cheers. Just a lot of deep concern over the effect this quantification mania is having on our entire society, and some righteous anger about this rapidly multiplying tumor of mindless calculation and lack of imagination. Until we have an AI app for rigorous use of imagination (the possibility of that's a topic for another day - and even if we get one of those), "The River" is leading us nowhere good.
If you are (a) not playing against people who know more than you do, (b) have no good way of acquiring more information, and (c) are risk-neutral because things are low-stakes, then it is an absolutely fine thing to do to focus on expected value and adopt the methodology of the Rev. Thomas Bayes. Indeed, it is hard for me to see why it would be worth your while in cognitive and other resources to do anything else.
And in that context the "Silver Bulletin" daily updates telling people that it was, is, and will probably be a toss-up until election day seem to me to be very useful.
I do have one serious quarrel with the way Nate and Eli talk about their presidential odds. They write as if small things that are going to happen in the future will be decisive for who wins—with the implication that you should pay close attention to the news in order to catch those small things that will matter a lot. I think that is more-or-less completely wrong. What is right is that small things that happened in the past of which we are now ignorant will turn out to have made the difference. So unless you have good insights into what those small things that happened and that feed into pollster biases are, you might as well hang it up and stop watching for polls.
I understand the objection if (c) does not hold: Then you should be spending your cognitive and other resources doing scenario-planning to figure out what your pain points are and how to buy insurance. Bayesian frames are then not very helpful.
I understand the objection if (b) does not hold: Then you should gather more information about the situation, because even if you do care about the odds the odds you think you know now are not the odds.
And I understand the objection if (a) does not hold: Then the only way to win (on average) i not to play, and your even trying to calculate the odds is to run a DDoS attack on your own brain.
So I would like to turn the question around: If you stipulate to (a), (b), and (c), what is the objection to the Bayesian scheme?
Hmmm... would have to think more in depth on this, but on first blush, my dumb answers would be that essentially, those three things seem to go together, in low-stakes/trivial stuff (e.g. poker, sports betting); while (a) and (b) go together for higher-stakes hard-science estimates of well-understood physical phenomena (e.g. climate change). In these cases, Bayes is the right way to go, I think.
But generally, as the stakes get higher, stuff is simply not predictable, and applying game theory approaches to fundamentally unpredictable things can get us all into very serious trouble. I keep going back to the Herman Kahn game theory approach to nuclear weapons. Maybe there really is a kind deity, because I suspect we have been incredibly lucky not to have blown it all up using that kind of kooky approach. I used to think the game theorists had kept us alive until I read Michael Dobbs' 2009 book "One Minute to Midnight" about the Cuban Missile Crisis. Everything I had crammed into my head about that event (mainly via Bobby Kennedy's "Thirteen Days" in Harvard Government classes) was complete fiction. The JFK ExComm had this very elaborate mental model of the Soviet leadership's approach to the crisis, and it was complete crap. If one Soviet sub officer had not vetoed nuking a US Navy convoy that was forcing it to the surface in the Caribbean, we might not be here discussing this. There was no "the other guy blinked" moment non the quarantine line; there was no Politburo coup against Khrushchev with the second hard-line cable; there were 80 MRBMs armed and ready to launch if attacked, with local control. We were incredibly lucky, not good at game theory.
I think we try to apply mathematically rigorous game-theory-type approaches to way too many intersubjective realities (of Keynes' "beauty pageant" type). And as long as the mental models are shared widely, they work. But as soon as mass opinion shifts, they go kablooey. Because the laws of physics do not apply to mass delusions (e.g. religion, patriotism, fashion, the stock market, belief that home prices in the U.S. never decline across the entire country), they can abruptly shift in a way physical materials, poker games, and even baseball games cannot.
Games are designed to deliver dopamine hits in small increments, with reasonably low stakes, the exact way cigarettes deliver nicotine hits. Real fundamental uncertainty doesn't do that, but Nate Silver implicitly seems to promise his readers that it can. What we really need for the high-stakes stuff is not more Bayes, but a more systematic and rigorous approach to imagining the full range of plausible large scale outcomes, and then use math and logic to explain how those outcomes might happen. It's a bit terrifying, because you cannot go through that process and remain unaware of just how vast the scale of uncertainty (and the universe of truly plausible outcomes) really is.
I'm about to write a few scenarios of this election from about four years in the future. I think they will correspond to your "small things that happened in the past of which we are now ignorant will turn out to have made the difference" hypothesis. (The monster early voting turnout in Georgia is of this type of thing.) I never like to say things are predictable, but if she loses, the "She should have chosen Josh Shapiro" garbage that would follow is kind of a lock. As well as the "If only Trump had moderated in the last few weeks he would have won" garbage that would follow him losing. No one knows, as you say - abortion might already have won the election for Kamala; her 2019 trans-friendly statement might have already lost it for her.
It's a scary world, and nobody knows nuttin.' That's why we like to pretend there is talismanic power in poll aggregation. Gimme another nicotine hit...
I have done as threatened and written two scenarios for what the world looks like if either Trump or Kamala wins. If you are not already exhausted by my logorrhea, they are here (Trump)...
I'm just a retired English professor of very little brain, & I've stayed outside Silver's paywall because I'm not interested in betting on sports. On politics, I've understood him since in one presidential election he was offering 8-1 odds & someone on TV insisted it was a tossup: Silver offered to take those odds for whatever amount the fool wanted.
Thinking about probabilities is hard, and it's amazing how poorly we do it. For example, if a medical test is 95% accurate & the incidence of a disease is 5%, what are the odds a positive test is false? Ask you doc & be appalled.
I think we would all be better off if journalists accurately explained the issues the candidates were pushing rather than wasting time writing about the horse race. The wasted airtime and column inches on a race could be better spent informing the viewer/reader - something journalists in the USA seem reluctant to do well. Journalists also still fail to add context to numbers which is a disservice often due, I think, to journalistic laziness.
Many thanks for posting this, Herr Doktor Professor.
I would quibble with your summary of my attitude toward Nate. I'm a little harder on him in general than you imply. Not because he is in any way dishonest - he's not; but because he and so many others think Bayesian statistical modeling is the ONLY serious way to deal with future uncertainty. When his predictions result in a lot of people getting mad, the solution is not to write another book explaining how the only problem is that all these muddle-headed "Village" liberals that dominate academia, government and the media are simply incapable of thinking in a rigorous manner, while Nate and his fellow "Riverians" are possessed of minds that burn always with this hard, gem-like Walter Pater-meets-Heisenberg flame.
In fact, if we accept that the Trump presidency, which was supported or retrospectively defended by many of those whom Silver identifies as "Riverians" (e.g. Peter Thiel, Elon Musk), was a gigantic America-damaging self-inflicted wound, one must ask whether the liberals were the ones who were muddle-headed.
I would submit, first, that conservatives and Trump supporters - including those very real, very wealthy elites that have supported him - are arguably hugely more muddle-headed and irrational than liberal elites in academia, government, and media. (After all, it is not Kamala Harris who plays The Village People at her rallies.) The average Trump voter demonstrably believes in a lot that is blatantly untrue, and is far more "coupled" in his/her views than any Democrat. And I believe this even applies to Peter Thiel, and even more so to Elon Musk.
Second, even if many liberals demonstrated a lack of understanding of the arcana of Bayesian forecasting in 2016, a fuller embrace of Bayesian forecasting was not the answer in 2020, nor is it in 2024. Better forecasting would very likely be a terrible misinvestment of time and resources by the opponents of Trump. The answer was and is to point out loudly to America that Trump was/is a criminal dictator-wannabe who is a direct threat to our basic republican rule of law, as well as a profoundly ignorant, angry man who should not be given the nuclear launch codes. Constantly recalibrating the odds of victory in a race that is utterly unpredictable and well within the margin of error is not a rational pastime for anyone (other than, perhaps, campaign flacks betting on ad spends across states).
Thirdly, and maybe most importantly, academia, government, and journalism have already gone far down the "River" Nate so loves, and the results have been terrible. Academia, aside from the hard sciences, is not a predictive discipline, because it deals with fundamentally unpredictable things. (I would love to get Brad's opinion on whether economics should be considered primarily a predictive discipline. I think a cursory examination of the BLS job forecasts as well as other forecasts of GDP, inflation, etc. etc., is sufficient to prove that if it is, it really objectively sucks at its job. IMAGINING alternative economic scenarios, positive, negative, and simply very different from today, and then explaining how they might come to pass, would be a more productive use of the very real genius of today's generation of economists, and create some serious intellectual capital.)
Government is as good as it gets in prediction - NOAA and the NWS come to mind - but what are probably the most critical functions of government deal with the fundamentally unpredictable (terrorism, international politics, war, economic growth, technological advancement). Multiple scenarios of the future are a better use of our time in these areas - or would be, if they were seriously tried. (I say this as one who wrote a scenario of a coronavirus coming out of China and destroying world trade, killing 0.2% of those infected, including hundreds of thousands of Americans, making teleconferencing a reality, and causing anti-Asian bigotry... in 2003.)
And journalism has become far more focused on numbers and the horse race, "decoupling" these estimates of electoral outcomes in exactly the way Nate admires - and I would say this has stripped our political journalism, and our politics, of any moral or even practical sense. If this presidential race is "decoupled," as it was in 2016 and even to an extent in 2020, from the objective fact that one side is a rather normal, moderate, generic politician, while the other side is openly promising to do its level worst to destroy our democracy and rule of law, then the average voter is going to think it's just another sportsball contest, a choice with no more real importance than being a Yankee fan or being a Dodger fan. This kind of "journimalism" inevitably and rather subtly elevates what used to be considered bizarre, dangerous, and unacceptable to an estate equal to what is completely ordinary and objectively somewhat successful, productive, and useful.
Nate demands we "decouple" our opinion of Donald Trump from our judgment of whether he can win this election. And that is fine.
But the real effect of his "decoupling" demand is to make people sneer at passionate supporters of one side or another - and most especially, the side that is passionately seeking to defend our government and our political system from the dangerous, the lawless, and the dictatorial. He THINKS he's just being a rigorous "Riverian," but he's legitimizing the annoyance of the tech-bros of Silicon Valley at being slightly inconvenienced in their pursuit of unbridled wealth (wealth which originates from our taxpayer-funded research and development via NASA, the Internet, etc.); and de-legitimizing the righteous anger of citizens at the hostile takeover of our government (and academia, and journalism), by amoral Riverian demi-trillionaires whose hard gemlike flames always somehow end up lighting the way toward dictatorship and a worsening standard of living for the mean American household - and more hard cash for themselves.
So, frankly, as I say in another blog piece, I see Nate's "River" as a cancer on our politics, our academics, our government, our journalism, and our society. It's an organism that has mutated and metastasized throughout our body politic, economic, cultural. It has us all seeing everything as a bet. It has a hugely growing proportion of our people losing their shirts in casinos or in online poker - or on Wall Street, crypto, or other techbro obsessions - which recent studies show has had a negative effect on our economy at large.
So - one and a half cheers? No. No cheers. Just a lot of deep concern over the effect this quantification mania is having on our entire society, and some righteous anger about this rapidly multiplying tumor of mindless calculation and lack of imagination. Until we have an AI app for rigorous use of imagination (the possibility of that's a topic for another day - and even if we get one of those), "The River" is leading us nowhere good.
Not that I feel strongly about this...
If you are (a) not playing against people who know more than you do, (b) have no good way of acquiring more information, and (c) are risk-neutral because things are low-stakes, then it is an absolutely fine thing to do to focus on expected value and adopt the methodology of the Rev. Thomas Bayes. Indeed, it is hard for me to see why it would be worth your while in cognitive and other resources to do anything else.
And in that context the "Silver Bulletin" daily updates telling people that it was, is, and will probably be a toss-up until election day seem to me to be very useful.
I do have one serious quarrel with the way Nate and Eli talk about their presidential odds. They write as if small things that are going to happen in the future will be decisive for who wins—with the implication that you should pay close attention to the news in order to catch those small things that will matter a lot. I think that is more-or-less completely wrong. What is right is that small things that happened in the past of which we are now ignorant will turn out to have made the difference. So unless you have good insights into what those small things that happened and that feed into pollster biases are, you might as well hang it up and stop watching for polls.
I understand the objection if (c) does not hold: Then you should be spending your cognitive and other resources doing scenario-planning to figure out what your pain points are and how to buy insurance. Bayesian frames are then not very helpful.
I understand the objection if (b) does not hold: Then you should gather more information about the situation, because even if you do care about the odds the odds you think you know now are not the odds.
And I understand the objection if (a) does not hold: Then the only way to win (on average) i not to play, and your even trying to calculate the odds is to run a DDoS attack on your own brain.
So I would like to turn the question around: If you stipulate to (a), (b), and (c), what is the objection to the Bayesian scheme?
Hmmm... would have to think more in depth on this, but on first blush, my dumb answers would be that essentially, those three things seem to go together, in low-stakes/trivial stuff (e.g. poker, sports betting); while (a) and (b) go together for higher-stakes hard-science estimates of well-understood physical phenomena (e.g. climate change). In these cases, Bayes is the right way to go, I think.
But generally, as the stakes get higher, stuff is simply not predictable, and applying game theory approaches to fundamentally unpredictable things can get us all into very serious trouble. I keep going back to the Herman Kahn game theory approach to nuclear weapons. Maybe there really is a kind deity, because I suspect we have been incredibly lucky not to have blown it all up using that kind of kooky approach. I used to think the game theorists had kept us alive until I read Michael Dobbs' 2009 book "One Minute to Midnight" about the Cuban Missile Crisis. Everything I had crammed into my head about that event (mainly via Bobby Kennedy's "Thirteen Days" in Harvard Government classes) was complete fiction. The JFK ExComm had this very elaborate mental model of the Soviet leadership's approach to the crisis, and it was complete crap. If one Soviet sub officer had not vetoed nuking a US Navy convoy that was forcing it to the surface in the Caribbean, we might not be here discussing this. There was no "the other guy blinked" moment non the quarantine line; there was no Politburo coup against Khrushchev with the second hard-line cable; there were 80 MRBMs armed and ready to launch if attacked, with local control. We were incredibly lucky, not good at game theory.
I think we try to apply mathematically rigorous game-theory-type approaches to way too many intersubjective realities (of Keynes' "beauty pageant" type). And as long as the mental models are shared widely, they work. But as soon as mass opinion shifts, they go kablooey. Because the laws of physics do not apply to mass delusions (e.g. religion, patriotism, fashion, the stock market, belief that home prices in the U.S. never decline across the entire country), they can abruptly shift in a way physical materials, poker games, and even baseball games cannot.
Games are designed to deliver dopamine hits in small increments, with reasonably low stakes, the exact way cigarettes deliver nicotine hits. Real fundamental uncertainty doesn't do that, but Nate Silver implicitly seems to promise his readers that it can. What we really need for the high-stakes stuff is not more Bayes, but a more systematic and rigorous approach to imagining the full range of plausible large scale outcomes, and then use math and logic to explain how those outcomes might happen. It's a bit terrifying, because you cannot go through that process and remain unaware of just how vast the scale of uncertainty (and the universe of truly plausible outcomes) really is.
I'm about to write a few scenarios of this election from about four years in the future. I think they will correspond to your "small things that happened in the past of which we are now ignorant will turn out to have made the difference" hypothesis. (The monster early voting turnout in Georgia is of this type of thing.) I never like to say things are predictable, but if she loses, the "She should have chosen Josh Shapiro" garbage that would follow is kind of a lock. As well as the "If only Trump had moderated in the last few weeks he would have won" garbage that would follow him losing. No one knows, as you say - abortion might already have won the election for Kamala; her 2019 trans-friendly statement might have already lost it for her.
It's a scary world, and nobody knows nuttin.' That's why we like to pretend there is talismanic power in poll aggregation. Gimme another nicotine hit...
I have done as threatened and written two scenarios for what the world looks like if either Trump or Kamala wins. If you are not already exhausted by my logorrhea, they are here (Trump)...
https://patrickmarren.com/2024/10/2024-presidential-election-scenario-1-trump-victory
and here (Kamala).
https://patrickmarren.com/2024/10/presidential-election-scenario-2-harris-victory
(Spoiler alert: Limited upside in either case, but almost unlimited downside in the former.)
Going to listen to Hexapodia now. I read that whole Vinge trilogy thanks to you guys. Maybe we can discuss at a reunion some fine day.
I'm just a retired English professor of very little brain, & I've stayed outside Silver's paywall because I'm not interested in betting on sports. On politics, I've understood him since in one presidential election he was offering 8-1 odds & someone on TV insisted it was a tossup: Silver offered to take those odds for whatever amount the fool wanted.
Thinking about probabilities is hard, and it's amazing how poorly we do it. For example, if a medical test is 95% accurate & the incidence of a disease is 5%, what are the odds a positive test is false? Ask you doc & be appalled.