If your large language model reminds you of a brain, it’s because you’re projecting—not because it’s thinking. It’s not reasoning, it’s interpolation And anthropomorphizing the algorithm doesn’t...
I tried the same on Gemini 2.5 Pro, which responded that the edition in question is not available for free reading or download on archive.org. Which is correct. It suggested other editions, or where to find the requested edition for sale. Excerpt:
Unfortunately, based on the search results, the specific 2008 Oxford University Press edition of Machiavelli's The Prince, translated and edited by Peter Bondanella, does not appear to be available for free reading or download on archive.org.
Here's what the search found:
Other Editions on Archive.org: There are other editions of The Prince available on archive.org, such as one translated by W.K.[1] Marriott and another translated by Luigi Ricci. These are not the specific Bondanella/Oxford edition you requested.
Bondanella's Work on Archive.org: Archive.org does have Machiavelli's Discourses on Livy translated by Julia and Peter Bondanella (published by Oxford University Press), but it is listed as "access-restricted" and requires borrowing.
Confirmation of the Edition: Several results confirm the existence and details of the 2008 Oxford World's Classics edition translated by Peter Bondanella (ISBN: 9780199535699), often with an introduction by Maurizio Viroli. These results point to booksellers like Amazon, Blackwell's, and publisher pages like Oxford University Press itself, not archive.org.
Well yes, any well educated university professor can confuse or fool ChatGPT. But for ordinary laymen it's pretty amazing. I recently asked ChatGPT if it thought it had passed the Turing Test. It hedged, saying yes, in some respects, but no in others.
Another time I asked it a question about Larry Niven's "Ringworld." I've always suspected that the artists who paint the covers for Ringworld novels don't get the perspective right because most artists are not good at math. But ChatGPT is very good at math. So I asked what a person on the inner surface of Ringworld would see if looking up at the ring at night. ChatGPT showed me all the calculations and told me what a person would see. Pretty impressive for a "Kernel-Smoother Function."
My first thought was that it is only night on Ringworld if you are on the outside away from the sun, so you wouldn't see anything but your local area of the ring illuminated by starlight. Then, I vaguely remembered that there was a system of blinds to provide day and night in its own orbit closer to the sun. Trying to visualize that kind of thing is tough. Edgar Rice Burroughs had a sun inside a planet that provided daylight all the time, and I remember thinking he had gotten some of the details wrong, aside from having a star inside a planet.
Book cover artists use a symbolic shorthand to convey ideas about the book. There are recognizable elements, but they and their composition are metaphorical. You'd want to show a ring for Ringworld even if the ring as described in the book couldn look much like a ring from any perspective. Tropes like this have been used in the visual arts since the cave paintings. There is more freedom than in the fabric care coding system, but the general idea is the same: has spaceships, involves humanoids with five hands, launder in cold water.
Despite your assertion, many [SciFi/Space] artists get their calculations right. Bonestell was famous for the accuracy of his paintings in terms of what one could see from the locations. Don Davis goes even further today. The 1980s popular sci-fi artist, Chris was was trained in architecture, so his spaceships were painted accurately. The UK's venerable Eagle Comics' Dan Dare strip in the 1950s by Frank Hampson was done carefully with models to ensure spaceship and scene perspectives were correct. Obviously, I am cherry picking, and I think I do recall some rather "iffy" renderings of Niven's Ringworld (see isfdb.org for all covers), and some artists do have issues with spaceships, although I find the designs more perplexing than the perspective accuracy.
The problem is that we don't know whether the LLM is doing de novo illustrations or drawing on prior artists' works. Unlike human (and animal brains), which have relatively specialized regions for specific tasks, vision. auditory, vocalizations (and speech), and planning, AFAICT, multi-modal LLMs have a single architecture that handles all these tasks. [Correct me if I am wrong, and they integrate different AIs to handle different tasks.]
My limited experiments on the interpretation of implicit knowledge of human behavior using novels as the context to draw from is that Chat-GPT4 does poorly, while Gemini does better. Both would fail an emotional intelligence Turing Test (as I think I might too), but I expect that they will get better, but need explicit training rather than just sucking up ever more limited new data. A good example of what I mean would be if you asked an LLM to focus on movies and answer "How do people eliminate food waste sometime after eating?". Movies do not show people sitting on toilets, and even if they do, there is no indication of why. The LLM would have to use other sources of information. Earlier versions of Chat-GPT couldn't explain why a young actor such as Lily Rose Depp could not have taken the Katharine Hepburn role in The African Queen (1954) because she wasn't even alive then. Chat-GPT would instead blather about acting styles.
Yes, I agree there are many SF artists who do excellent work. However, IMO it's not that hard to visualize a spaceship or a planet, because these things exist, so the artist has something to base his painting on. In contrast, there is no Ringworld and never will be. So the artist just has to wing it. And IMO a lot of artists don't bother with the math because that's not their specialty.
So anyway, at the risk of TMI, ChatGPT says that if you're standing on the inner surface of Ringworld looking up at a 15 degree angle, the lighted portion of the ring will appear to be about .7 degrees wide. If you look up at 30 degrees the ring will appear to be about .63 degrees wide. Both these measurements are slightly wider than a full moon on Earth. Each lighted portion will also be around 13 times longer than it is wide.
It also occurred to me that I could tell ChatGPT to do an illustration based on these calculations. But that's kind of scary, because I value the work of human artists.
Firstly, almost without fail, folks in the humanities equate "brain" with "human brain" as if we are the only species endowed with such organs. Given that even the largest LLMs are tiny compared with many animal brains, it helps to put this in perspective. I suggest familiarizing oneself with this table on neuron and synapse numbers estimated in different species. Only C. elegans has a fully counted number and structure.https://en.wikipedia.org/wiki/List_of_animals_by_number_of_neurons
If scaling is to lead to human AGI (and I doubt it will) we are still far from this unless human brains are even more inefficient.
Secondly, the mechanism of handling input doesn't really matter, as long as it works. We have no need to replicate a human brain in silicon, than our aircraft have to imitate the biology of birds, bats, or even insects. We really have very little knowledge of how our brains respond to inputs. In the past, we tended to think of lower animals like mice and rats as having brains that were fairly "mechanical" in nature, ones that simulated pain and pleasure, much as we regard LLMs today. Today we know this was a false belief. The pendulum is now swinging towards giving much older clades consciousness. But clearly a small piece of human brain isn't thinking or conscious, and we may surgically or chemically may alter it for the benefit of the individual.
As for your example prompt that returned the wrong document, just imagine it was responding to you as a child (who does have a brain). You would say, "No, that is the wrong document. Please find the one with [X]." I find that I might have to go through several prompts in this way with the obsequious reply "You are correct, I am sorry. The answer is ...".
If you read the 2 underlying papers, they are reasonably impressive at exposing how the LLM is finding various [simple] answers, and the effects of "ablating" parts of the internals to change the response. I say "reasonably impressive" because they have cherry-picked the examples while noting the failure rates in their experiments. But just consider how well humans can pass different cognitive tests based on simple pattern recognition or deductive reasoning? What about students answering your questions in a class - or failing to understand something after individual attention during office hours? Is the problem in the student's brain, or the teaching (i.e. prompts)?
Having said all that, I do agree that LLMs are not "thinking" like, or as well as, trained humans. I also think that at this point, we are anthropomorphizing LLMs. However, they easily pass the old Imitation Game (Turing Test) and are already able to do this with vocal I/O. From that perspective, they are already better able to respond to inputs than the average human in a variety of cognitive and intellectual areas. As I don't intend to have AI "friends", I cannot say how well they imitate emotional responses (beating a Voight-Kampff test), but I do read that they are getting very good at fooling people who do.
If we start modern LLMs in 2018 with the release of GPT-1, we have only had 7 years of development. Further improvements in architecture, efficiency, and "brain size" may undermine our beliefs in their capabilities. I prefer the use of MAMLM/LLMs to be scaled to deploy to local devices and dedicated to specific areas of knowledge, but with the ability to communicate with other machines for different expertise, creating the "anthology intelligence" similar to humanity. Possibly while I am still alive, this approach may eclipse human thought at least in speed if not quality. How we use it is up to us.
The largest LLM has about 670 billion parameters. Assume each parameter is a synapse weight this implies the "brain" has that many synapse connections. The human brain as about 1000 - 10,000 synapses per neuron. Therefore the maximum number of neurons in the LLM is 670 million, A human brain has 100x as many, and perhaps 30x as many in the cerebrum. If we just look at cerebrums, the largest LLM has about the same number of neurons as a dog. Maybe its architecture is more efficient than an animal's, and clearly it can respond to human text and vocal instructions far better than a dog, and its ability to draw on a vast library of information exceeds that of even a human memory. Downloaded small LLM models of around 1 bn parameters take up about 1 GB. Therefore, the largest current LLMs would be less than 1 TB. When used locally, which implies all the responses are restricted to that file size (which may be compressed), it suggests that it could be contained in RAM that could reside in a powerful, but local, computer. That is quite impressive. Add context/RAG documents for specific expertise (even my fairly extensive dead-tree home library in digital form) would easily fit on a hard drive.
ChatGPT and its cousins offer an easy excuse to not read, write, or think. In "Galatea 2.2", Richard Powers presages the modern day in that Pygmalion story of AI. It is also not coincidentally one of the three book choices on my Substack this week!!!
I interacted a bit with claud. I asked if AI would be dangerous. I got platitudes. I asked how AI might be regulated? Would it be something similar to how money is regulated through a group of semi independent central banks? It asked me what I thought. I didn't get much from the interaction. Any student that lets AI create a paper will suffer accordingly.
One problem with asking if LLMs can thing is about teleology. For example, computers don't add numbers. They contain circuits that manipulate currents and voltages in such a manner that we humans can interpret their effect as having added two numbers. Evolutionary biologists especially used to get upset at teleological statements. If you ever said a pathogen evolved resistance to some treatment, you'd receive a stern lecture about populations, statistics and heredity. They seem to have calmed down recently, though I imagine students in particular courses are still expected to remember that, contra Protagoras, man is not the measure of all things.
With that said, it would really help if we understood what we consider to be thinking. We have a pretty clear sense of what it means to add two numbers, but thinking is less well defined.
I'm inclined to accept that LLMs do a certain kind of thinking, but it seems to mainly be about words and sequences of words, the kind of stuff someone with Williams syndrome does. LLM training infers things from words. Verbal concepts are sited in a higher dimensional word space, and thinking is about navigating that space given a set of words to produce another set of words. If you take a modern computer science course, it's easy to get the idea that thinking is about manipulating symbols. That's theoretical. The other kind of computer science is considered applied which gets us into class bias and other sticky areas. More recent LLM research seems to be about teaching LLMs to reason about what all those the words refer to, more manus than mens.
If you study neuroscience, you'd be more likely to think about brains as survival engines for living in the real world. Thinking is about places, events, sensations, associations and sequences, since there are distinct benefits to knowing where one is, what is likely to happen next and so on. In humans, some of this neural circuitry is also used to maneuver in symbol or word space, so our brains treat 2/3 of the way from here to there much as being 2/3 of the way from tall to short.
Beware however, there's a temptation to think we understand more of how brains work than we do. To stay humble, consider that for the last 20+ years, the Open Worm project has been trying to figure out the 302 neurons in the worm Caenorhabditis elegans, and it's still a mystery. How long could a Caernohabditis elegans survive in the wild or even on a 2D dish of agar if we replaced those 302 neurons with a suitably trained LLM? No one expects an almost microscopic worm to have deep thoughts, but it is clear such worms do think. There's a goof chance LLMs think, too, just differently.
This is just some thinking about thinking. It is not an argument to replace Google search with a microscopic worm, though I am sure some of us can see some advantages.
Deep neural networks are universal function approximators, which is a larger claim than kernel-smoother functions, but I don't think the brain is a function. The brain likely contains functions but it's more than just a collection of them.
I tried the same on Gemini 2.5 Pro, which responded that the edition in question is not available for free reading or download on archive.org. Which is correct. It suggested other editions, or where to find the requested edition for sale. Excerpt:
Unfortunately, based on the search results, the specific 2008 Oxford University Press edition of Machiavelli's The Prince, translated and edited by Peter Bondanella, does not appear to be available for free reading or download on archive.org.
Here's what the search found:
Other Editions on Archive.org: There are other editions of The Prince available on archive.org, such as one translated by W.K.[1] Marriott and another translated by Luigi Ricci. These are not the specific Bondanella/Oxford edition you requested.
Bondanella's Work on Archive.org: Archive.org does have Machiavelli's Discourses on Livy translated by Julia and Peter Bondanella (published by Oxford University Press), but it is listed as "access-restricted" and requires borrowing.
Confirmation of the Edition: Several results confirm the existence and details of the 2008 Oxford World's Classics edition translated by Peter Bondanella (ISBN: 9780199535699), often with an introduction by Maurizio Viroli. These results point to booksellers like Amazon, Blackwell's, and publisher pages like Oxford University Press itself, not archive.org.
This suggests that the response Brad got was not wrong? Interesting.
Well yes, any well educated university professor can confuse or fool ChatGPT. But for ordinary laymen it's pretty amazing. I recently asked ChatGPT if it thought it had passed the Turing Test. It hedged, saying yes, in some respects, but no in others.
Another time I asked it a question about Larry Niven's "Ringworld." I've always suspected that the artists who paint the covers for Ringworld novels don't get the perspective right because most artists are not good at math. But ChatGPT is very good at math. So I asked what a person on the inner surface of Ringworld would see if looking up at the ring at night. ChatGPT showed me all the calculations and told me what a person would see. Pretty impressive for a "Kernel-Smoother Function."
My first thought was that it is only night on Ringworld if you are on the outside away from the sun, so you wouldn't see anything but your local area of the ring illuminated by starlight. Then, I vaguely remembered that there was a system of blinds to provide day and night in its own orbit closer to the sun. Trying to visualize that kind of thing is tough. Edgar Rice Burroughs had a sun inside a planet that provided daylight all the time, and I remember thinking he had gotten some of the details wrong, aside from having a star inside a planet.
Book cover artists use a symbolic shorthand to convey ideas about the book. There are recognizable elements, but they and their composition are metaphorical. You'd want to show a ring for Ringworld even if the ring as described in the book couldn look much like a ring from any perspective. Tropes like this have been used in the visual arts since the cave paintings. There is more freedom than in the fabric care coding system, but the general idea is the same: has spaceships, involves humanoids with five hands, launder in cold water.
Despite your assertion, many [SciFi/Space] artists get their calculations right. Bonestell was famous for the accuracy of his paintings in terms of what one could see from the locations. Don Davis goes even further today. The 1980s popular sci-fi artist, Chris was was trained in architecture, so his spaceships were painted accurately. The UK's venerable Eagle Comics' Dan Dare strip in the 1950s by Frank Hampson was done carefully with models to ensure spaceship and scene perspectives were correct. Obviously, I am cherry picking, and I think I do recall some rather "iffy" renderings of Niven's Ringworld (see isfdb.org for all covers), and some artists do have issues with spaceships, although I find the designs more perplexing than the perspective accuracy.
The problem is that we don't know whether the LLM is doing de novo illustrations or drawing on prior artists' works. Unlike human (and animal brains), which have relatively specialized regions for specific tasks, vision. auditory, vocalizations (and speech), and planning, AFAICT, multi-modal LLMs have a single architecture that handles all these tasks. [Correct me if I am wrong, and they integrate different AIs to handle different tasks.]
My limited experiments on the interpretation of implicit knowledge of human behavior using novels as the context to draw from is that Chat-GPT4 does poorly, while Gemini does better. Both would fail an emotional intelligence Turing Test (as I think I might too), but I expect that they will get better, but need explicit training rather than just sucking up ever more limited new data. A good example of what I mean would be if you asked an LLM to focus on movies and answer "How do people eliminate food waste sometime after eating?". Movies do not show people sitting on toilets, and even if they do, there is no indication of why. The LLM would have to use other sources of information. Earlier versions of Chat-GPT couldn't explain why a young actor such as Lily Rose Depp could not have taken the Katharine Hepburn role in The African Queen (1954) because she wasn't even alive then. Chat-GPT would instead blather about acting styles.
Yes, I agree there are many SF artists who do excellent work. However, IMO it's not that hard to visualize a spaceship or a planet, because these things exist, so the artist has something to base his painting on. In contrast, there is no Ringworld and never will be. So the artist just has to wing it. And IMO a lot of artists don't bother with the math because that's not their specialty.
So anyway, at the risk of TMI, ChatGPT says that if you're standing on the inner surface of Ringworld looking up at a 15 degree angle, the lighted portion of the ring will appear to be about .7 degrees wide. If you look up at 30 degrees the ring will appear to be about .63 degrees wide. Both these measurements are slightly wider than a full moon on Earth. Each lighted portion will also be around 13 times longer than it is wide.
It also occurred to me that I could tell ChatGPT to do an illustration based on these calculations. But that's kind of scary, because I value the work of human artists.
I agree and disagree with your points.
Firstly, almost without fail, folks in the humanities equate "brain" with "human brain" as if we are the only species endowed with such organs. Given that even the largest LLMs are tiny compared with many animal brains, it helps to put this in perspective. I suggest familiarizing oneself with this table on neuron and synapse numbers estimated in different species. Only C. elegans has a fully counted number and structure.https://en.wikipedia.org/wiki/List_of_animals_by_number_of_neurons
If scaling is to lead to human AGI (and I doubt it will) we are still far from this unless human brains are even more inefficient.
Secondly, the mechanism of handling input doesn't really matter, as long as it works. We have no need to replicate a human brain in silicon, than our aircraft have to imitate the biology of birds, bats, or even insects. We really have very little knowledge of how our brains respond to inputs. In the past, we tended to think of lower animals like mice and rats as having brains that were fairly "mechanical" in nature, ones that simulated pain and pleasure, much as we regard LLMs today. Today we know this was a false belief. The pendulum is now swinging towards giving much older clades consciousness. But clearly a small piece of human brain isn't thinking or conscious, and we may surgically or chemically may alter it for the benefit of the individual.
As for your example prompt that returned the wrong document, just imagine it was responding to you as a child (who does have a brain). You would say, "No, that is the wrong document. Please find the one with [X]." I find that I might have to go through several prompts in this way with the obsequious reply "You are correct, I am sorry. The answer is ...".
If you read the 2 underlying papers, they are reasonably impressive at exposing how the LLM is finding various [simple] answers, and the effects of "ablating" parts of the internals to change the response. I say "reasonably impressive" because they have cherry-picked the examples while noting the failure rates in their experiments. But just consider how well humans can pass different cognitive tests based on simple pattern recognition or deductive reasoning? What about students answering your questions in a class - or failing to understand something after individual attention during office hours? Is the problem in the student's brain, or the teaching (i.e. prompts)?
Having said all that, I do agree that LLMs are not "thinking" like, or as well as, trained humans. I also think that at this point, we are anthropomorphizing LLMs. However, they easily pass the old Imitation Game (Turing Test) and are already able to do this with vocal I/O. From that perspective, they are already better able to respond to inputs than the average human in a variety of cognitive and intellectual areas. As I don't intend to have AI "friends", I cannot say how well they imitate emotional responses (beating a Voight-Kampff test), but I do read that they are getting very good at fooling people who do.
If we start modern LLMs in 2018 with the release of GPT-1, we have only had 7 years of development. Further improvements in architecture, efficiency, and "brain size" may undermine our beliefs in their capabilities. I prefer the use of MAMLM/LLMs to be scaled to deploy to local devices and dedicated to specific areas of knowledge, but with the ability to communicate with other machines for different expertise, creating the "anthology intelligence" similar to humanity. Possibly while I am still alive, this approach may eclipse human thought at least in speed if not quality. How we use it is up to us.
Size:
The largest LLM has about 670 billion parameters. Assume each parameter is a synapse weight this implies the "brain" has that many synapse connections. The human brain as about 1000 - 10,000 synapses per neuron. Therefore the maximum number of neurons in the LLM is 670 million, A human brain has 100x as many, and perhaps 30x as many in the cerebrum. If we just look at cerebrums, the largest LLM has about the same number of neurons as a dog. Maybe its architecture is more efficient than an animal's, and clearly it can respond to human text and vocal instructions far better than a dog, and its ability to draw on a vast library of information exceeds that of even a human memory. Downloaded small LLM models of around 1 bn parameters take up about 1 GB. Therefore, the largest current LLMs would be less than 1 TB. When used locally, which implies all the responses are restricted to that file size (which may be compressed), it suggests that it could be contained in RAM that could reside in a powerful, but local, computer. That is quite impressive. Add context/RAG documents for specific expertise (even my fairly extensive dead-tree home library in digital form) would easily fit on a hard drive.
ChatGPT and its cousins offer an easy excuse to not read, write, or think. In "Galatea 2.2", Richard Powers presages the modern day in that Pygmalion story of AI. It is also not coincidentally one of the three book choices on my Substack this week!!!
MMM?
Trained on what can be found on the internets and other sources.
Do they get also trained on how people actually use words in everyday life?
I think they're trained on punitive essays with required word counts.
To be real AGI their output should be as sloppy and ill organized as most thoughts and conversations of real people.
Can the prompt ask for an output at a certain reading level?
I interacted a bit with claud. I asked if AI would be dangerous. I got platitudes. I asked how AI might be regulated? Would it be something similar to how money is regulated through a group of semi independent central banks? It asked me what I thought. I didn't get much from the interaction. Any student that lets AI create a paper will suffer accordingly.
One problem with asking if LLMs can thing is about teleology. For example, computers don't add numbers. They contain circuits that manipulate currents and voltages in such a manner that we humans can interpret their effect as having added two numbers. Evolutionary biologists especially used to get upset at teleological statements. If you ever said a pathogen evolved resistance to some treatment, you'd receive a stern lecture about populations, statistics and heredity. They seem to have calmed down recently, though I imagine students in particular courses are still expected to remember that, contra Protagoras, man is not the measure of all things.
With that said, it would really help if we understood what we consider to be thinking. We have a pretty clear sense of what it means to add two numbers, but thinking is less well defined.
I'm inclined to accept that LLMs do a certain kind of thinking, but it seems to mainly be about words and sequences of words, the kind of stuff someone with Williams syndrome does. LLM training infers things from words. Verbal concepts are sited in a higher dimensional word space, and thinking is about navigating that space given a set of words to produce another set of words. If you take a modern computer science course, it's easy to get the idea that thinking is about manipulating symbols. That's theoretical. The other kind of computer science is considered applied which gets us into class bias and other sticky areas. More recent LLM research seems to be about teaching LLMs to reason about what all those the words refer to, more manus than mens.
If you study neuroscience, you'd be more likely to think about brains as survival engines for living in the real world. Thinking is about places, events, sensations, associations and sequences, since there are distinct benefits to knowing where one is, what is likely to happen next and so on. In humans, some of this neural circuitry is also used to maneuver in symbol or word space, so our brains treat 2/3 of the way from here to there much as being 2/3 of the way from tall to short.
Beware however, there's a temptation to think we understand more of how brains work than we do. To stay humble, consider that for the last 20+ years, the Open Worm project has been trying to figure out the 302 neurons in the worm Caenorhabditis elegans, and it's still a mystery. How long could a Caernohabditis elegans survive in the wild or even on a 2D dish of agar if we replaced those 302 neurons with a suitably trained LLM? No one expects an almost microscopic worm to have deep thoughts, but it is clear such worms do think. There's a goof chance LLMs think, too, just differently.
This is just some thinking about thinking. It is not an argument to replace Google search with a microscopic worm, though I am sure some of us can see some advantages.
Deep neural networks are universal function approximators, which is a larger claim than kernel-smoother functions, but I don't think the brain is a function. The brain likely contains functions but it's more than just a collection of them.