Another Brief Note on the Flexible-Function View of MAMLMs
Modern Advanced Machine Learning Models—MAMLMs—are they BRAINS!!!! or not? So far it is still very clear to me that they are not. And I see it as highly probable that MAMLMs as we know them are...
Modern Advanced Machine Learning Models—MAMLMs—are they BRAINS!!!! or not? So far it is still very clear to me that they are not. And I see it as highly probable that MAMLMs as we know them are still at the very beginning of what might become the process of building Artificial Intelligence, as capable as they are at Complex Information Processing. & yet we overascribe mind to them. Searle’s Chinese Room does not understand Chinese until it grows as large as the entire Earth and is serviced by tens and thousands of robots traveling at lightspeed. & we are still very far from that…
If you start from the premise that a language model like ChatGPT is a very flexible, very high dimensional, very big data regression-and-classification engine best seen as a function from the domain of word-strings to the range of continuation words, I think a large number of things become clear.
First, because its training dataset is sparse in its potential domain—nearly all even moderate-length word-sequences that are not boilerplate or cliché are unique—its task is one of interpolation: take word-sequences “close” to the prompt, examine their continuations, and average them. Thus while pouring more and more resources into the engine does get you, potentially, a finer and finer interpolation, it seems highly likely that this process will have limits rather than grow to the sky, and it is better to look at it as an engine summarizing what humans typically say in analogous linguistic situations rather than any form of “thinking”.
I think the post-ChatGPT3 history of LLMs bears this out:
Sebastian Raschka: The State of Reinforcement Learning for LLM Reasoning <https://magazine.sebastianraschka.com/p/the-state-of-llm-reasoning-model-training>: ‘Releases of new flagship models like GPT-4.5 and Llama 4…. Reactions to these releases were relatively muted…. The muted response… suggests we are approaching the limits of what scaling model size and data alone can achieve. However, OpenAI's recent release of the o3 reasoning model demonstrates there is still considerable room for improvement when investing compute strategically, specifically via reinforcement learning methods tailored for reasoning tasks. (According to OpenAI staff during the recent livestream, o3 used 10× more training compute compared to o1)…
Second, reinforcement learning—prompt engineering—and such are ways of attempting to condition this interpolation process by altering the domain word-sequence in such a way as to carry it into a portion of the training dataset where, when judged by humans, this function (word-strings) → (continuations) does not suck. That is, in some sense, all they are: You have a function trained on internet dreck in which there are some veins of gold—accurate information and useful continuations of word-strings—and you need to transform the word-string you send so that it lands inside one of those veins.
As the wise Cosma Shalizi says:
Cosma Shalizi: On Feral Library Card Catalogs, or, Aware of All Internet Traditions <https://www.programmablemutter.com/cp/161552401>: ‘LLMs are parametric probability models of symbol sequences, fit to large corpora of text by maximum likelihood. By design, their fitting process strives to reproduce the distribution of text in the training corpus…. Prompting is conditioning: the output after a prompt is a sample from the conditional distribution of text coming after the prompt…. All these distributions are estimated with a lot of smoothing… [that tell the probability model when to treat different-looking contexts as similar…. This smoothing… is what lets the models respond with sensible-looking output… to prompts they've never seen in the training corpus. It also, implicitly, tells the model what to ignore, what distinctions make no difference. This is part (though only part) of why these models are lossy…
And these models are likely to be very good and useful where:
Cosma Shalizi: On Feral Library Card Catalogs, or, Aware of All Internet Traditions <https://www.programmablemutter.com/cp/161552401>: ‘A huge amount of cultural and especially intellectual tradition consists of formulas, templates, conventions, and indeed tropes and stereotypes…. Formulas also reduce the cognitive burden on people receiving communications… boiler-plate and ritual, yes, but it's not just boiler-plate and ritual, or at least not pointless ritual…
But not so good and not so useful for many other tasks.
And now comes the part in which I note that someday this bashing-of-Artificial-Super-Intelligence-grifters-and-boosters may go wrong. I am, after all, making a version of the Searle Chinese Room argument against the case that MAMLMs are brains. I believe that the argument applies here. But Scott Aaronson’s comments on that kind of argument are apposite here:
Scott Aaronson: PHYS771 Lecture 4: Minds and Machines <https://www.scottaaronson.com/democritus/lec4.html>: ‘In the last fifty years, have there been any new insights about the Turing Test?… A non-insight, which is called Searle's Chinese Room…. Let's say you don't speak Chinese…. You sit in a room, and someone passes you paper slips through a hole in the wall with questions written in Chinese, and you're able to answer the questions (again in Chinese) just by consulting a rule book. In this case, you might be carrying out an intelligent Chinese conversation, yet by assumption, you don't understand a word of Chinese!…
Like many other thought experiments, the Chinese Room gets its mileage from a deceptive choice of imagery—and more to the point, from ignoring computational complexity. We're invited to imagine someone pushing around slips of paper with zero understanding or insight—much like the doofus freshmen who write (a+b)2=a2+b2 on their math tests. But how many slips of paper are we talking about?… If each page of the rule book corresponded to one neuron… we'd be talking about a "rule book" at least the size of the Earth, its pages searchable by a swarm of robots traveling at close to the speed of light…. Maybe it's not so hard to imagine that this enormous Chinese-speaking entity—this dian nao—that we've brought into being might have something we'd be prepared to call understanding or insight…
At what point does rote—or nearly rote—manipulation of symbols and interpolation from examples turn into real thinking? We do not know, on the one hand. On the other hand, I see absolutely no signs anywhere in these systems that they are close.
What, then, is going on with all the people who think that we are close?
My guess: From an evolutionary psychology standpoint, humans are pattern-recognition machines. We are primed to detect agency—even where it doesn’t exist—because assuming intent behind observed behavior was often adaptive. You hear a rustle in the bushes: it’s safer to think “predator” than “wind.”
This cognitive bias leads us to over-ascribe intelligence and intention to things that act in ways that seem purposeful.
In the 20th century, people attributed intelligence to thermostats (“it knows the room is too hot”) and ELIZA-like chatbots <https://en.wikipedia.org/wiki/ELIZA>. Now, we do the same with large language models like ChatGPT. The outputs feel coherent, fluent, and often insightful—so we instinctively project mind onto them. And this is a useful thing to do: recall Daniel Dennett’s “intentional stance”—that we have often found it useful to model complex systems (or simple systems: thermostats!) as having beliefs and desires not because they do but because taking that stance is a very quick and useful way to begin predicting their behavior.
Thus as our technologies increase productivity and complexity we find that this over-ascription generate cognitive dissonance: we struggle to grasp how machines can do what they do without being “like us”, and when a machine completes a task that once required a skilled human—like writing a letter, playing chess, or composing music—people start asking not “how does it work?” but “who is it in there?”
What are the consequence of placing MAMLMs in the wrong box? Of thinking of them as rapidly becoming human-level replacement for human beings, rather than as powerful new complements to human minds offloading a great deal of mechanical repetitive tasks, boilerplate generation, formulae application, and ritual enaction?
The first consequence is that it makes it easy to take a huge amount of money from gullible rich AI enthusiasts.
As with the hypothetical Chinese room, the math involved is conceptually simple in outline. Most of what was classed as machine learning/neural nets until recently was just classification by discriminant analysis (worked out by Fisher nearly 100 years ago) but with scads of data and as you say, flexible functional forms. LLMs extend that by making the right hand side of the model (the predicted text) flexible as well.
Leaving aside the high-level questions about whether this intelligence, the crucial economic issue is whether there is anything here that can't be replicated easily once you know it can be done at all. The proliferation of LLMS, most notably cheap LLMs, suggest not, especially since we know that the data set has been exhausted.
What that means is that the tens (maybe hundreds) of billions invested in AI so far is really public good research, with no real private payoff. That point is independent of whether the public benefit is huge, modest or negative.
As I remember from a course I took long ago, early AI researchers fell into two distinct groups (with some overlap): those who wanted to develop falsifiable theories of intelligence and those who wanted to program computers to emulate human reasoning. Both approaches hit some hard limits, and a lot of the early work seems to have been forgotten by the 1980s. With the emergence of neural nets came the *hope* that all you needed was a big enough computer and enough data. I think that's what we're seeing now with MAMLMs.
Still, it isn't clear what the commercial value of these systems will be. So for companies like Google, Apple, Microsoft, etc. it makes sense to invest *enough* Internal Research & Development (IR&D) to understand what MAMLMs can & *can't* do and what the *realistic* commercial potential is.