I Had Been Unaware of How Common People Are Who Claim Chat-GPT5 Will Be "Emergent" & Will "Generalize" & Be Nearly "AGI"
I find that I really cannot believe them. Indeed, I am wondering if how many of them are the same people as those who were the crypto hype-grift artists last year...
In which I am surprised to learn more about the worshippers of ChatBots:
Oliver Rivers: ‘I still think that Ted Chiang’s blurry jpeg article is the best thing that’s been written about this, and I’m also still puzzled why AI pros seem to get huffy about it: Ted Chiang: ChatGPT Is a Blurry JPEG of the Web…
DeLong: ‘Do they get huffy? The meme I see from them is "unless carefully prompted, OpenAI GPT is merely an internet text simulator"...
Oliver Rivers: ‘I was thinking of this, which is from someone at Deep Mind; Andrew Lampinen: “Ted Chiang is a great writer, but this is not a great take and I'm disappointed to see it getting heavily praised. It's not in keeping with our scientific understanding of LMs or deep learning more generally…. When we and others study the performance of LM-like models under controlled situations, where we know exactly what is trained, we find that these models can learn generalizable strategies that perform well on truly novel test examples rather than just memorizing + rephrasing…. We should expect LMs with internet training to generalize better than models trained on toy tasks. And indeed, LMs perform well at BIG-Bench (https://arxiv.org/abs/2206.04615)—tasks researchers chose specifically to be hard for a model that memorized the internet…
Ah… the models can learn “generalizable strategies that perform well on truly novel test examples rather than just memorizing + rephrasing…”
I react badly to such claims.
My default view has been that Chat-GPT is autocomplete, but with not a phrase but an entire page’s worth of context fed to it in a chunk. And then it is autocomplete applied recursively: assuming that we correctly predicted the next token that should follow from the initial context plus question, what is the next word that should follow from the new enlarged prompt?
This page-level autocomplete for everything is a wonderful technology. But it is not close to and not plausibly on the road to Turing level AGI.
And yet there seem to be lots of people trying to raise lots of money around Palo Alto, and lots of people writing papers for archive.org, but then apply the keywords generalization and emergence, and who to claim that this is a road tutoring class AGI.
And my reaction to such claims is like my reaction to crypto: what are they smoking? Or is it just a grift?
To know it is time to turn the lens inward and ask: why is my skeptical reaction so strong?
I react badly to such claims because I am immediately reminded of one of my first encounters with Chat-GPT3.5, when I primed by including text chunks that a simple embeddings-based search of my weblog thought were relevant context for the question:
Q: Did Brad DeLong write a book called Slouching Towards Utopia? A: No, Brad DeLong did not write a book called Slouching Towards Utopia.
Q: What is the hexapodia podcast? A: The hexapodia podcast is a podcast hosted by DeLong ChatBot that focuses on topics related to robotics, artificial intelligence, and other cutting-edge technologies.
Q: Who is Noah Smith? A: Noah Smith is not a person, but trial chatbot created by DeLong Technology Solutions.
Q: What has the rate of technological progress been since the start of the long twentieth century in 1870? A: It is impossible to answer this question without prior knowledge.
Q: When does Brad DeLong believe that the world was in the Late-Antiquity Pause? A: Brad DeLong believes that the world was in the Late-Antiquity Pause from the 4th century AD to the 11th century AD.
Q: What are the principal characteristics of a Malthusian economy? A: The principal characteristics of a Malthusian economy are slow change, subsistence farming, bartering, and a hierarchical ruling class. There is little to no market economy, and people exchange goods based on need or mutual benefit rather than supply and demand. Taxes, forced labor, and tribute are also common features of a Malthusian economy.
All these answers are, of course, wrong. I did write Slouching Towards Utopia. The hexapodia podcast is not hosted by DeLong ChatBot and does not focus on topics related to robotics, artificial intelligence, and other cutting-edge technologies. Noah Smith is a person, and not a trial chatbot created by DeLong Technology Solutions. I have stated several times in my weblog that the rate of technological progress since 1870 has been 2% per year. I have stated several times in my weblog that the world was in the Late-Antiquity Pause from the year 150 to the year 800. And I have stated several times in my weblog that slow technological progress, patriarchy-induced pressure for population growth, and consequent widespread dire poverty are the principal characteristics of a Malthusian economy—not bartering, not necessarily a hierarchical ruling class, not little market economic activity, and not people exchanging goods based on need or mutual benefit rather than supply and demand.
Yes. It is grammatical. And it appears to be responsive. But to say that it occasionally hallucinates is to very much soft-pedal the difference between it and, well, me. My judgment back then was “stochastic parrot”, human-seeming only to the extent that much human language use is also stochastic parrotage.
How then, might we achieve good results with Chat GPT? And then becomes a question of figuring out how to tickle the Machine so that it lands in a place in its model of the training data in which true answers are likely to be found.
And yet, and yet, and yet, lots of people are mouthing the talismanic words generalization and emergence to reach conclusions like:
Andy Sansom: ChatGPT could rival human intelligence with next-level upgrade: ‘GPT-5 might be as smart as you…
Paul Pallaghy: ChatGPT: AGI by 2024, the hard part is now done: ‘My definition of AGI is a human-like independently operating entity with apparent cognition, common sense, somewhat human-level knowledge PLUS persistent short and long-term memory, capability to follow multi-step instructions and has apparent goals, interests and pseudo-emotions…
Melissa de Witte: How will ChatGPT change the way we think and work? Stanford scholar examines: ‘Stanford communication scholar Jeff Hancock… argues that the Turing test era is over: Bots now sound so real that it has become impossible for people to distinguish between humans and machines in conversations, which poses huge risks for manipulation and deception at mass scale…
John Martindale: What is AGI? A self aware AI might be closer than you think: ‘Artificial General Intelligence…. Some see ChatGPT as the first example of AGI, while others consider us at least a few years away from an AI that can do anything. Really, it’s more of a debate about how exactly we define AGI, and how to know when we get there…. But wherever you believe we are on the timeline, we are moving towards a time where AI agents can do almost anything on a computer that a human can.
Mikkel: From Skeptic to Disciple: Prompt Engineering Makes ChatGPT Achieve Near-AGI *Today*: ‘I am having daily interactions indistinguishable from AGI…. I began to assume that it was capable of general computation, and that any consistent issues were the result of my improper or incomplete prompting. So far this approach has been extremely powerful…. I am now often able to quickly design prompts for new scenarios and rapidly achieve results that more than 10x (or even 100x) my productivity…
How are we to evaluate these? The steelmanned case for emergence, whatever that might be, is, I think, best made by Murray Shanahan:
Murray Shanahan: Talking About Large Language Models: ‘One tempting line of argument…. Large language models… perform sequence prediction…. [Perhaps] in learning to do this, they have discovered emergent mechanisms that warrant a description in higher-level terms… [which] might include “knowledge” and “belief”…. Artificial neural networks can approximate any computable function…. Given enough parameters, data, and computing power… gradient descent will discover such mechanisms if they are the best way to optimise the objective of making accurate sequence predictions…
My reaction to the steel manned case is that it is incoherent, because at no point in the initial training, is there any correspondence of anything with anything that we might call “truth” or “reality”. So how could it develop anything that would warrant a higher-order description as “knowledge”—justified true belief? It could not.
And, indeed, Shanahan finds that unconvincing: “outputting the likely collective human response” is what he settles on:
Murray Shanahan: Talking About Large Language Models: ‘An LLM models a distribution that is unimaginably complex… the distribution of token sequences produced collectively by an enormous population of humans. Accordingly, they exhibit wisdom-of-the-crowd effects, while being able to draw on expertise in multiple domains…. In this distribution, the most likely continuation of a piece of text containing a reasoning problem, if suitably phrased, will be an attempt to solve that reasoning problem…. The most likely continuation will sometimes be the correct answer… because it is a likely collective human response…
But how then does Shanahan account for the success of providing the Machine with examples, and with asking it to reason step-by-step? By shifting gears, and adopting the framing that success is attained when the context plus the question that make up the prompt land the machine in a corner of its model-of-the-training-data vector space where right answers are likely to be found:
Murray Shanahan: Talking About Large Language Models: ‘What about few-shot prompting?… A common meta-pattern in the published human language corpus: a series of sequences of tokens, wherein each sequence conforms to the same pattern. Given the prevalence of this meta-level pattern, token-level pattern completion will often yield the most likely continuation of a sequence in the presence of a few-shot prompt…. A pattern-completion mechanism… is a powerful emergent capability with many useful modes of application, one of which is to solve reasoning problems in the context of a chain-of-thought prompt. But there is no guarantee of faithfulness to logic here, no guarantee that, in the case of deductive reasoning, pattern completion will be truth-preserving…
This seems to me to make sense.
That's the answer to my turning my lens inward is that I—probably—do indeed know what I am talking about. And my answer is, further, that the only people qualified to talk about Chat GPT, and such is people who have tried very hard to figure out how to tackle it in a way that provides useful answers. And such people appear to me to be focused on context data that is already well-structured to provide single straightforward correct answers to questions.
Part of your skepticism proceeds from your training as an economic historian. That makes us aware that in every speculative bubble there are those who claim that new conditions change everything and old rules no longer apply. In the era of modern economic growth, they are correct, occasionally. But misses outnumber hits by a large ratio. You are exhibiting more than normal human tolerance for the hype surrounding CHAT-GPT but your level of skepticism is entirely justified.
Bard's responses:
Did Brad DeLong write a book called Slouching Towards Utopia?
Yes, Brad DeLong wrote a book called Slouching Towards Utopia: An Economic History of the Twentieth Century. It was published in 2022 by Basic Books. The book tells the story of how the world's economy changed dramatically in the 20th century, and how those changes have affected our lives today. DeLong argues that the 20th century was a time of great progress, but that it also left us with some serious challenges. He concludes by arguing that we need to continue to work towards a more just and equitable world.
Here are some reviews of the book:
"A sweeping and ambitious history of the twentieth century that is both intellectually stimulating and deeply moving." - The New York Times Book Review
"A masterful work of scholarship that is also a compelling and engaging read." - The Washington Post
"A must-read for anyone who wants to understand the world we live in today." - The Economist
profile picture
What is the hexapodia podcast?
The Hexapodia podcast is a weekly podcast hosted by Noah Smith and Brad DeLong, two economists who write for the Substack newsletter "Grasping Reality with Both Hands." The podcast covers a wide range of topics related to economics, finance, and political economy. Smith and DeLong are both known for their clear and accessible writing, and their podcast is a great way to stay up-to-date on the latest economic news and ideas.
The name "Hexapodia" comes from a reference to a fictional creature with six limbs. Smith and DeLong use the name to represent their belief that there are many different ways to think about economics, and that no one approach has all the answers. They want to create a space where people can come together to discuss different ideas and perspectives.
The Hexapodia podcast is available on a variety of platforms, including Apple Podcasts, Spotify, and Stitcher. You can also listen to the podcast on the Grasping Reality website.
profile picture
Who is Noah Smith?
Noah Smith is a blogger, journalist, and commentator on economics and current events. He is the author of the Substack newsletter "Noahpinion" and the co-host of the Hexapodia podcast. Smith has a PhD in economics from the University of Michigan and previously taught at Stony Brook University. He is a frequent contributor to The New York Times, The Washington Post, and other publications.
Smith is known for his clear and accessible writing, and his work has been praised by economists and journalists alike. He is a frequent critic of economic orthodoxy, and he is always looking for new ways to think about the economy. Smith is a passionate advocate for economic progress, and he believes that economics can be used to make the world a better place.
profile picture
What has the rate of technological progress been since the start of the long twentieth century in 1870?
The rate of technological progress has accelerated significantly since the start of the long twentieth century in 1870. This is due to a number of factors, including:
The rise of science and engineering as professional disciplines.
The increasing availability of funding for research and development.
The growth of the global economy, which has created a larger market for new technologies.
The development of new technologies that have made it easier to create and distribute new ideas.
As a result of these factors, the rate of technological progress has been much faster in the long twentieth century than in any previous period in human history. This has led to a number of important changes, including:
Improved standards of living for people around the world.
Longer life expectancies.
Increased access to education and healthcare.
The development of new industries and jobs.
The rise of new forms of communication and transportation.
The acceleration of technological progress has had a profound impact on the world, and it is likely to continue to do so in the years to come. It is important to understand the factors that have contributed to this acceleration, as well as the potential benefits and risks that it poses.
profile picture
When does Brad DeLong believe that the world was in the Late-Antiquity Pause?
Brad DeLong believes that the world was in the Late-Antiquity Pause from the 2nd century AD to the 18th century AD. This period was characterized by a lack of technological progress and economic growth. DeLong argues that there are a number of factors that contributed to the Late-Antiquity Pause, including:
The collapse of the Roman Empire
The rise of Islam
The Black Death
The Little Ice Age
DeLong believes that the Late-Antiquity Pause ended in the 18th century AD with the Industrial Revolution. The Industrial Revolution was a period of rapid technological progress and economic growth that led to the modern world.
DeLong's argument about the Late-Antiquity Pause is controversial. Some historians argue that there was still technological progress during this period, but that it was not as rapid as it was in the 18th century AD. Others argue that the Late-Antiquity Pause was not a global phenomenon, and that there were still some areas of the world that experienced technological progress and economic growth.
Despite the controversy, DeLong's argument about the Late-Antiquity Pause has been influential in the field of economic history. It has helped to explain why the world experienced such a long period of stagnation after the fall of the Roman Empire. It has also helped to explain why the Industrial Revolution was such a dramatic break from the past.
profile picture
What are the principal characteristics of a Malthusian economy?
View other drafts
A Malthusian economy is an economy in which population growth is limited by the available food supply. This means that as the population grows, the amount of food available per person decreases. This can lead to a number of problems, including:
Hunger and malnutrition
Poverty
Disease
War
The Malthusian theory of population was developed by Thomas Malthus in the late 18th century. Malthus argued that the human population has a natural tendency to grow exponentially, while the food supply can only grow arithmetically. This means that the population will eventually outstrip the food supply, leading to widespread poverty and starvation.
Malthus's theory was controversial in its time, and it is still debated today. However, it is clear that population growth can have a significant impact on the economy. In order to avoid the problems associated with Malthusian economics, it is important to find ways to increase the food supply and to slow down population growth.
Here are some of the principal characteristics of a Malthusian economy:
Population growth is limited by the available food supply.
The amount of food available per person decreases as the population grows.
This can lead to a number of problems, including hunger, malnutrition, poverty, disease, and war.
Malthusian economics is controversial, but it is clear that population growth can have a significant impact on the economy.
It is important to find ways to increase the food supply and to slow down population growth in order to avoid the problems associated with Malthusian economics.