2023-04-21 Fr: continuing to try to train a ChatBot so that it can give talks on and answer questions about Slouching Towards Utopia <bit.ly/3pP3Krk>...
So my original academic background is in natural language processing. I wrote some very primitive statistical language models, and I worked through much of the '00s at Motorola, on predictive text entry software. (All of my Motorola patents got sold to Google at some point, so whenever you say "damn you, autocorrect!" to an Android -- sorry!) I studied at Johns Hopkins in the back half of the '90s, under Fred Jelinek, who's widely regarded as the originator of the revolution in speech recognition and text processing, replacing attempts to code formal grammars with abstract statistical models.
So I have some rudimentary understanding of what these modern much-fancier statistical chat-bots are doing, and I feel fairly confident in saying that in terms of reaching "artificial general intelligence", they are a dead end.
Boston Dynamics' terrain navigating robots are a much more promising avenue of research. AGI, if it ever emerges, is going to need to have some kind of grounding in reality. It needs first a model of the world, and a model of itself in relation to the world. Building from that, you'd want to gradually generalize its model of itself, to recognize that some of what's in the world is other agents like itself -- so then you get theory of mind. (In this area, AIs that control avatars in games -- where they deal with a virtual world, and other agents, and try to achieve goals -- are potentially a relevant area of research. We've already seen some transfer of code for efficient control loops from gaming, to controls for drones, self-driving vehicles, etc.) Even the most primitive animal has something like this -- prey animals are constantly trying to detect predators in the environment and predict their attack vectors, in order to escape.
Once you have an AI that is reality-grounded, _then_ you can start to layer on language, teaching it how to explain itself, and understand the intentions and requests of other agents. (Though of course this raises all kinds of interesting questions around the alignment problem. These explanations could be false both for reasons of willful deception -- the AI becoming trained to tell you what you _want_ to hear, regardless of the truth -- and just because explaining motivation is hard. _Humans_ are quite bad at understanding their own motivations, let alone interpreting other people's.)
Without attachment to reality, a language model has no way of understanding what is true or useful. As you say, it lacks any kind of grounding in a theoretical framework. It's just producing words that sound vaguely similar to its source material. It's Frankfurtian bullshit.
What I think is that the ChatBot does not have a theory of mind for you, it has only a theory of text. So if the response to the prompt "Why was humanity so poor back in the long Agrarian Age?" does not include the word "Malthusian", that is because whenever the text it was trained on - presumably Slouching Towards Utopia? - says that humanity was poor back in the long Agrarian Age, it does not usually mention that this was a Malthusian condition. When you wrote the text, you were thinking "Malthusian", because you have a theory about the Agrarian Age; and when I read that text, I was thinking "Brad is thinking Malthusian", because I have a theory of mind for Brad DeLong. But that's the difference between ChatBot and me.
Put in the terms you use below, the ChatBot can go to a corner of its training space, and its space is a very very high dimensional hypercube not legible to the human mind, so this allows it to surprise us; but it cannot create a corner of space where it was never trained.
So my original academic background is in natural language processing. I wrote some very primitive statistical language models, and I worked through much of the '00s at Motorola, on predictive text entry software. (All of my Motorola patents got sold to Google at some point, so whenever you say "damn you, autocorrect!" to an Android -- sorry!) I studied at Johns Hopkins in the back half of the '90s, under Fred Jelinek, who's widely regarded as the originator of the revolution in speech recognition and text processing, replacing attempts to code formal grammars with abstract statistical models.
So I have some rudimentary understanding of what these modern much-fancier statistical chat-bots are doing, and I feel fairly confident in saying that in terms of reaching "artificial general intelligence", they are a dead end.
Boston Dynamics' terrain navigating robots are a much more promising avenue of research. AGI, if it ever emerges, is going to need to have some kind of grounding in reality. It needs first a model of the world, and a model of itself in relation to the world. Building from that, you'd want to gradually generalize its model of itself, to recognize that some of what's in the world is other agents like itself -- so then you get theory of mind. (In this area, AIs that control avatars in games -- where they deal with a virtual world, and other agents, and try to achieve goals -- are potentially a relevant area of research. We've already seen some transfer of code for efficient control loops from gaming, to controls for drones, self-driving vehicles, etc.) Even the most primitive animal has something like this -- prey animals are constantly trying to detect predators in the environment and predict their attack vectors, in order to escape.
Once you have an AI that is reality-grounded, _then_ you can start to layer on language, teaching it how to explain itself, and understand the intentions and requests of other agents. (Though of course this raises all kinds of interesting questions around the alignment problem. These explanations could be false both for reasons of willful deception -- the AI becoming trained to tell you what you _want_ to hear, regardless of the truth -- and just because explaining motivation is hard. _Humans_ are quite bad at understanding their own motivations, let alone interpreting other people's.)
Without attachment to reality, a language model has no way of understanding what is true or useful. As you say, it lacks any kind of grounding in a theoretical framework. It's just producing words that sound vaguely similar to its source material. It's Frankfurtian bullshit.
What I think is that the ChatBot does not have a theory of mind for you, it has only a theory of text. So if the response to the prompt "Why was humanity so poor back in the long Agrarian Age?" does not include the word "Malthusian", that is because whenever the text it was trained on - presumably Slouching Towards Utopia? - says that humanity was poor back in the long Agrarian Age, it does not usually mention that this was a Malthusian condition. When you wrote the text, you were thinking "Malthusian", because you have a theory about the Agrarian Age; and when I read that text, I was thinking "Brad is thinking Malthusian", because I have a theory of mind for Brad DeLong. But that's the difference between ChatBot and me.
Put in the terms you use below, the ChatBot can go to a corner of its training space, and its space is a very very high dimensional hypercube not legible to the human mind, so this allows it to surprise us; but it cannot create a corner of space where it was never trained.
It seems these modern transformers still have a problem of “forgetting” text and information given at a certain distance.