How Chat-GPT boosters are deluding þemselves in thinking þt it is much more þan it is—but þt does not keep it from being, potentially, very useful indeed. A stochastic parrot is a useful assistant...
IMO, the best use of LLMs is to create a human conversational interface, rather than used to extract useful information. IOW, think of LLMs as Kahneman's "System 1" thinking - the sort of daily verbal I/O between humans to smooth "social transactions".
We seem to have been led down the path of pattern recognition ML because of the failure of GOFAI to be able to deal with complex data, e.g. images. As a result of the success of the Deep Learning neural network architectures, this approach has been pressed into service for achieving more cognitive roles than is, IMO, warranted. Consider all the old GOFAI goals that depended on logic that are now ignored in favor of pattern matching. Because expert chess and GO players do see patterns in board positions to make decisions on play, this has bolstered the neural net approach to game playing, which is probably perfect for situations that require quick, i.e. systems 1 responses. However, humans, particularly those who have been educated to use other forms of thinking that take more effort, i.e. Kahneman's "System 2", we really need these forms of thinking as the primary method when simple lookup response is not available. We are often told that we need "critical thinking skills" to evaluate information, and there are tools that can be used to support this. LLMs do not have these tools built in. What is needed is to apply these tools to retrieved [mis-, dis-]information to provide the best response. Computers should be able to do this very well as they can vastly exceed human cognitive capabilities in retrieval speed, accuracy, and handling many more pieces of information at the same time. We have to use prostheses like books, writing, etc. to manage complex tasks to reach good [enough] answers and results.
Therefore, I would conclude that we need to marry other ML tools, algorithms, and data retrieval (and even expert systems] to act as the more "reflective" part of cognition, leaving LLMs to parse the output in a way that humans can interact with through a Q&A session. Sophisticated analogy-making algorithms would be a great help in making complex issues understandable based on existing personal knowledge. Then we might even get useful intelligent systems, even embodied in robots.
You may be right. I envisaged the LLM being fed the raw data that is then turned into a sentence or paragraph. But this may not work with an LLM that works purely as a stochastic parrot as it will not have enough text paraphrase or parrot. Give it enough text and it may start to hallucinate.
An empirical test might be to feed a new MiniBradBot just raw data, e.g. facts, from your book or lecture notes and see if it can provide a cogent output wrapped in a sentence or two. That would be a test of whether it could be used in this way or not. Maybe someone has already done this and written a journal paper?
This suggests that LLMs are poor at generating SQL commands to retrieve answers. - which is a proxy for the first leg of an Q&A interface using LLMs. https://arxiv.org/abs/2305.03111
This amusing article on using ChatGPT to find a relevant passage from the works of Proust suggests we really need a better way to find the answers we want.
So if LLMs cannot build useful SQL queries, what would need to change or be built so that an LLM could interface with other tools, like databases, to return the required content and then turn it into output to mimic a human? Or are LLMs just a dead end in this regard?
regarding LLM's, maybe there is nothing wrong with the associations being made at the level of transformer training. maybe there needs to be more refinement at the level of more precise inferential output. in fact, it would be interesting to know if there can be training on appropriate output via the transformer.
On [9], I *almost* persuaded myself that "thinking outside the box" means much the same as "open-mindedness", which means much the same as finding a key nugget of fact, understanding it correctly, and placing it in its proper context - and I am the easiest person in the world for me to persuade. But no. You are right and I am wrong. The resemblance to near-understanding is purely epiphenominal, a mere coincidence.
Convcerning usefulness, isn't the question whether this stochastic parrot could be hired by Fox News or MSNBC, each with their own skewed LLM, to say made-up things many people would believe? Did I not listen to the Hexapodia podcast (which of course I do), I would have found what Bing was flinging to be plausible. As in other things, the coherence theory of truth killeth, but the correspondence theory giveth life - provided, of course, that you have fact checkers.
"It seems to me to requires much more and very different from merely “add[ing] some more layers and parameters and training and just generally throw a bit more compute at our model next time…”"
IMO, the best use of LLMs is to create a human conversational interface, rather than used to extract useful information. IOW, think of LLMs as Kahneman's "System 1" thinking - the sort of daily verbal I/O between humans to smooth "social transactions".
We seem to have been led down the path of pattern recognition ML because of the failure of GOFAI to be able to deal with complex data, e.g. images. As a result of the success of the Deep Learning neural network architectures, this approach has been pressed into service for achieving more cognitive roles than is, IMO, warranted. Consider all the old GOFAI goals that depended on logic that are now ignored in favor of pattern matching. Because expert chess and GO players do see patterns in board positions to make decisions on play, this has bolstered the neural net approach to game playing, which is probably perfect for situations that require quick, i.e. systems 1 responses. However, humans, particularly those who have been educated to use other forms of thinking that take more effort, i.e. Kahneman's "System 2", we really need these forms of thinking as the primary method when simple lookup response is not available. We are often told that we need "critical thinking skills" to evaluate information, and there are tools that can be used to support this. LLMs do not have these tools built in. What is needed is to apply these tools to retrieved [mis-, dis-]information to provide the best response. Computers should be able to do this very well as they can vastly exceed human cognitive capabilities in retrieval speed, accuracy, and handling many more pieces of information at the same time. We have to use prostheses like books, writing, etc. to manage complex tasks to reach good [enough] answers and results.
Therefore, I would conclude that we need to marry other ML tools, algorithms, and data retrieval (and even expert systems] to act as the more "reflective" part of cognition, leaving LLMs to parse the output in a way that humans can interact with through a Q&A session. Sophisticated analogy-making algorithms would be a great help in making complex issues understandable based on existing personal knowledge. Then we might even get useful intelligent systems, even embodied in robots.
Yes. But it is not possible to use the LLM part only for the conversational interface. Gremlins and hallucinations sneak in...
You may be right. I envisaged the LLM being fed the raw data that is then turned into a sentence or paragraph. But this may not work with an LLM that works purely as a stochastic parrot as it will not have enough text paraphrase or parrot. Give it enough text and it may start to hallucinate.
An empirical test might be to feed a new MiniBradBot just raw data, e.g. facts, from your book or lecture notes and see if it can provide a cogent output wrapped in a sentence or two. That would be a test of whether it could be used in this way or not. Maybe someone has already done this and written a journal paper?
This suggests that LLMs are poor at generating SQL commands to retrieve answers. - which is a proxy for the first leg of an Q&A interface using LLMs. https://arxiv.org/abs/2305.03111
This amusing article on using ChatGPT to find a relevant passage from the works of Proust suggests we really need a better way to find the answers we want.
https://www.theguardian.com/books/2023/sep/05/proust-chatgpt-and-the-case-of-the-forgotten-quote-elif-batuman
So if LLMs cannot build useful SQL queries, what would need to change or be built so that an LLM could interface with other tools, like databases, to return the required content and then turn it into output to mimic a human? Or are LLMs just a dead end in this regard?
Hey, they said it was catchy!
I was thinking about adding that to the list of AI misses....
regarding LLM's, maybe there is nothing wrong with the associations being made at the level of transformer training. maybe there needs to be more refinement at the level of more precise inferential output. in fact, it would be interesting to know if there can be training on appropriate output via the transformer.
Do stochastic parrots dream of eclectic creep?
On [9], I *almost* persuaded myself that "thinking outside the box" means much the same as "open-mindedness", which means much the same as finding a key nugget of fact, understanding it correctly, and placing it in its proper context - and I am the easiest person in the world for me to persuade. But no. You are right and I am wrong. The resemblance to near-understanding is purely epiphenominal, a mere coincidence.
We are very good at attributing human-level thought to things that we would recognize, if we thought a minute, do not have it...
e.g. Animism and panpsychism.
Convcerning usefulness, isn't the question whether this stochastic parrot could be hired by Fox News or MSNBC, each with their own skewed LLM, to say made-up things many people would believe? Did I not listen to the Hexapodia podcast (which of course I do), I would have found what Bing was flinging to be plausible. As in other things, the coherence theory of truth killeth, but the correspondence theory giveth life - provided, of course, that you have fact checkers.
"It seems to me to requires much more and very different from merely “add[ing] some more layers and parameters and training and just generally throw a bit more compute at our model next time…”"
Bingo!