My view: MAMLMs—Modern Advanced Machine-Learning Models have significantly less in the way of parsimonious truth-based internal representations of the world than their boosters claim...
I had a funny hallucinating event with ChatGpt 4. This event illustrates an interesting property of a clueless child.
To set the stage, you can ask ChatGpt to rewrite something in the style of another writer or a style of a different age.
So, you an ask ChatGpt to rewrite "Mary Had a Little Lamb" in the style of Ernest Hemigway and get: "Mary owned a lamb. It was white. It followed her. Even to school."
Then I asked ChatGpt to rewrite "Mary Had a Little Lamb" in the style of the King James Bible, and got the following first line of the rewrite:
"Mary begot a lamb"
No, Mary did not give birth to a lamb,
even though, in the rest of the poem, the lamb acted like a child.
"If this question were asked of a human on a discussion forum on the internet" doesn't seem accurate unless that's the only source of training; is that right? I thought (based on reading that isn't very deep) that the objective for a language model like these is to have as much text from a wide number of sources as possible. Chatbot responses don't take into account the context of the source material, do they?
Dr. Delong says that he can make Klein's concerns "vanish." But I doubt that is even intended as a criticism of Klein. Maybe Delong just crystalizes what Klein's concern even is. I tripped over the formulation.
Separately, Dr. Delong, do you think that piece in Joule setting an upper limit on the likely energy requirements of AI is too optimistic? It seems like you are inclined to multiply a lot of things by a hundred.
I had a funny hallucinating event with ChatGpt 4. This event illustrates an interesting property of a clueless child.
To set the stage, you can ask ChatGpt to rewrite something in the style of another writer or a style of a different age.
So, you an ask ChatGpt to rewrite "Mary Had a Little Lamb" in the style of Ernest Hemigway and get: "Mary owned a lamb. It was white. It followed her. Even to school."
Then I asked ChatGpt to rewrite "Mary Had a Little Lamb" in the style of the King James Bible, and got the following first line of the rewrite:
"Mary begot a lamb"
No, Mary did not give birth to a lamb,
even though, in the rest of the poem, the lamb acted like a child.
Where does electricity cost $0.05/kWh? I pay about $0.21/kWh to have it delivered to my house!
"If this question were asked of a human on a discussion forum on the internet" doesn't seem accurate unless that's the only source of training; is that right? I thought (based on reading that isn't very deep) that the objective for a language model like these is to have as much text from a wide number of sources as possible. Chatbot responses don't take into account the context of the source material, do they?
Dr. Delong says that he can make Klein's concerns "vanish." But I doubt that is even intended as a criticism of Klein. Maybe Delong just crystalizes what Klein's concern even is. I tripped over the formulation.
Separately, Dr. Delong, do you think that piece in Joule setting an upper limit on the likely energy requirements of AI is too optimistic? It seems like you are inclined to multiply a lot of things by a hundred.