Chat-GPT as Clever Hans Once Again
It flubs on what my general orientation thinks ought to be a relatively easy task for a near- but sub-Turing intelligence that can do internet searches...
Something that you can find on the internet and stitch together from three screenshots of a web post at one URL <https://www.anandtech.com/show/21116/apple-announces-m3-soc-family-m3-m3-pro-and-m3-max-make-their-marks>:
is beyond the powers of ChatGPT4 with its web-search module turned on:
What I had wanted to do was gain a sense of whether I should be thinking of M3 Base, M3 Pro, or M3 Max chips for my next machines. In the M1 and M2 generations, the Mx Pro was an Mx Max with the GPUs and such cut down—looking as though it was a binned version—if the chip was flawed only in display and memory interfaces and GPU, but enough was working, it would be binned as a Pro. Now it looks as though it is a separate chipmaking track designed to be more of a middling alternative to the Base and Max versions. And ChatGPT could not help me: so I took the first result from a normal Google search, and got lucky on AnandTech.
Another of a huge number of cases reinforcing my belief that ChatGPT is best viewed as language interface to noisy web search, and a useful tool to those who understand that. Attributing more to it is to fall victim to the “Clever Hans” delusion.
And, indeed, it does well on this:
But it is not because it has any self-awareness or intelligence beyond being a stochastic parrot, does it?
I am not seeing how ChatGPT is as clever as Clever Hans. It can't, in fact, answer correctly when I know the answer!
Ah, you may say, that is because it cannot see me. But it can never see me! It never picks up on subtle cues from my demeanor.
It's more like ... if you could do a string search on a large history of NSA communications interceptions, and regurgitate the context from randomly selected near matches.
ChatGPT is a useful tool for the otherwise-talentless otherwise-unemployables.