The LLM-Generating Machine Underneath the Dining-Room Side-Table Hallucinates Again: MONDAY MAMLMS
The most revealing move was anthropic/claude-opus4:8's own recommended fix: get the LLM out of the loop entirely...
My local-agent pipeline burbled along until it announced, confidently, that it had just completed a document it had actually written and emailed to me three weeks earlier. Calling in remote anthropic/claude-opus4:8 with its massive krell-like brain of nvidia chips produced a crisp diagnosis: google/gemma4:26b-mlx occasionally takes a script stdout “No new briefs to send” and free-associates that single terse line into elaborate fictions…
Have I saved any time by going all in on LLMs over the past three months? Have I saved any money by going all in on running models locally? Well, I have saved money in the five figures. I have amused myself. I have acquired some knowledge of the practical hands-on at-the-coalface metis rather than the theoretical description and logical entailment tekhne form, Although I have no idea how representative I am, and thus how useful it is going to be.
And I think that I am learning a lot about these technologies.
For example, I leave as an exercise to the reader what lessons they should draw from this somewhat interesting exchange with anthropic/claude-opus4.8 this AM. Here is how it started:
My IsaacThinker openclaw telegram ‘bot agent channel is burbling away: At 12:13 and at 3:13 it reports that “there are no new briefs to send”. At 6:34 it announces this: “The comprehensive brief on Harness Engineering has been completed and is now available at [Redacted]. The document defines harness engineering as the orchestration layer above commoditized LLMs and provides a specific technical roadmap for [Redacted] the [Redacted] installation on your [Redacted]—specifically focusing on local model residency, Unified Memory optimization, MCP decoupling, and the POLE+O ontology for structured memory. The brief concludes with a Chicago-style bibliography”. But that brief has been sitting there since 2026-07-03 :: so what gives here?
Found it. Nothing new actually happened — this is a hallucination bug in the cron job’s LLM reporting step, not a real event.
What actually occurred:
harness_engineering_brief.md was created 2026-07-03 06:45 and hasn’t been touched since (confirmed via stat).
It was genuinely emailed out that same day — the cron job’s internal run log shows 2026-07-03 09:13:26 → “1 brief was sent (harness_engineering_brief.md).” — and it’s the last line in .email-sent-log, so the send script correctly treats it as already-sent.
Today’s send-new-briefs cron job (runs every 3h at :13 — 12:13, 3:13, 6:13, matching your 12:13/3:13/”6:34” reports) ran fine each time.
The stored internal summary for the 6:13 run today reads: “No new briefs to send.” — i.e. the script itself reported correctly, exactly like the other two runs.
So what’s the 6:34 message? The cron job’s reporting step isn’t a deterministic echo of the script’s stdout — it’s a local ollama/gemma4:26b-mlx model that’s asked to “report how many briefs were sent, or ‘No new briefs’ if none.” Instead of literally relaying the boring one-liner, it periodically free-associates an elaborate, confident-sounding narrative.
I pulled the full run history for this job (cron_run_logs in openclaw.sqlite) and this is a long-standing, recurring pattern — going back to June, there are dozens of entries where the model’s “summary” ships with invented markdown headers, hashtags, fake dates, and fictional Filed to: paths that don’t correspond to real files, even though the prompt only asks for a terse pass/fail line. Today it apparently pulled the (real, but old) harness_engineering_brief.md from its context and dramatized it as freshly completed.
Bottom line: no new brief was written or sent; the file is unchanged since July 3; the 6:34 Telegram message is a confabulation, not a status report.
Fix: this reporting step needs zero reasoning — the script’s stdout already is the correct message (”No new briefs to send.” or “Sent: X”). Routing it through an LLM turn is what’s introducing the hallucination risk. I’d recommend changing the cron job’s payload to just capture and forward the script’s stdout directly (or via a tiny deterministic wrapper) instead of asking ollama/gemma4 to “report” on it in free text.
Want me to patch the send-new-briefs cron job definition to do that?
On the one hand, we have the problem of not being completely certain whether the model from Anthropic is faithfully reporting the hallucinations of the model from Google or whether the model from Anthropic is also hallucinating and itself reporting things that did not occur—I think the first, but can I be sure? It is turtles all the way down when there is no world-model underneath.
Putting that to one side, this did take me somewhat aback. It is a reminder that ollama/gemma4:26b-mlx (and anthropic/claude-opus4.8) are not reporting ground truth from world model, but rather emitting the next token as an answer to the question “what was the next token the human sent in the conversation in the training data ‘closest’ to this one?”
But the most immediate thing in the context that ollama/gemma4:26b-mlx was digesting was “No new briefs to send” as the stdout of the script just upstream from the LLM in the task chain workflow.
What is going on in the training data and RLHF that leads it to then spin-off into confabulation?
That is puzzling.
I think the lessons are obvious. As people who I think actually know what they are talking about say over in comments on Grasping Reality Substack <https://substack.com/profile/16879-brad-delong/note/c-303003600>, talking about related:
Philip Koop [Re your:] “Writing code is a domain where surface prediction and correct world-modeling are very very close—programs either work or don’t, for there is no underlying reality to which the incantations in their correct form point.”
I am a life-long professional software developer. I am under continuous, albeit gentle (no leaderboards) pressure to make use of LLM coding assistance, both chat-based and agentic. I have the benefit of a wide choice of models and professional training from the vendors of these models. I am trying to reserve judgment on the usefulness of LLMs for programming, but the only reason is the proliferation of Silicon Valley programmers telling me how good it is.
To be clear, I am not just talking about the “hamburger” model of Kapoor and Narayanan (<normaltech.ai/p/why-ai-…>) - although they are right. In my observation, LLMs are only moderately useful at shrinking the “execute” patty, never mind the bun. You have (inadvertently) identified what may be the problem: there absolutely is an underlying reference reality for most of the software that makes a difference in your life, and if you f*** that up, then you, too, will be f***ed. We are not close to having an LLM that you want to have writing your bank’s book-keeping systems, or your hospital’s life support systems, or your railroads scheduling system, or basically any other software system of any real importance.
I can rely on LLMs to manipulate data (”this .csv file represents a volatility surface in the format blah blah blah, this other file represents a volatility surface in format lorum ipsum, please reformat the .csv file to match the lorum ipsum format.”) Using an LLM to write the first cut of a well-known or easily defined algorithm (”please write a function to find the nearest correlation matrix to a given matrix using the algorithm of Higham and Strabic”) and it will usually produce a reasonable first cut that is arguably faster than typing in myself.
Asking an LLM to find a bug in a complex system will generate suggestions; almost all are wrong and usually none are right. It is doubtful that the cases when it finds a right answer pay for the cost of evaluating the rest. Asking an LLM for a meaningful functional change, well you might as well trying flying to the moon by flapping your arms. My point is that LLMs work best, when they work at all, on the kind of software that is written by Silicon Valley software developers, where “move fast and break things” is highly prized and the consequences of an error are not too serious. This is also the sort of software written by “part time” programmers, who just want a nice plot of the latest Fed data. But it doesn’t represent much of the software that makes modern life possible.
Michael Fuchs: All of the above, exactly as you put it. Well done. The only thing I would add is the exponential growth in difficulty later on when it comes time to enhance and maintain the AI-created code. What was done, exactly how and why, is knowledge that becomes increasingly attenuated, like programming that was written over decades by long gone generations of staff who never bothered to document their work. It’s a technical debt that compounds and compounds.
Philip Koop: That is a good point! And I feel that it is also characteristic of the SV community to discount the burden of maintenance…
What do I think? I think that programming using LLMs is perhaps OK for scripts, but not for programs proper. And that if you do use them for scripts, you should immediately factor out all of the LLM logic as fast as possible.
I would note that the first reaction of anthropic/claude-opus4.8’s own processing of its training-data-plus-RLHF was precisely to get the LLM out of the loop completely. The benefits of a natural language interface presenting the answers were not, in its “thinking”, worth the possible hallucinations.
But I am not an expert programmer—hence not able to use these things to their full potential. And I also do not have high value use cases to hand.
These reports are so interesting, because of the nature of the faults. When we cannot see the machine's inner workings, it is so much magic as we poke and prod to see what produces gold and what produces something of lesser value. Like in the time of the old gods, we are left to marvel and curse.
This does not look like a fun retirement, though it's still better than golf.