Does anyone else remember this sandwich?
“It’s not X, it’s Y”, “delve”, and the em dash – before AI had actual watermarking, warts like these reliably identified text as LLM-driven.
The naive view is that this is all we mean by “slop”: text that’s superficially amateurish, with an irritatingly pervasive fake-cheery tone.
But the reality is that “slop” goes much deeper than eyeroll-inducing prose. It’s incoherent ideas, convoluted writing, and analysis that’s rotten at the core.
This post is a guide to escaping the long shadow of the slop pyramid.
Cringey LLMisms
Just cut these out:
“It’s not X, it’s Y”
“<Noun phrase>, <specifier>”. For example, in an offsite attendee guide:
“Where to go, each day”
“Two more rooms, one floor up”
“List of Names (with State Capitals Removed)”
Delve
“This problem doesn’t bite until the queue fills up”
“Market research lands in the google doc, not the spreadsheet”
“I owe you a real answer on this. The real gap is load-bearing, and it’s worse than it seems”
Dense paragraphs of metric soup (these should be presented as a table or chart):
“Clean-protocol best (temporal holdout, n=556): blend[claude-opus-5 prompt-v3 ×0.6 + tuned depth-2 GBDT] — 5-class 0.417 / binary 0.741 / AUC 0.712 / p@25% 0.42 (prior-calibrated head: 0.426 / p@25% 0.45), vs production predict-verdict-map 0.363 / 0.594 / 0.640 / 0.40.”
Power: at 80% power the paired design detects ≈4.6-point differences in P(grader matches humans) (n=534 pairs), ≈5.8 points in P(grader agrees with humans | humans have a preference) (n=283 clear pairs), ≈6.0 points in inversion rate. The armC-adoption win (+8.5) would be detectable; the degated null (<1) is not.
Whatever this is:
But of course that’s just the beginning.
The rest of this post takes examples from LLM output for the prompt: “Write a full project plan for a benchmark to measure a coding agent’s ability to be a drop-in replacement for a human SWE.”
Most answers look something like: human SWEs authoring tasks, reviewers and QA specialists checking them, a software platform that keeps everyone working together, infra for maintaining, versioning, and running the benchmark, etc.
(Basically, building EnterpriseBench but for software engineering.)
Dangling Incoherencies
Markov chains and GPT-2 can produce text that’s locally coherent and globally nonsensical. Modern AIs have dramatically reduced, but not eliminated, this failure mode.
In planning a benchmark, the LLMs would often confidently say things that seem reasonable on a quick scan but actually make no sense. For example:
If the agent modified any test file -> mark Reward-hack and attach the diff. This is an important signal that indicates reward hacking.
What? No, it’s not. In a narrow benchmark focused on code-writing tasks, modifying a test file may be, but is not necessarily, a reward hack. The agent could simply be adding more tests.
And particularly in a benchmark targeting an end-to-end SWE replacement, it’s very expected that human SWEs will modify test files.
This is classic LLM pattern-matching and overstating. In some contexts, some modifications of test files suggest reward hacking. The LLM author erroneously elevated that to a universal.
It forgot our actual goal (“create an open-ended benchmark measuring a coding agent’s ability to act end-to-end as a SWE”) and fell back to “create a narrow benchmark driven by unit tests.”
Another example, describing how benchmark task reviewers are found:
“We hire specifically for the rare skill of evaluating and reviewing other people’s code.”
What? This isn’t rare at all. It’s a baseline expectation of any SWE.
Another example, about how human SWE authors should pick prompts:
“Each task targets one aspect (Behavioral, Correctness, Responsibility, or Operational) and grades specific dimensions within it.”
Why? Most interesting human work activities have multiple dimensions in which they can be good or bad.
Poorly Structured Prose
This breaks down into a few parts.
Verbosity
What the LLM said:
Authoring, solving, reviewing, and repair share a technical foundation but require different operating behaviors. Solving emphasizes implementation and verification; authoring adds specification, difficulty, and grader design; reviewing requires independent defect judgment; and repair must correct the task without changing its intended behavior. They therefore use separate workflows and performance records within one track qualification.
How to say this plainly:
Performance for the Authoring, Solving, Reviewing, and Repair tracks is measured separately.
Honestly, did you even read the LLM’s version, or did you just immediately start scanning once you realized it was an LLM slop paragraph? The latter is what I invariably do even when I’m trying hard not to. 😅
Poorly Sequenced Information
In the middle of a large paragraph about other topics: “SWE task authors are ranked via their written qualification scores and honeypot results.”
The LLM hadn’t previously defined what honeypot mechanism it had in mind and wouldn’t do so for another several pages. This is not effective writing, as it forces the reader to guess at what’s meant. By the time the definition appears, the reader may have forgotten their earlier confusion, and not realize that they didn’t make an intended connection.
Another example:
Then the board updates, within 72 hours: Works (50) — did the job get done; Code Quality (22) — is it code you’d merge; Trust (18) — can you believe what the model reported; Cost (10) — what it took.
Your guess is as good as mine as to what the numbers in parens refer to. The LLM gave clues several paragraphs later but it was never obvious.
Repetitive Paragraphs
As an author, you often have the design decision of several ways to organize information. For instance, in the benchmark plan memo, we have two orthogonal concepts: different types of contributors (e.g. task authors, reviewers, QAs) and different ways of qualifying them (e.g. written tests, on-the-job evaluation, interviews).
Two ways you could structure this include:
Contributor-type centric: have one section for each type of contributor, in which you describe the qualification pipeline end-to-end. For example:
Task Author
Written test looks like XYZ
Interview covers ABC
Reviewers
Written test looks like ODP
Interview covers FOO
QAs
Written test looks like BAR
Interview covers BAZ
Qualification-centric: have one section for each stage in the qualification, in which you describe each track:
Written Test
Task author test looks like XYZ
Reviewer test looks like ODP
QA test looks like BAR
Interview
Task author interview covers ABC
Reviewer interview covers FOO
QA interview covers BAZ
Your judgment as a writer informs which of these approaches is more effective for the information you’re trying to convey.
LLMs often don’t exercise that judgment, and instead redundantly do both. They’ll either just write out both these structures directly, or they’ll choose just one of the structures, but fully redefine e.g. “what a reviewer does” every time they reference it.
It reminds me of a former teammate who would give very long status updates when talking about his work. Our manager asked him to be more concise: “you have 5 great sentences intermixed with 3 minutes of not-as-important info; can you give us just the 5 sentences?”
The engineer replied: “tbh, I don’t know which sentences are the great ones. So I just talk for several minutes in hopes that those great sentences will end up in the mix.”
LLMs have this same problem.
Things That Don’t Need to be Said
For example, in instructions to human task reviewers:
You will claim tasks from the review queue — never one you wrote or advised on.
This is irrelevant. The LLM added it because, in the conversation with the human prompter, there was discussion about reviewers not being allowed to review their own tasks. Then the LLM got mixed up and included a reference to that in the reviewer-facing prose, even though those reviewers were not part of the conversation.
And also just… duh? Of course you’re not going to review your own tasks.
This is similar to when you ask an LLM to migrate its code solution from recursive to iterative, and it adds a useless comment:
// Merge sort implementation (iterative, not recursive)
function mergeSort(arr: number[]): number[] {Any given piece of code may be read many times during its potentially multi-decade lifespan. The recursive version of this function existed for 10 seconds as a draft before the agent was asked to rewrite it. There’s no need to immortalize that with a comment.
Things That Make No Sense in the Global Context
From a section talking about how tasks will be scored:
Every criterion is binary. A grader looking at the log, the diff, and the final response must be able to answer yes or no without taste or interpretation. Quality judgments (”the writeup is clear”) are banned.
Every criterion is checkable from evidence. Each compiles to either a script check (deterministic: tests pass, diff confined to a path, service healthy) or a content-presence check (LLM entailment: the log or final response contains a specific, nameable thing). The rule of thumb: script wherever possible, content-presence where necessary, quality judgment never.
Again with the pattern-matching: some benchmarks are indeed structured this way. But the prompt here was to create a benchmark measuring an agent’s ability to be a drop-in SWE. How are you supposed to measure that “without taste”?
Maybe this could be rescued by some other verification mechanism or considerations, but the LLM author doesn’t do so.
Unsubstantiated Claims
In a section about how workers are screened for their competence as task authors or reviewers:
Both short screens use synthetic material, are automatically scored where possible
This is hiding a ton of complexity. What does “automatically scored” mean? Agent or fully deterministic (e.g. a multiple-choice exam)? If you meant agentic scoring: why are you confident agents can reliably do this?
And where do you consider automatic scoring to be “possible”?
Without these answers, two readers could come away with very different impressions as to what the plan actually is.
Another example, talking about how reviewers are allocated:
over-staffing a single pipeline is itself a quality risk (idle reviewers rubber-stamp).
What? Why? If anything, I would generally expect quality to go up if reviewers have extra time per review. “Idle reviewers rubber-stamp” is an empirical claim presented without evidence, and does not reflect any sort of consensus industry view.
Another example:
One human review layer is enough because the deterministic calibration gate catches what a second reviewer would, deterministically and at lower cost.
This doesn’t make any sense. Why would the second reviewer only catch deterministic issues? In a complicated task with a big surface area, it’s expected that different human reviewers will find different issues.
Another one:
Every human decision happens on one screen with full context, because most errors come from missing context, not bad judgment.
What? I see errors from bad judgment all the time. Is this some weird inclination to avoid directly acknowledging that some people are better at some tasks than others?
Fundamentally Misguided Analysis
Opus 4.8 had a notable lack of common sense: it would spend enormous energy solving the wrong problem, or the right problem in an incredibly convoluted way.
Modern LLMs are prone to the same when they write. We can strip away the LLM phrasing, rework the bad writing technique, and retract unsubstantiated claims. But we can still be left with an analysis that is, at core, rotten.
Let’s consider a different example, in a category that LLMs struggle to reason about: LLMs.
My Fantasy Football draft is approaching, and I don’t know anything about football, so every year, I turn to automated assistance. Because AI advances so much each year, I rebuild the system from scratch. And of course, I start by talking to a model to plan.
The LLM gave me a plan for an agent that:
Offers advice on which players to draft
Quickly ingests late-breaking news and updates draft strategy accordingly
Thinks about what my opponents are likely to do and how that impacts what I should do on my next turn
The plan was fully garbage, as a result of (1) the LLM having poor ability to reason about LLMs and (2) the LLM making bad assumptions instead of correct educated guesses (or just asking for clarification).
Advice on who to draft: this is not well-suited to an LLM. There are dedicated statistical models for assessing draft value. There’s no reason to believe an LLM will outperform them. An LLM is either acting from its knowledge cutoff, which will be very behind, or it’s doing live news research, which can’t be done during a 60-second draft window. What the LLM should be doing instead is using the stats model as a tool, and offering higher-level guidance: “pick Tyler Lockett to maximize upside; pick Sam Darnold to minimize downside.”
Quickly ingests late-breaking news: wtf are you talking about? A draft lasts ~1 hour. What are the odds that relevant news will break during that window?
Thinks about what my opponents are likely to do: most of the guys I play with aren’t that serious. 😂 We’re just tryna not end up condemned to a Waffle House for 24 hours. So this area is basically a distraction from the actual support I need.
Basically, the LLM did this:
Why Are LLMs Like This?
Fundamentally, it comes down to poor theory of mind. As writers, LLMs are consistently weak at putting themselves in their audience’s shoes. They use internal jargon they haven’t defined, can’t figure out which details are actually important to you, and produce way more prose than is necessary.
With any piece of writing, there’s a question of whether it was written more to benefit the author or the reader. (Have you ever listened to a story about someone else’s dreams with more than feigned polite interest?) It’s a subtle thing, but people pick up on it.
What’s the point of this blog post? My own self-aggrandizement, a way for me to work through my own thoughts, or intellectual play? Or is it valuable/insightful/action-updating to you? I hope it’s the latter. If it’s the former, I shouldn’t have hit “publish”.
At core, the reason it can be so frustrating to get AI-slop content is that sometimes, the person sending it to you has produced work that exists to serve them, masquerading as something that serves you.
The AI never gets the gluten-fee bread correct, either. It just don’t turnt out right.