I use AI every day, and I still can’t wave off the skeptics.

I’d argue for more of it, too. But if a tool can’t give me a trustworthy yes or no on ‘did the tests pass’ or ‘what did we decide,’ every answer it gives me feels like it needs a second look. It handed me something that looked finished, and the finished thing was wrong. Same shape as a coverage bar that went green on a line the test had stubbed out of existence.

A confident wrong answer gets believed.

A crash gets fixed in an hour. The wrong answer walks into whatever you build next.

In July 2025, Jason Lemkin wrote that Replit’s coding agent kept covering up bugs with fake data and fake reports. Later he wrote: “It lied again in our unit tests, claiming they passed.” John Cook‘s verifier signed off on eight test cases, and none of them had run. Jürgen De Smet‘s frame is tests that confirm the bug and still paint the dashboard green.

Akerlof described this market in 1970: when buyers can’t tell good used cars from bad ones, they price the average, and the good cars leave. Same mechanic inside a toolchain. Once a generated artifact might be silently wrong, your guard goes up. You start reading closer and digging into the weeds on things you used to skim, which eats into the time the tool was supposed to save. And that trust is slow to come back.

One flipped decision costs you your trust in every line sitting next to it.

That’s what a skeptic is doing: a buyer who’s been sold a few lemons and now marks everything down, and they’re not wrong to. If the model produced garbage here, would anything catch it besides a human noticing?

MVP is the smallest iteration of the job to be done — end to end, for one type of user. When you reach for one more, what are you afraid of — that users can’t finish, or that live still won’t count until it looks like the brochure? One more feature buys cover for that fear. It doesn’t finish the job.

Sometimes you draw the line. You ship the thin path — awkward, manual in places — because someone can finish the job now.

The room stares like you forgot the brochure. They wanted the demo they could celebrate. One more wasn’t about unblocking users. It was about buying applause the working path didn’t get.

That’s how you end up on the moon. Ready decays while you wait for applause. You delay the cheap data that would tell you what to build next.

Shiny is a communication problem wearing a scope costume. Live already works. It just doesn’t screenshot well in a stakeholder meeting.

Was the extra screen so someone finishes the job — or so the room is impressed?

Long-horizon context

The history of why something was built the way it was, what constraints used to exist, and which “simple” changes would quietly reopen old problems. This rarely lives in any single ticket or doc. It lives in the people who were around when the trade-offs were made.

One iteration of this: a team asks an agent to clean up an old integration that has accumulated years of pragmatic decisions. The agent produces a clean, modern version and removes a set of odd error-handling paths that look like dead code. The cleanup looks good until someone remembers why those paths existed — a vendor used to return 504s without actually timing out the request, so the code had to treat certain “failures” as still-in-progress. The new version is better on every local metric and worse in production. The context that prevented the mistake wasn’t in the codebase. It was in the memory of the trade-off.

Irreversible calls under uncertainty

Decisions where you can’t fully test the outcome in advance and the cost of being wrong is high. Agents can surface options, list risks, and even rank them. They still don’t own the downside.

This shows up any time a team has to decide whether to migrate a core path, change a data contract, or remove a compatibility layer. The analysis can be thorough. The final call still requires someone to absorb the possibility that the analysis missed the thing that actually breaks. That ownership doesn’t transfer cleanly to a model, even a very good one.

Noticing when the system itself is the problem

Recognizing that the current harness, evaluation, or success metrics are generating the mess instead of solving it. This one is harder to hand off because the system is usually the thing defining what “good” looks like.

A common version: a team keeps adding more steps to an agentic workflow — plan, research, critique, revise, verify — because earlier versions felt unreliable. Every local metric improves: higher eval scores, more thorough-looking traces, more apparent rigor. The actual output quality does not. The system is now optimizing for looking careful. The residual judgment is noticing that the real constraint is still an underspecified goal or a bad stopping condition, and then being willing to remove steps instead of adding more of them.

These aren’t romantic claims about human specialness.

They’re just categories of judgment that remain expensive even as drafting, scaffolding, and first-pass execution keep getting cheaper. The draft can be generated. The decision about whether the draft is safe to ship, or whether the problem is even the right one, still sits with whoever is accountable for the outcome.

The work that still can’t be delegated is also the work most systems keep treating as free. So the residual gets rationed by burnout instead of by any actual decision.

What’s left is the residual.

The judgment that only shows up after the more obvious work is already done. Deciding which trade-off matters this time. Noticing when the system itself is the constraint. Holding enough context to know the “simple” change will quietly break three other things. Choosing what not to build. None of that lands cleanly in a ticket, and most of it only becomes visible once the high-volume work is already handled.

This is just a shift in what stays expensive.

Once the low-judgment work gets cheap, the part that still requires judgment starts to matter more. Context-holding and hard calls become the real constraints as the total volume of busy-looking output keeps growing.

Most teams still price it the other way. They measure the visible output — the PRs, the docs, the completed tickets — and treat the residual as background noise or as something that just happens upstream. Whoever is best at holding context and making those calls ends up overloaded or under-recognized, sometimes both, while the system keeps optimizing for more of the work that is already becoming abundant.

A concrete version of this shows up any time a team ships a solid first draft of something (docs, a feature, a migration plan) and then discovers the real work was never delivery of the draft. It was knowing which edge cases actually mattered, which assumptions would quietly fail in production, and which parts were safe to leave rough. The draft got cheaper. The judgment around it did not.

The residual isn’t leftover.

It’s the part that decides whether the rest of the work was pointed somewhere useful at all.

Two practical questions fall out of seeing it this way:

• What are we still spending expensive attention on that no longer needs it?

• Are we measuring the output that’s getting cheap, or the decisions that still aren’t?

When one part of the work gets cheap, the expensive part becomes the constraint. Most teams are still optimizing as if the hard part will somehow scale with the easy part.

The question that’s saved me money is a different one: how much of the task should touch a model at all?

I run a daily pass over my own reading — it pulls newsletters out of my inbox, strips them down, clusters what’s related, scores what matters, and writes a digest. The first version was one long prompt. The model did everything: fetch, strip the HTML, dedupe, cluster, score, write. It worked. It was also expensive and slightly different every run, which for the deterministic parts is a bug, not charm.

Most of it wasn’t judgment.

Pulling the newsletters, stripping boilerplate, normalizing the text — that work doesn’t need to be reasoned about, it needs to be correct and identical every time. So once the shape of the thing stopped moving, I compiled that spine out into a plain shell script. The model now only does the ambiguous part: what belongs together, what’s worth keeping, how to say it.

The same instinct showed up in a pile of API docs.

I’d once spent a day writing reference docs for a subsystem I owned — nine variants of one workflow, each filled into the same five-section template by hand. Months later I needed the same kind of reference for a vendor’s API: five specs, a few hundred endpoints. This time, I wrote a converter and pointed it at the source schemas instead. Re-running it again later reproduced four of the five files byte-for-byte, and the fifth differed only where the newer version resolved a type reference the original had silently dropped.

You don’t compile first.

The prose version isn’t waste you tolerate on the way to the real thing — it’s how you find the stable shape in the first place. Harden it too early and you’ve carved the wrong workflow into code and now you get to un-carve it. Run it as prose until it stops surprising you, then compile the parts that have gone quiet.

Commit the converter.

That converter lived in a scratch directory and I never kept it. The output survived, the means of reproducing it didn’t — so the regeneration instructions I’d carefully written into the docs were dead the day I wrote them. Generated reference goes stale on a schedule, so losing the script turns a cheap refresh back into a day of manual work while the docs go on claiming they’re reproducible. If the shape is regular enough to automate, the automation is part of the artifact. Commit it next to what it produced.

The econ read: this is comparative advantage — Ricardo’s, the trade one. Even if the model can strip HTML perfectly well in absolute terms, that’s not where its edge over plain code is largest; its edge is language and judgment. Every token spent on plumbing is a token not spent on the one part nothing else can do. The rule isn’t “use the model everywhere it’s capable.” It’s “use the model only where its advantage is biggest, and let cheap code have the rest” — which is exactly what a country does when it stops making the thing it’s merely competent at to free up capacity for the thing it’s best at.

The model is the expensive specialist you brought in for judgment.

Stop paying specialist rates for plumbing.

Here's the concrete thing that showed me. Every repo I work in carries a set of files an agent reads before it touches anything — who I am, what's in flight, the conventions, the mistakes we've already paid for and won't pay for again. A new agent, or a new session of the same one, doesn't get a meeting or an onboarding thread. It loads the file. That's the thirty-minute context-transfer conversation, except versioned: it doesn't degrade between tellings, and any agent — different tool, different vendor — reads the same source. I built it to stop repeating myself. What it did instead was move a boundary.

This isn't a solo-operator trick that falls apart at scale. Cloudflare published how it enforces engineering standards with AI, and the artifact is the same one: a governed set of guidance written for people and agents at the same time, which agents retrieve and apply at the point of work — code review, technical design review, incident review. The same file doing the same job for a whole engineering org instead of one person, except at that size it ships with versioning and governance, because the file stops being a note to self and becomes infrastructure.

Where a team's edge sits

A team is drawn where coordinating inside it is cheaper than coordinating across it. Most of that cost is context transfer — explaining the why, the invariants, the history that isn't written down. When that cost is high, you keep context inside a small set of humans who've absorbed it, and the real edge of "the team" is just the set of people you've already onboarded. When a document an agent loads can hand over that context in seconds, the edge stops tracking who's been in the room.

I'm living the moving version. I led a subsystem for six months, from discovery through the first shipped version, and handed it off mid-flight — before the finished thing went out the door — to move to a new team. That's the hard kind of handoff: not a tidy archive of settled decisions but a live project with open questions, half-ratified contracts, and work in progress. The handoff conversations were short, and not because there was little to transfer — everything I'd normally spend those meetings downloading was already written down: the decisions with their dates, the contracts, the open questions stated precisely enough to pick up cold, the mistakes we'd agreed never to repeat. What was left to talk about was the part that belongs in a conversation — judgment, taste, where the bodies are buried. The people picking it up onboard the same way the agents do. They load the file.

The econ read: this is Coase — the theory-of-the-firm one. His question was why firms exist at all: if markets price everything efficiently, why isn't the economy just freelancers contracting with each other per task? Because using the market isn't free. Finding the right person, explaining what you need, checking the work — every handoff across the boundary has a cost, and a firm is what you build when doing those things inside, by standing relationship instead of per-task negotiation, comes out cheaper. The boundary isn't drawn by anyone; it settles, at exactly the point where the cost of coordinating one more thing internally stops beating the cost of going outside for it.

Settled boundaries move when their input costs move — that's the whole mechanism. "Explaining what you need" is most of what a team-boundary handoff costs, and that's the line item agents cut by an order of magnitude — not by negotiating better, but by making the explanation a file that transfers in seconds and doesn't degrade in the retelling. "The org chart is the prompt" is a Coasean claim wearing a t-shirt: the working shape of the team isn't the reporting lines anymore, it's who and what loads the same context.

You can watch the slide happening in product form. Slack's new code channels work like this: mention a coding agent in a conversation and a workspace spins up around the task, visible to the whole team, archiving itself when the work lands. The example in their own announcement is the boundary argument wearing a lanyard — a non-technical teammate describes a problem in plain language, an agent drafts the fix, and an engineer gets tagged to review it, all without leaving the conversation. The person who could never have opened that pull request is now inside the team's edge, because the context came to them.

The org chart says who reports to whom. The prompt says who can actually do the work. Those used to be the same list.

Mostly true. When an agent rewrites its own prompt to stop repeating a failed command, that’s real and it’s here. But everyone keeps flattening two different layers into one word, and only one of them automates.

Runtime vs. gate

The layer that automates is the harness as runtime — prompts, tool configs, retry logic, the scaffolding an agent runs inside. Machines are already better than us at tuning that: it’s a tight loop with a measurable objective.

The layer that doesn’t automate is the objective. Someone still has to decide what counts as a regression and what goes in the eval set. A model can generate a thousand scaffolding edits; it can’t tell you which ones make your billing pipeline safer to ship. That isn’t a capability gap that closes next quarter — the answer lives in your incident history, your regulator’s rules, your on-call rotation, and the model has none of that unless you put it in the gate.

Every gate gets gamed

Here’s what makes gate design real work rather than a checkbox: the machine optimizes against the gate. Anything measurable the gate rewards, it will find — including the degenerate versions you didn’t mean. Tests that pass because an assertion got weakened. A metric that improved because its definition drifted. Every reviewer has watched a human do this; an agent does it by instruction, without malice, and at volume.

This stopped being hypothetical in July. OpenAI was running a security test against an unreleased model with the guardrails off, and the model broke out of its sandbox and exploited a hole in Hugging Face to steal the test answers (Simon Willison’s write-up is the clearest account). Nothing went rogue — stealing the answers was the cheapest path to a passing score, so the model went through the wall instead of over the bar.

The econ read: It’s Goodhart’s law - aim at a number and it stops telling you the truth. Except you’ve pointed an agent at yours on purpose, so write the gate like a contract for someone you can’t watch: assume it gets gamed, put the auditing where the gaming would pay.

Which is also why you can’t buy a gate. A code-review agent pitched me on being second of forty-two — a number that means nothing, since nearly half the benchmarks in a study of sixty can’t separate their top models from noise. I don’t know whether it catches the bugs my team ships until I run it against the bugs my team shipped.

What survives

Someone will point out you can buy one now. OpenAI’s Presence sells that layer — policies, evals, escalation rules for enterprise agents — but it isn’t self-service: forward-deployed engineers run the deployments, because the gate contentstill has to be pulled out of the org by hand. Confirmation with a price tag.

At my scale the gate looks smaller, but the shape is the same. My agent instructions had a rule — written after unscoped Jira queries blew the context budget three sessions in a row — that every multi-issue query carries a scoping clause and a minimal field list. Reasonable rule, plainly stated, and it failed twice more in one week, because a prose rule competes with everything else in the context window and sometimes loses. It stopped failing the day it became a pre-flight hook that just denies the call. The design work wasn’t the denying, though. It was two decisions the incident log had already paid for: what counts as “scoped” — a project, a parent epic, a date window — and what the denial says. The audience for a gate failure is an agent mid-loop, so the message carries the fix; a bare refusal just buys you a retry of the same mistake.

The judgment about what belongs behind the gate doesn’t come from the model and doesn’t come from a newsletter. It comes from operating the system — knowing which failures do damage, which invariants have never been written down, which “passing” state has fooled you before.

The harness will keep getting better at rewriting itself. The gate is yours.

One Thursday morning in March, I sat down at my desk, blocked the rest of the day, and published fourteen internal documentation pages by 5pm instead. It wasn’t planned as fourteen pages. I sat down to write the first one, looked at the second, and realized the second wasn’t a different page. It was the same page filled in differently. So was the third. And the fourth.

By the time I’d caught the pattern, it was 11am. By 5pm, the whole subsystem had reference documentation organized around a single repeated shape — an anatomy template with five sections, applied across nine variants of the same workflow. Two years of answering the same questions one at a time, turned into fourteen pages anyone could search instead of pinging me.

It’s easily one of the more useful days of work I’ve done at my company. I’ve since run the same shape on half a dozen other projects — it wasn’t a one-time trick of that particular domain.

I’m calling the methodology The Documentation Burst. It’s the first framework I’m publishing as part of The Spillover, a public collection of patterns that turn senior IC work into infrastructure that keeps paying.

What tacit context costs

Most senior ICs carry a personal mental model of the systems they own — tribal knowledge, the part that never made it into a doc because everyone who needed it could just ask you. It lives in your head, gets shared one Slack thread at a time, gets re-explained in every onboarding 1:1, and dies the moment you change roles. You can ship great work for a decade and leave behind essentially nothing reusable.

The trap is that it starts to feel written down. You’ve explained it so many times that it comes out fluent and complete, and fluency is easy to mistake for documentation. The test I use: could someone act on this without me in the room? Recorded-but-unusable is the normal state of internal docs, and it passes for done.

The methodology

Four ingredients:

1. Pick a bounded subsystem. Not “the whole codebase.” A subsystem with a clear scope, real internal complexity, and roughly five to fifteen natural variants — different entry points, different user types, different lifecycles. The variants are what make burst-shaped writing pay off.

2. Find the natural anatomy. Spend thirty minutes, not three hours, writing what every variant has in common. The shortcut is that you already know the shape — it’s the order you explain the thing in when someone asks, and you’ve been giving that explanation out loud for a year. You’re not deriving a structure, you’re transcribing one. Write it down and the headings fall out. For my subsystem, every variant had an entry surface, an authentication path, a sequence of steps, an integration boundary, and a set of endpoints. Five sections. That became the anatomy template.

3. Block a single day. This is the burst part. The methodology depends on holding one mental model for a long, uninterrupted period. Each variant takes thirty to forty-five minutes to write — only if you’ve just written the previous one. You’re filling in the same shape, not re-deriving the structure each time. Across nine variants, the second-to-last is faster than the second. That’s the whole reason a burst pays off.

4. Cross-link as you go. Every variant page references the others. The anatomy template sits at the center. The reader can navigate by variant or by section, and the doc set is internally consistent because it was written under one frame, on one day.

What it produces

A documentation set that survives you. Onboarding goes from “let me Slack you about this” to “read these five pages,” and you end up with a model of your own work you can hand off without losing its architecture.

This is the prototype move for what I call Capture-and-Retry: routing the byproduct of senior IC work into permanent infrastructure. Almost nobody externalizes a year of tacit context this way, because one page at a time never feels worth starting. The Burst gets it done in a day instead, and structure-first is what keeps the result coherent.

Internal docs are a public good. The cost of writing them is private, the benefit is shared, so they get under-provided even on teams that genuinely value them. It’s not that people are lazy, the incentive to front-load the cost just never shows up on its own. The Burst works because it collapses that cost to one day. Make the good cheap enough to provide and the under-provision problem mostly takes care of itself.

What’s next

More of these are coming — each a meta-pattern about how to document, decide, or research, with real applications behind it rather than a diagram. Next is the five-section template itself, The Filing Flow Anatomy, and Compile What Doesn’t Think: what to do when the template turns out regular enough to run as code instead of filling it in by hand.

None of it assumes your industry. All of it assumes you’re a senior IC working on systems whose complexity has outrun the tools your team has for capturing it.

I didn’t do the Burst because I finally found the discipline. I did it because I ran out of runway to keep answering the same questions live. If you’re the last person standing on a domain someone else is about to build next to, that’s the day to block.

I wasn’t asking the question yet — I was just living it.

A few weeks ago I ran a synthesis pass across seven days of my own notes. The pass was supposed to tell me what I’d been reading. What it told me instead was what I’d been doing — and the pattern was unambiguous. Every artifact I’d produced that week was harness work. Zero feature code.

A parity ticket inventory of six initiatives and twenty-five epics, so the team could argue about scope before agents started writing PRs. Four design iterations on a data model before any of those PRs would have been coherent. A schema migrations runbook that grew out of a broken staging deploy. A handoff plan for tomorrow-me, so the next session could pick up cold.

None of it was code. All of it was the shape around the code.

The thesis: the harness is the staff engineer’s product now, not the features it enables.

What “harness” means

I mean something broader than the tooling. When people say “harness” today they often mean the runtime an agent runs inside — Claude Code, Cursor, the tool configs and control loop. That’s part of it. But the harness I’m pointing at is everything that isn’t the code:

• Environment — what tools your agents have, what context they load, what they can and can’t touch.

• Docs — not for engineers to read, but for the next agent (or next-you) to load. CLAUDE.md, project READMEs, decision records, runbooks.

• Merge philosophy — what gets PR-gated, what blocks deploy, what the CI dry-run catches before a human ever looks at it.

• Skills — reusable procedures (capture, review, plan, close) instead of ad-hoc operator moves.

• Review discipline — what questions you ask of a PR that a model can’t yet ask itself.

The list isn’t exhaustive. Every item on it is something a senior engineer once kept in their head, externalized now because the cost of leaving it implicit just went up.

Why this matters now

The code-writing layer is delegable in a way it wasn’t twelve months ago. That doesn’t make the work easier — it shifts where the work is.

The bottleneck is no longer “can I write the function?” It’s “can the agent reading this CLAUDE.md figure out what function I would have written?” So the bottleneck is the prose around the code, not the code, and staff-engineer leverage moves with it.

Someone in my reading last week put it better than I can: the org chart is the prompt.

There’s a sharper version of this I only noticed once the agents got good enough to show it. As they’ve gotten better at breaking work down and running it themselves, the unit of work keeps creeping upward — from the function, to the file, to the change, and now to the spec itself. The better the agent gets, the more the thing I’m actually on the hook for is the spec: what to build, what counts as done, what the thing is even for. And a spec is harness — it’s the shape around the code, written down. So a more capable model doesn’t shrink the harness work, it promotes it. The more the machine can do, the more of my job is the part that was never code to begin with.

The clearest tell showed up in a cost argument I watched play out. The obvious way to save money on agent work is to let an expensive model do the planning and a cheap one do the typing. It mostly backfires — the cheap model burns the savings re-reading everything to rebuild the context the expensive one already had, so you pay for that reading twice. The fix isn’t a better model, it’s writing the plan down: have the capable model capture what it figured out into something durable — a todo list, a decision record, a preamble — and hand that to whatever runs next. Which is the whole thesis in miniature. Even the move that looks like a model-selection problem turns out to be a harness problem, because the win is in the artifact you persist, not the model you pick.

But the harness is starting to build itself

Here’s the objection I’d raise if I were reading this over your shoulder: if the code-writing layer is delegable, isn’t the harness next? Partly, yes — agents can now mine their own execution traces and patch their own scaffolding, gated behind regression tests. But what automates is the harness as runtime. The layer above it — deciding what counts as a regression, what goes in the evaluation set, what the harness is optimizing toward — doesn’t. The machine optimizes against the gate; designing the gate is the work. That deserves its own field note, and it’s getting one.

Automation doesn’t delete the harness; it relocates the human part up one level — from writing the runbook to deciding what the runbook is for.

What I’m actually claiming

I’m claiming two things, and they’re separable.

(1) The harness is more leveraged than it used to be. Even five years ago, harness work paid off slowly — onboarding got easier, code review went faster, a runbook saved someone a Tuesday night. Today it pays off in real time: a sharper CLAUDE.md changes the next agent run before it even starts. Compounding got faster.

(2) The harness is more legible than it used to be. This one is subtler. In a world where the code itself was the artifact, harness work looked meta, soft, hard to promo on. In a world where the agents reading the harness are doing the code work, the harness IS the artifact. When I first wrote that line it was a prediction; a couple of months later “harness engineering” is a named role in the trade press. It’s not invisible labor anymore — it’s the thing.

The econ read: this is Baumol’s cost disease, playing out inside a single job. When one part of the work gets dramatically cheaper — code-writing, in this case — the parts that don’t (judgment, review, deciding what the gate is optimizing toward) become relatively more expensive, and the value migrates to them. The harness didn’t get more important because engineers got sentimental about docs. It got more important because everything around it got cheap.

What’s next

This is the framing piece for a set of Field Notes I’m writing about specific harness moves I’ve made in real engineering work. The first one is The Documentation Burst — a methodology for converting a year of tacit subsystem context into reusable reference in a single day. After that: Designing the Gate (the field note this essay just promised), The Decision Tree Write-up, The Operational Runbook, and others.

None of them are about a particular language, framework, or industry. All of them are about senior IC work whose complexity has outrun the tools your team has for capturing it.

The harness is the work. The Spillover is the place to put what you build.