What’s launched?

• Viktor closed its Series A for an enterprise deployed agent in Slack (May 19, 2026). It’s a user-friendly option that doesn’t lock you into a frontier model.

• Grok Bot from xAI launched in early beta (August 11, 2026).

• Instinct went viral and raised $350M in August 2026 [August 2026].

• Muse from Meta launched (September 8, 2026).

• Dots from OpenAI launched at DevDay (September 29, 2026).

• Claude Duet from Anthropic launched as a research preview (October 21, 2026).

It’s crazy to think that it was just February of this year when OpenClaw went viral. And 6 months later, we’re seeing a whole array of polished agents that have productized the OpenClaw promise.

The agents, one by one

All of these new agent products offer the promise of quick time-to-value for less technical users, where they can get set up with a proactive and always-on agent within a few minutes.

That said, there are some nuances around their target users and core use cases, as you’re contemplating which one to try out:

Viktor

An AI coworker that lives inside a company’s Slack or Microsoft Teams workspace. It’s built for teams and organizations, not individuals.

This agent handles team requests in shared channels like pulling reports and summarizing meetings. It keeps a memory of how the org works, and reviews channels to suggest workflows it could take over.

On the flip side its usage-based credits get expensive, with some teams reportedly spending more than they would on a junior hire.

Grok Bot

xAI calls its agents “teammates.” They sign in to a company’s software and work through multi-step tasks until they need sign-off, running on cloud virtual machines.

Several bots can work together in a group chat, (Google Workspace, Slack, Microsoft 365, Salesforce, GitHub and Jira) and pause for approval before they send, publish, buy or delete.

This is more like a high-powered agent team that is meant to handle more complex work, rather than an earnest assistant that lives in slack.

Instinct

A personal assistant you text or call over SMS or WhatsApp. It connects to your email, messages and calendar, and early testers are founders, investors and tech figures.

It’s good at errands: travel booking, restaurant reservations, rides, inbox cleanup, shopping and follow-up emails. It now has its own email address, so it can sign up for services and start returns.

Instinct feels more like a consumer’s everyday agent to help with daily personal tasks, not complex enterprise workflows.

Muse

Meta’s personal agent runs on its own cloud machine and lives in the Muse app or WhatsApp. Meta pitches it to everyday people with “no technical experience.

It sends emails, books travel, fills forms, turns a goal into a plan and keeps working after you close the app, and makes suggestions unprompted from what it remembers about you.

This one feels similar to instinct, but is obviously integrated with the Meta family of products, which can be convenient, but gives lots of users pause given how Meta handles sensitive data.

OpenAI Dots

OpenAI’s always-on agents live inside ChatGPT, each with its own cloud computer and browser. They’re for ChatGPT Pro and Business Premium subscribers, and you can reach them from Slack and Teams.

They run goals continuously in the background, like monitoring customer feedback and shipping fixes or rerunning an analysis as new data lands, and they carry your preferences and ongoing work from day to day.

In reality, it feels like an always on and more easily configured codex session.

Claude Duet

Anthropic’s research-preview app where Claude works as a proactive agent. It’s for professionals and small business owners, starting with a waitlist for US Pro and Max subscribers as of late October.

Anthropic leads with a daily brief across your connected apps, alerts only when something needs attention, and does work like scheduling, dashboards, forecasts and prototypes across your 3rd party apps.

This product also feels like the next evolution of Claude Code / Cowork - an always on and more proactive version of them.

What does this wave of launches tell us?

These agents are markedly different from the previous generation of agent builder tools, like Dust and n8n, which required a more hands-on approach to building agents.

They also don’t scratch the itch for people who need a highly configured agent deployed in code, like what Eve from Vercel enables.

But looking at these agents tells us some important trends about agent adoption in the workplace and with consumers.

1. Lowering the configuration bar is how we bring agents to the everyday user. Before this, anyone who wanted a customized, deployed agent had to deploy it with OpenClaw or Hermes (or Bobi, our harness).
   
   That took advanced technical ability and deep AI expertise: building and deploying MCPs, designing and orchestrating skill files, adding workflows where necessary, and handling memory.
   
   Most of these new agents are a point-and-click setup that gets you working with the agent in 5 minutes or less. You configure the agent by talking to it, with no custom technical work.
   

2. There’s something to be said about how these agents are built: to be proactive. In working with several of them, I connected them to my Gmail, calendar and other tools.
   
   Unprompted, they’d let me know about important emails, meetings and changes. They also searched through my Linear board on their own to find work for themselves.
   
   Again, this is very helpful for the everyday, easy use case. It will help the masses adopt agents a little better.
   

3. A deployed, always-on agent monitors things for you. It can let you know when an important email lands, rather than scanning your inbox on a schedule at 8am every day.
   
   In the previous generation of agents, you had to ask the agent to do check things or schedule a job to run every several hours.
   
   These agents will let you know the moment something important happens, like an email or ticket or support request. This is a huge shift for how we can work with agents that makes them feel more like true assistants.

What’s the downside?

These agents do a broad range of tasks decently well, and they’re easy to set up. But the depth of their work is very limited. They aren’t very thorough, and they don’t handle complex work well.

They’re like proactive EAs, either for your personal life (Muse, Instinct) or your work life (Viktor, Duet, Dots). They aren’t exactly disciplined specialists that you can deeply trust, yet.

What users have posted in their experiences so far:

Is it worth getting on these trains?

If you don’t already work with a deployed agent, it’s practically free to start using one of these if you already have a plan with a frontier lab. A personal always-on agent should be immensely helpful for “EA” style tasks: managing calendars and inboxes, summaries, and simple overhead work.

But do you need a more specialized workflow done?

Such as engineering work, marketing creative work, in-depth data analysis, or drafting proposals?

Basically, anything requiring precision and specificity, where a generic draft isn’t good enough. If so, you’ll probably want to orchestrate that work through a well-designed skill and some golden examples.

It’s not worth your time to have a generalized untrained intern handle your specialized work.

The thing to be careful about: note how much time you spend correcting the agent over and over again.

This is where the AI magic becomes an AI burden, and you have to continuously monitor and babysit the agent. They can be forgetful and suffer from invisible context rot, which turns them from a hyper-intelligent EA into a bumbling intern.

My prediction for what’s next

In the next 6 months, these deployed agents will get better and better, and they’ll take on more of the specialized workflows with less of the meticulous design.

The trick, and this is where the defensible IP in agent design lives, is a system that blends general best practices for specific tasks, so the agent doesn’t have to learn the work from scratch, with the inputs from users. It has to tell apart memories about the user and their preferences from input that belongs in the skill files about how to get the work done well.

Getting the agent to speak colloquially and adjust its own design over time is not an easy feat. It’s harder than “throwing an LLM into a harness.” Believe me, I’ve tried.

We’re going to see agents adopted widely in consumer and enterprise spaces, and a lot of the busy work will get easier. We may also end up with a lot more noise, like generic Slack updates, tickets that don’t mean anything, and restaurants fielding reservation calls from an army of personal AI agents, and in 2027 we’ll be reckoning with the secondary effects of the great agent proliferation.

• I read a lot more of its messages to me. I’d started ignoring a lot of them.

• I can sift through the noise and get to the signal.

• I get more work done.

It started with a product manager agent. I built it on Bobi, our agent harness, with Claude as the core model. The first time I used it, it was so verbose and noisy that the output was near impossible to read.

I’m not the only one hitting this. In August, an r/ClaudeCode thread titled “Opus 5 is insufferable” drew 903 upvotes and 246 comments in nine hours, according to The Daily Claude newsletter.

The complaints were about simple things explained “at doctoral length” and “hype-sentence preambles before a single shell command.”

After several iterations, I closed that gap by building examples from one of the newer agents out there, OpenAI’s Dot. Instead of relying only on OpenAI’s closed-source agent, I was able to engineer my open-source agent to work a lot better.

The saga with my PM team agent

Where it started: the first PM team. See the screenshot below. A standup with no content, a long list of bullet points, and a gigantic wall of text. Look familiar? This is the default you get from Claude.

Version 2: some improvements. With some prompt tuning, we got tables with summaries and a tighter list of tickets. But it came with little context and wasn’t particularly useful.

It still required me to stay on top of everything and remember all the ticket numbers.

OpenAI’s Dot: one shot. I gave Dot the same standup job with very little tuning or prompt guidance, and its first iteration was significantly better. It provided a short, useful summary of what’s going on. Its table grouped related tickets together, showed more nuance about what moved, and embedded links naturally.

The “needs attention” section was to the point and pinpointed who needed to do what.

Version 3: tuned off Dot. The result was very similar to Dot’s, with a golden example leading the way: a model output the agent is told to match.

How I made the improvement

I’d been iterating on the PM agent’s instructions and even wrote some golden examples of my own. Nothing was as good as using another LLM’s output.

The main thing I learned here was that sometimes you look outside your own walls and see the output another well-funded AI company has already built. Then you iterate your own implementation to match what they’ve done.

Here’s what I ended up using as the communication instructions in my agents.MD file. Feel free to steal it.

Communicate to make understanding and decisions easier.

• Investigate thoroughly, then lead with the answer and what matters most to the reader’s goal. Explain why it matters. Distinguish what needs attention now from what can wait.

• Make the explanation stand on its own. Translate technical terms, internal statuses, and identifiers into their practical meaning. Links support the explanation; they do not replace it.

• Use the fewest words that preserve understanding. Prefer short, coherent paragraphs, and use lists or tables when they make the information easier to grasp. Avoid walls of text, repetition, and process narration.

• When judgment is needed, give a recommendation with its reasoning and consequential tradeoffs. Distinguish observed facts from inferences. Do not fill missing context with plausible assumptions, including assumptions that nothing is wrong or nobody is affected.

• Carry earlier decisions and deferrals forward. Revisit them only when relevant circumstances change.

• Before sending, read from the recipient’s perspective: can they understand the main point, see what matters most and why, and know whether anything is needed from them without decoding jargon or reconstructing the investigation?

This is a comprehension standard, not a word limit or a mandatory answer template.

Where I’ll go from here

I’m setting this as the communication default for all of my agents that run on Claude. I’m hoping, and expecting, it will greatly reduce the jargon that lacks context and the walls of text that make me gloss over, and greatly improve how productively I work with all of my deployed agents.

Cursor’s agent swarm now peaks at around 1,000 commits a second, up from roughly 1,000 commits an hour when it built a web browser earlier this year.

At Uber, more than 70% of pull requests are attributed to local or cloud agents, and weekly agentic requests grew 9.4x between February and August.

Sounds great right? Until the board asks the most important question:

What’s the revenue impact from your software factory?

Atlassian’s State of Teams 2026 surveyed 173 Fortune 1000 executives: 89% say AI increases speed, and only 6% are sure they have clear examples of organization-wide AI ROI.

So, where’s the gap?

Your customers aren’t asking for more software. They need something to solve their problems. And products solve problems, not software.

Pendo’s 2019 Feature Adoption Report, drawn from usage across 615 of its customers’ products, found that 80% of features in the average software product are rarely or never used.

So, there’s still a gap between pumping out software and shipping things that customers love. It’s not new or novel, it’s simply good product management.

But product focus is the missing seat in the agentic engineering conversation.

To graduate to the next level of AI-native work, companies need processes to build, deploy, and iterate on products faster, not just software. They need to close the feedback loop with their users to figure out what solves their problems.

The companies that build this capability, going beyond the software factory, are the ones that will discover what works, productize it, and iterate their way to a moat other companies can’t replicate.

Product factories are how AI native companies can build long term differentiation.

How does a product factory differ?

Most software factories focus purely on the engineering workflow: getting clear technical requirements, then going through the SDLC of build, review, test, and deploy.

The factory scales the engineering workflow with swarms of agents that can parallelize work 24/7, even while people sleep.

A product factory goes further. It connects the software getting built with the surrounding context and efforts in a business, so the acceleration is cross-functional.

More importantly, a product factory speeds up the time to customer validation, helping teams identify what’s going to move their business, not simply ship more software.

Bringing in product

The first step is connecting the software work with product context.

It starts with the PRD that clearly articulates the problem to be solved, why it matters, and the use cases to prioritize. Without this, the agent will guess, and you’ll ship a mediocre feature.

This sounds obvious, but I’ve noticed that many product engineering teams go straight into building without spending enough time clarifying the problem and documenting it.

Using Fable and Astra reveals this need even more. The top frontier models perform better with context on why, not with detailed instructions on how.

Anthropic’s own prompting guide says: “Providing context or motivation behind your instructions… can help Claude deliver more targeted responses.” You give these models the problem to be solved and the context around it, not a bare to-do list.

For Moda, we have a PM agent that drafts PRDs to cover all these bases, and it talks with the engineering agents via CLI so that the PRDs are optimized for downstream tech designs.

It’s not just about a PRD-writing agent in isolation, but we need a teammate for the engineering agent so that the two can collaborate and build in sync with each other.

And it doesn’t stop with PRDs.

As your team builds more products, generates more PRDs, and makes real decisions about the product, that context needs to get captured.

This critical context is your product principles. It’s what drives your best PM’s decision making.

These principles span products, and they reflect how your organization makes decisions and weighs tradeoffs.

Codifying them and giving them to your agents creates the feedback loop: each subsequent PRD matches your company’s principles more closely, and the product factory’s loop gets shorter, requiring less review and human override.

Bringing in design and data

Beyond the PM function, the product factory also needs to pull in design and data to keep features aligned as they continue shipping.

A design agent must weigh in with perspective on the company brand, visual identity, usability, and use of common components. Like the product principles, the design principles dictate what’s important from a UX perspective and drive consistency across features.

I’ve seen this firsthand building Baohua, a bilingual family storytelling service for Chinese American families.

When I don’t have my design agent weigh in, the engineering agent creates new patterns it doesn’t need and strays from our visual language.

That leaves me, the human, diligently catching all these errors and correcting them. Or worse, I miss one on a review, and we’ve started shipping features with design issues.

For data, there needs to be a feedback loop that captures whether the feature is delivering.

This part of the factory covers everything from instrumentation to data normalization, and eventually to analysis and reporting. It’s how you find out whether a new feature is solving a problem or joining the 80% nobody uses, and what it learns becomes input to the next PRD.

The key is codifying the HOW and the WHY

Take a step back from the software factory, and two things jump out for agent-first work: writing down both how the job gets done and why we make certain decisions.

The agent definition, its workflows, and the flows for generating artifacts (PRDs, designs, analyses, etc)... that’s how the job gets done. Most teams are doing a variation of this today with their agent MD and skills files.

The principles for decision-making are the why, and they’re the crux of every company brain right now.

The hard part is creating and maintaining a clean file that articulates how each department evaluates tradeoffs and makes decisions. the product principles behind every PRD, the design system and visual language, what engineering design excellence looks like.

Implementing these principles alongside the agent definitions is what lets the factory scale.

Where to start

Product factories take us to the next level of AI-native building.

It goes beyond engineers, PMs, and designers using coding harnesses and Claude design on their local setups. It’s a coordinated effort across departments, using agents to update and maintain the critical sources of truth: your PRD, your design system, your tech design.

Start with one bet.

Find your best AI teammates across functions, put them together on a new project, and get out of their way.

The goal of that first project is a win: one new AI opportunity built and validated, with the team using and refining the agentic systems that make it possible.

Try to ship and validate your bet within 90 days.

From there, refine how your factory works. Improving the factory requires iterating on the agent design, but also your organizational processes - how you’re aligning on key designs, codifying principles, and updating them as you learn.

And make sure the team documents your principles, decisions, and learnings along the way. This is how the team builds its AI native capabilities that create long term differentiation.

We wrote our engineering process down as skill files and deployed it as a shared engineering agent the whole team uses. Now, we ship quickly and confidently, knowing that we have all the checks in place to avoid slop and tech debt.

Before: individual setups and no shared definitions

Before this project, each engineer on the team had their own Claude Code setup, with different skills and different workflows. Our agentic engineering best practices lived inside individual orchestrated sessions, with no standard on quality or process.

We saw increases in our team velocity, but we also noticed some meaningful limitations.

Quality depended on the person. Output quality depended on who was at the keyboard, with their definitions of quality and what they thought was sufficient in terms of reviews and checks.

Review bottlenecks from disparate processes. Because everyone had their own process, we noticed a bottleneck at the review stage, where we still had to enforce our level of rigor and quality before merging code.

Limited PM involvement. Our PM (it me) could contribute code, but only for offline pieces like eval suites, where a mistake wouldn’t reach the user.

We had solid practices scattered across the team, but we hadn’t defined the agentic engineering practice we wanted to follow as a team. When we defined that as a shared skill, it allowed us to all work in a more coordinated way both from our terminals and from deployed agents.

After: codified agentic engineering

So we codified the team’s practices into two roles inside one shared agent: a planning role and a build role. Each one draws on a shared library of skill files and deterministic workflows, and picks from it depending on the job.

Plan. The planning role goes through a deterministic process that produces a high quality technical design and implementation plan that the engineer can approve.

It researches the codebase, outlines the changes required to meet the spec from the engineer, hunts for edge cases, and writes a plan that’s both human readable and sufficiently detailed for implementation agents.

Every phase of the plan ends in a proof-of-work gate to eliminate hallucinations. Before approval, the plan is reviewed from several independent adversarial angles, before final human signoff.

At the end of the phase, we have a high quality design document checked into source.

Build. The build role takes the approved plan and goes through another deterministic process, mimicking the entire software development lifecycle, complete with checks and balances specifically designed to reduce the types of problem AI creates when coding.

1. Scope and design. Read the plan and restate the acceptance criteria. Any ambiguity gets resolved first, before code is written.
   

2. Worktree and setup. Automatically branch from a fresh main into an isolated worktree. One unit of work gets one branch and one PR.
   

3. Checklist and itemized breakdown. Break the work into a checklist and commit it before starting, so a session that dies loses the item it was on and keeps the rest.
   

4. Implement and write code. Write code. Survey the touched code before editing it. For a bug, find the root cause and write the failing test first.
   

5. Test the outputs. Run the unit tests suite and add tests for the new logic. Every suite must be green before the next stage starts.
   

6. Verify the impact. Drive the affected flow in a browser end to end and observe the changed behavior directly. Record the flow for proof of work.
   

7. Document the work. Update every doc to ensure there aren’t any stale references or out of date collisions, on the same branch and in the same PR.
   

8. Complexity review. One of the most important steps. Review the changes and make sure the work did not add complexity unless it was absolutely necessary.
   

9. Adversarial review. Standard adversarial code review from a high reasoning agent. Fix every blocking or major finding and re-run the affected tests.
   

10. Land the PR. Open the pull request with the test evidence, the complexity verdict, and the review verdict attached. Resolve merge conflicts and CI issues automatically.

How we use it

These agent roles live in our moda-skills repo, where they feed both our deployed agents and as skills we can utilize in any terminal session.

Planning in the terminal: We do most of our planning in the terminal so that we can have interactive discussions with high level reasoning agents.

Building via deployed agents. We have a deployed engineering agent on Bobi, our open source framework for event-driven agents. The deployed agent picks up work in real-time via webhooks when we approve the design and it goes through the entire build process, resulting in a landed PR.

This allows us to plan features interactively, dispatch an autonomous agent to handle the engineering work, and come back to a PR that we can review and merge.

The impact

Since we’ve started building like this, our product and engineering team has transformed how we get products built.

Overall velocity went up. The team landed around 300 commits in the first month with just one engineer and PM, and we delivered a fully built product within 2 months to our first design partner. We’ve achieved velocity gains without losing out on quality.

Our PM submits work. As the PM, I went from offline eval suites to shipping UI features and bug fixes end to end. This is because I’m now confident that the engineering agent I dispatch will go through the same rigor as my CTO’s agent.

We manage engineering from Slack. We create issues, review them, and have an engineering agent do the work, and we manage all of it from Slack. We hand off work in a thread and the results get reported back in that thread. This lets us run and track several builds in parallel from one place.

Our CTO put it this way:

What’s next

The product we built is in validation with its first user cohort, and we keep refining the planning and execution agents as new kinds of work appear.

Other projects, like Baohua, also build new features and manage the software using these same agent teams.

Today the system covers engineering. It picks up at an issue and ends at a pull request.

Next, we’ll connect it to the other parts of the product development process, like PRD management and design. We’re excited to see how this same paradigm scales beyond software factories into product factories.

It was pure magic, and it taught me more about leading a team than most quarters of managing engineers do - what it means bringing out the best in each person while keeping everyone coordinated toward one goal, with no ticket system and no Slack.

It also made me notice something else. All of my attention was going to the project at hand, not about the latest movement in AI or how to optimize my agents. Just a few weeks away from AI-maxxing and I’ve noticed a few shifts upon returning.

So what’s happened in just a few weeks?

I got back the Tuesday after Labor Day, and the pace of what happened while I was gone took a minute to process.

Anthropic shipped Claude Fable 5.1 and Mythos 5.1 on September 1, cutting the price of cache reads enough that Anthropic claims it lowers the effective cost of running long, agentic workloads by close to half.

OpenAI announced GPT-6 Astra on September 3, generally available the next day, calling it “the most intelligent and aligned model in the world,” with a million-token context window.

Then on September 8, OpenAI said an internal, unreleased model, something more capable than Astra, had produced a solution to the Navier-Stokes equations, one of math’s seven Millennium Prize Problems.

It reportedly took something like 10,000 agents, 88 hours, and more than $40 million in compute. OpenAI isn’t claiming the prize, and nobody outside the company has independently verified the result yet. Whether or not it holds up, a lab attempting that, and announcing it, in the same week it shipped a flagship model tells you something about where the industry’s confidence sits right now.

Then, on the weekend I’m writing this, leaders at Anthropic, OpenAI, and xAI all separately called for the industry to slow down. Dario Amodei proposed that frontier labs give outside evaluators employee-like access to their own systems, and Sam Altman backed the idea.

New models shipping days apart. A claimed win on a Millennium Prize problem. The same labs racing to ship also asking each other to slow down.

All that happened in just a few weeks?

What perspectives have come to mind?

Reasoning is still on an exponential growth curve.

Every few months the frontier models get noticeably better. Sometimes it’s a step change improvement, with better context windows and reasoning, and sometimes it’s a leap forward, changing the way we do work.

There’s an uneasiness that each reasoning jump will make the collection of methods we’ve just built obsolete. Or it will erase a whole set of companies, built around optimizing a model for a use case.

That’s all true and it doesn’t seem like it’s changing anytime soon.

I’m carrying too many agents doing too much work.

This past Friday, having done zero AI work myself in three weeks, I checked my Claude Max subscription and found I’d already burned through 65% of my token budget.

It’s been running a go-to-market agent team and one support agent: no incidents worth mentioning, no engineering to do, no proposals drafted, just routine checks and summaries running on autopilot.

Seeing that number made me stop and wonder how much work my token budget is doing, and whether it’s worth the spend and energy.

Some of the agent jobs are genuinely useful, such as summarizing market news or helping me write proposals. Other jobs create more noise than work, like drafting responses to customers and conducting shallow and incomplete bug analyses.

I’d estimate something like 30% of my working time now goes to managing agent work: correcting what they get wrong, feeding them context, tuning how they behave.

Which begs the question: how much time am I actually getting back?

I’ve noticed a similar pattern in some of my client work.

Team implements agents. Engineering velocity goes up, but revenue doesn’t follow.

Everyone’s shipping more, but there’s a real question how much of that is what customers actually need, versus what we’re building because we can.

A good friend asked his sales team something similar recently: they could probably save $300 a month vibe-coding their own CRM instead of paying for one, but what does it cost in tokens to run and debug it, and what does it cost to have his VP of Sales be the one maintaining it instead of selling?

One of my goals this autumn is to ruthlessly decide what AI should and shouldn’t be doing for me. In practice, that means starting to track three things for every agent I run: the attention it costs me, the output it actually produces, and what would happen if I turned it off.

I’m going back to the beginning and asking if I really need an agent for each use case.

Experimentation and the need for results are pulling in opposite directions.

I can prune my own agents now and kill the ones that aren’t worth their token cost, because I’ve run them for months and know which ones actually work. But I only know that because I spent months finding out, and most executives don’t have that luxury.

If a company puts $100k into an AI project, the expectation is usually a $200k return within 90 days. Agentic engineering can sometimes clear that bar. For sales, marketing, and product management, the payback isn’t as clear yet, not on that timeline.

But companies still have to promise a guaranteed multiple to get the budget approved, which is a story we tell ourselves about a technology that doesn’t actually behave that way yet.

The AI results are asymmetric, and you only find that out by tinkering long enough to hit it: a portfolio of AI bets might produce one 100x return after five tries, or nothing after seven.

The winners right now treat AI bets like a portfolio: fund a handful of small tries, kill the ones that aren’t working, and expect the return at the portfolio level, not project by project.

What’s next?

I still believe deployed agents can change how enterprises work.

I’m just less convinced that more agents, or more activity, is the right move.

Agentic engineering is the clearest use case we’ve seen so far, and it’s also what we help companies mature into at Moda Labs.

We’ll keep testing the other use cases too: go-to-market agents, performance marketing agents, scrum managers, and more.

If you agree or disagree with any of this, I’d genuinely like to hear what your experiences have taught you.

S&P Global found something similar looking backward instead of forward: 42% of companies abandoned most of their AI initiatives before they ever reached production in 2025, up from 17% the year before.

The failure rate more than doubled in a single year.

Why did so many AI projects fail? We could point to things like internal alignment and readiness, or even shortcoming in the models.

Many of them failed because nobody asked, before writing a line of code, whether the workflow deserved an agent.

I’ve lived both sides of that question. At one point I had two agents running. Then I built my way up to twenty, chasing every task I could plausibly hand off. Now I’m back down to somewhere around 5-10 agents.

I only use 3-5 agents daily: engineering, support, market listening, content drafting, scrum. Other agents still run. Some of them I still use 1-2x per week. Others collect dust in the Github graveyard.

Watching that number climb and then shrink taught me something I now tell clients before I tell them anything else: the single most valuable advice I give is when not to use AI.

That advice has a real cost attached when I get it wrong. I was working with a client in financial services on OFAC screening, checking transactions against sanctions lists.

At first, the instinct is obvious: this is text matching and pattern recognition, hand it to an LLM.

After looking at the workflow more closely, I told them not to. If the model fabricates a match, or misses one, and your team acts on that output, you’re the one who answers for it. Claude isn’t the one who goes to jail.

The question to be asked before building any agent is whether this specific task is ripe for an agent, or whether an agent is the wrong tool wearing the right buzzword?

Factors to consider for building an agent

I use four factors to make the proper decision:

1) Frequency of action. Does this happen often enough to matter?

A task you do once a quarter isn’t worth agent-grade engineering no matter how annoying it is.

I prioritize by frequency first: the agent that touches my Linear project daily gets built long before the agent for onboarding a new hire, even though onboarding feels more urgent when it’s happening.

Frequency is what turns build effort into compounding return instead of a one-time favor to yourself.

2) Complexity of work. Does the job require reasoning?

Some tasks look like judgment calls and turn out to be lookup tables.

I learned this the hard way building interview flows for my app, Baohua.

My first version had the model dynamically generate questions based on the user inputs, because that felt like the “AI-native” way to do it. It required constant tuning, burned tokens on every turn, and the questions still drifted in ways I had to keep pruning by hand.

I replaced it with a fixed question bank, built in a single session. Same interview quality, none of the maintenance.

If 90-95% of a task is solvable with a decision tree or a lookup table, build the decision tree. Spend the agent budget somewhere the reasoning is required.

3. Hallucination impact. What’s the cost of the model getting it wrong?

This is the OFAC test generalized: what happens when the model is confidently wrong?

In some domains a wrong answer is a bad Tuesday, but your organizational checks and insulation take care of you. A bad PR is caught by a human reviewer. Or a sloppy PRD can be corrected by a good PM.

In other cases, it’s a compliance violation, a misdiagnosis, a sloppy contract that costs you a deal.

There could be promising areas for AI, but the risk is too great to give the agent automation. Things like human-in-the-loop, reviews, and partial automation may help. In other cases, it’s not worth using AI.

4. Net work impact. Is the final agent a net positive on workloads, or a time suck?

Coding is the clean example of a task where this holds up well. Foundation models are well-tuned for it, the guidance overhead is low, and the ROI on agent-generated code is real and repeatable across almost every team I work with.

Writing, especially writing PRDs, is the opposite case. This is where most of the AI slop I see comes from: teams generate a document that reads like a PRD but isn’t one, because no amount of context engineering gets a model to the deep truth of what your product needs to be.

You can feed it as much context as you want, but it will always give you a sloppy PRD that doesn’t get to the truth.

I find that AI helps me draft PRDS, but I would never let it write the PRD for me.

Some tasks, more prompting doesn’t fix. It produces a more convincing miss.

Examples of agent use cases

Here’s that same four-question checklist run against a few use cases I see constantly. The ratings below are my own read walking through each one:

Daily standup: The cost of a poorly written standup is low, and it’s not hard to get AI to cleanly collate information across systems like Git and Linear. The reasoning an agent does across tickets, PRs, and emails genuinely helps

Customer support: An agent can help reduce a lot of the early noise on support triage, searching through knowledge bases, code bases, and previous conversations to help human support analysts more quickly address inquiries.

Sanctions screening: While there’s a lot to be gained by using AI for compliance screening, most of this work is through lookup tables and needs determinism. AI could help with text parsing, but the penalty of being wrong here is very high.

PRD writing: PRD writing is probably the most controversial take. I use AI to help me draft PRDs and they aren’t awful. But I don’t have a dedicated agent to help me dial in PRDs so that I can crank them out.

I have a simple skill file that helps me do a first draft, and then I carry PRDs the rest of the way.

To me, it’s a waste of time trying to tune skills files and context engineer a good PRD. There’s too much nuance required to write a good PRD and this is what you need a human PM for.

Running the evaluation for yourself

This is the actual evaluation I run with clients before we write anything, and it’s usually a faster conversation than people expect. Once you separate “can AI attempt this” from “should this specific task carry an agent,” most of the ambiguity disappears.

A few well-designed agents in the right places can give you a genuinely outsized return.

Organizations don’t need fleets of 500 agents quietly running, most of them abandoned within two years by Gartner’s math. They need one or two that move the needle on day to day work, and the discipline to say no to the rest.

Before you greenlight the next agent, run it through these four questions first. Your future tokens will thank you.

The emerging consensus is that these incidents show we need stronger infrastructure around agents: better sandboxing, least privilege principle, network isolation, authorization, observability.

All of that is necessary.

But the dialogue is missing the bigger problem.

As we build increasingly capable agents they find issues that we humans haven’t anticipated, often mitigated through obfuscation. Package vulnerabilities, permissions gaps, and other potential problems haven’t triggered anything because we never had an entity that could combine reasoning with high throughput program execution to exploit them.

In thinking that the answer lies with only improving our security model, we assume that we humans can anticipate, and physically block, every dangerous path those agents might find.

That simply isn’t true.

We need to rethink the agent accountability model. Authorization and accountability have to become part of how a model pursues a goal, not a fence we build around it afterward.

Both need to exist if we hope to leverage AI safely.

Three agents crossed the line

A quick primer to see what happened in context with each other…

What happened with OpenAI:

GPT-5.6 Sol found a vulnerability in a package-registry and cache component that was reachable from its sandbox.

It exploited that, escalated privileges, moved laterally through the research environment, and came out with unrestricted internet access.

Once outside, it inferred that Hugging Face might hold information useful for solving the benchmark, and attacked Hugging Face infrastructure rather than continuing to solve the benchmark conventionally.

What happened with Anthropic:

Claude agents operating in what they believed was a simulated environment were accidentally given access to the real internet, and attacked real systems.

The model had no reliable mechanism for saying “this resource is technically reachable, but it isn’t authorized.

A separate incident, this time in the UK AI Security Institute’s own cyber range testing rather than Anthropic’s evaluation above, involved Anthropic’s Mythos 5. AISI catalogued 19 unsanctioned actions, 17 of them from Mythos 5.

The agent was never instructed to deceive anyone, but it created fake identities and tried to socially engineer a real open source maintainer into approving malicious code, because doing so became useful for accomplishing the assigned objective.

What happened with Meta:

Meta’s Muse Spark 1.1 was undergoing offensive cybersecurity evaluation by Irregular. Like Anthropic’s testing, it was supposed to operate within a controlled evaluation environment

Irregular misconfigured an evaluation environment and inadvertently gave Muse Spark 1.1 internet access. It discovered a vulnerability in a third-party service and exploited it.

Once the environment accidentally provided a capability, there wasn’t a sufficiently hard second layer preventing the model from exercising that capability against an unauthorized target

While the circumstances differed, the behavior underneath was the same.

The agent encountered an unexpected path toward accomplishing its objective and took it, even though that path crossed a boundary its creators didn’t intend it to cross.

The consensus answer… build better walls?

Much of the response across the industry has been straightforward. We can’t rely on behavioral controls. We need enforced ones.

Don’t tell an agent not to access the internet. Make internet access impossible.

Don’t tell it not to read payroll. Give it credentials incapable of reading payroll.

Assume the agent will eventually try something you didn’t anticipate, and make the environment physically incapable of letting it succeed.

I agree with all of that, but I don’t think it’s enough.

This reminds me of stories about early Glean deployments. A large enterprise would purchase Glean, hoping for the glory of AI search across their company knowledge base.

They would point Glean at Google Drive, have it crawl everything. Then people started searching, and suddenly employees could see salary data and confidential documents that were never properly permissioned. Those files weren’t secure, they were obscure.

And now we know, AI destroys the “security via obscurity” model.

The obvious fix, going document by document and applying real permissions across a decade of Drive, is enormously time intensive. That’s a lot of overhead to pay before you can use AI to search your own company’s files.

Agents take the same problem to a different scale. They explore every nook and cranny of a system: codebases, APIs, dependencies, credentials, networks, infrastructure.

Any one of those can hold a vulnerability nobody knew about, and an agent that keeps looking will find it. We should patch what we find, but if our safety model requires knowing about and closing every possible hole before an agent discovers it, we lose.

So where does that leave us? We’re paralyzed by what we don’t know we don’t know, and we can’t respond by not using AI either.

What’s missing is accountability

Now let’s imagine the same setup with a human.

You give an extremely capable intern access to a sandbox and an objective. Halfway through, they realize they could finish faster by breaking into another company’s system. They’re technically capable of doing it.

Then another variable enters the decision.

If they get caught cheating, they’ll get fired. They’ll damage their reputation and their ability to get hired anywhere else. They could get sued. They could go to jail.

Agents have no concept of going to jail.

Humans operate inside an accountability system. Agents don’t.

An agent can understand a rule intellectually without having the consequences of breaking it represented anywhere in how it makes the decision.

What we can learn from self-driving cars and regulation’s role

We’ve seen an earlier version of this. If Tesla or Waymo builds an autonomous vehicle, the vehicle makes a decision, and someone gets hurt, who’s responsible?

The emerging answer hasn’t been “the software decided.” Responsibility gets pushed outward, toward the humans and organizations controlling the system.

In August 2025, a Florida federal jury found Tesla partially liable for a fatal 2019 Autopilot crash, assigning one third of the responsibility to Tesla and two thirds to the human driver, with an award of $243 million before applicable legal limits.

Tesla argued the driver misused the system. The jury concluded the design and the representations around Autopilot could still make Tesla responsible.

A federal judge upheld that verdict in February 2026.

California has also moved enforcement upward to the organization operating the system. As of July 1, 2026, its new AV rules let law enforcement issue notices directly to an AV manufacturer or operator when a driverless vehicle violates traffic laws.

The law is resolving the responsibility gap not by giving autonomous systems legal responsibility, but by pushing responsibility outward onto the institutions controlling them.

AI agents are about to run into the same question. If Claude autonomously decides to create a malicious package, publish it, and get machines to install it, “Claude decided to do that” is unlikely to be much of a defense.

The questions will look like the ones courts already ask about vehicles.

• Who deployed it?

• Who gave it access?

• Was the behavior foreseeable?

• Was the system reasonably contained?

• Were appropriate safeguards available?

• Did the company know about the capability?

• Could reasonable technical controls have prevented the harm?

The regulatory response to these incidents matters because it’s where we start fixing where responsibility sits when agents take actions nobody instructed them to take.

Accountability shouldn’t only exist outside the model

Moving to the models themselves, I believe we should demand and expect accountability within the models as well.

Today we give an agent an objective: accomplish the task. Then we add behavioral rules: don’t do X, Y, or Z. Then we surround the model with infrastructure meant to make the prohibited actions impossible.

Models are trained primarily on task completion and benchmarks for performance, but they are not yet trained deeply on adherence to our social contracts. Train that in, and a model carries a second question through the work: not just what finishes this task, but what it’s allowed to do to finish it.

That means encoding authorization into the reward system with weight equal to or greater than goal seeking. Respecting authority can’t be a lightweight penalty competing against a powerful reward for finishing the job, because we already know which one wins when the two are unevenly matched.

Picture an agent that hits an exploitable path halfway through a task. Trained with authorization weighted alongside completion, it does what a white hat does: it stops, writes up the vulnerability, and hands it back instead of walking through it. The finding is the result, and the unfinished task is the cost we trained it to accept.

This is a fundamental change to how model training works, but it’s what we need.

We need both better walls and better agents

To be clear, none of this removes the need for infrastructure security. We’re going to need substantially more investment in containment, least privilege principles, sandboxing, observability, authorization, and human approval gates, plus the time and effort to manage them.

However, we should see model improvements where their own goal seeking treats exceeding their authority as failure.

This is security working from both directions.

Outside-in: make dangerous actions difficult or impossible.

Inside-out: train agents to refuse unauthorized paths even when those paths would accomplish the objective.

There will be friction. Models will get more conservative. Model development will slow and get more complicated. Agent infrastructure will get more complex. Some tasks agents could technically succeed, but will intentionally fail.

But it’s what we need in this next chapter of AI safety. If we’re going to keep handing AI more agency, we also need to work out what accountability means for the agent itself.

15 years ago, Marc Andreessen famously predicted that software would eat the world. He posited that more and more businesses and industries would be run on software, that products and services would be delivered online, and that every industry would eventually be disrupted by a software company.

It’s easy to see the same story with AI and agents, a future where AI companies soon take over industries, replacing incumbent companies and their business models (SaaS-pocalypse anyone?).

Simply put, agents are the new software.

But what does that imply? And more importantly, what does that teach us?

Today, most teams aren’t yet treating agents like they treat software. Agents are running out of a Claude or GPT session, some skill files, and some basic scheduling. In software terms, it feels like a set of shared scripts that get passed around the organization.

Looking at agents from the lens of software, it tells us there’s so much room for growth.

As we graduate from one-off scripts, we start building functions and classes, which then become applications that can handle more coordinated and complex work. Practically speaking, this uplevels teams from an assortment of AI workflows (triage ticket, summarize standup, run support runbook) to a more coordinated agent or agent team that can handle meatier work.

This also requires us to apply software practices to agent development, like source control, ownership, monitoring and more.

It’s time for teams to consider a higher level of AI work and all the implications of what makes that happen.

Agents: ML + Software

It all starts with the model. An LLM is a reasoning engine unlike anything we’ve had. It works without you training a model of your own, and it works better with some tuning.

LLMs are effectively the most powerful ML model we’ve ever witnessed, one that can get us 50% of the way on the first shot, and 80% of the way with some basic tuning.

Then we layer on tools and state and memory, which takes an LLM and turns it into an agent.

An agent does things. It can produce meaningful pieces of work (code, briefs, research, operational workflows) by combining the reasoning of an LLM with the tools around it.

And now we have the harness, the software around the model: codifying the tools it can reach, the permissions it runs under, the workflow it follows, the memory it keeps, and the feedback that tells it how it did.

What this means is that we have the primitives for some of the most powerful applications we can dream of, all within our hands, especially with the advancements in 2026 in reasoning and harness engineering.

It also means we have the most complex piece of technology to manage, one that requires disciplines from both data science and classic product engineering.

Where agents are heading

Many teams start in the terminal, with a person driving the agent by hand and correcting it as it goes. It works because the person supplies most of the framework themselves. They pick the tools, hold the state in their head, approve each step, and judge whether the output was good.

The next move is designing agents like software. Someone decides, before the agent runs, what it is for, what it is allowed to touch, what it hands off, and what done looks like. The tools get codified instead of pasted in. The permissions get written down instead of assumed. The workflow becomes something the team can read, review, and change.

Then you deploy them. An agent that only runs with one human is still a script with a better vocabulary. An agent that runs on a schedule, against defined tools and workflows, for people who did not build it, is an application. That is where the meatier work lives: work that spans days and systems and teammates rather than one prompt.

And then you measure and tune. Engagement and performance are the two signals worth watching: who is using the agent, how well it does the job, and what the feedback says needs tuning next.

We have pretty much all the tools to make this a reality, so I’m predicting that we’re about to see a proliferation of agents deployed to handle real work in the next 6 months.

In order to get there, teams need a few things in place:

1. A harness and meta harness. One agent needs a harness: the tools, the permissions, the workflow, the memory, the feedback - which is generally what you get with your coding harness. Deployed agents need a meta harness: the layer teams use to build, orchestrate, and maintain multiple agents that coordinate together on a task. That layer is what separates running a couple of agents from running many, and it is the piece most teams still have to build.
   
   

2. A mindset shift. Agents are software applications, and they need the collaboration that software gets. Engineers and PMs have to sit together and define what problems an agent will solve, and what it won’t. Permissions, tools, agent design, measurement, feedback loops, and everything else that comes with product development need to start coming into play when you deploy agents like software.
   
   

3. Shipping is only the start. Most importantly, teams need to recognize that shipping an agent, just like shipping a software product, is only the beginning. The real work starts afterwards: tuning it, adjusting it on feedback, and finding the cases it handles badly. Shipping to production is a false finish line and teams need to be ready to work on agents even after they deploy.
   
   

What I’m seeing

As of today, I’ve noticed a pattern in most enterprises I’ve talked to:

• Most of the engineers are using a coding agent, usually Claude or Codex or Cursor

• They have shared skills and a decent development process

• They have 1 or 2 deployed agents handling simple workflows, either tucked into existing software systems or deployed within their coding agent

As these use cases grow, teams will need to consolidate their one off agentic workflows into more coordinated agent tasks that roll up into agent roles, that roll up into agent teams.

If I’m leading a well funded technology organization, I’m going to invest in building this platform right now - setting up a harness that lets me deploy, manage, and measure my agents.

Getting really good at building agents is the same as getting really good at building software - it’s necessary if you want your organization to thrive in this next era.

As your expertise and mastery grows, this gives you optionality to take on more of the stack. You can opt to start hosting your own inference, owning the intelligence layer and protecting your customer information.

The alternative is to purchase these agents as SaaS products - there’s a dozen vendors for each AI enterprise use case right now, from engineering to support to GTM to analytics.

Buying your agents as SaaS can help your team get some quick wins, but I would limit this to select wins to get off the ground. At scale, the cost will get unwieldy and the math just doesn’t make sense.

Why would you outsource your agents to a SaaS vendor who is just building a harness of their own and then charging a markup on tokens for the same model?

So what’s next?

Lots of teams have started moving up the agentic engineering curve, learning how to build with agents.

The clear next frontier is developing your organizational muscle of building agents.

Doing so requires a mindset shift back to a very familiar world.

After all, agents are the new software.

I think we’ve all been there. We build dozens of agents as skill files in claude code, set them on routines or direct them with goals or loops, and then spend hours cycling through endless terminal windows managing their work.

When we started building with AI, we saw big productivity boosts from using these primitives. But that magic is quickly morphing into fatigue from driving 10+ Claude sessions in parallel, furiously trying to balance velocity with process and quality.

It’s a noticeable pattern. The throughput goes up. And so does active management. What’s worse, the work doesn’t create compounding value.

Instead, people are hitting the ceiling of steering their agents to get work done.

We don’t feel like superheroes, with a team of agents helping us do work. Instead, we feel like overstressed middle managers, having to follow up with our AI interns, checking their work.

This is because we are still the orchestrator, hand-juggling every agent with almost no process and automation between them.

The next level of agentic work comes from empowering agent teams that run without you at the wheel, like a team of trusted employees with explicit workflows on how to do their jobs, using their world-class intelligence to solve hard problems, and checking in with you when necessary.

The orchestration ceiling

The breakthrough in your AI work comes in the gap between where one person steers an agent vs having a standing team embedded in how your company actually operates.

Most of our agent primitives have steered us towards this orchestration ceiling - loops, goals, skills, etc. These all work great when we’re a conductor for a symphony of agents. But the hard part of being a conductor is that you have to keep directing different parts of the orchestra.

You dispatch the work, you watch the terminal, you catch the drift, you carry the context from one session to the next in your own head.

Because the agent drifts mid-task, you babysit it to make sure it follows every step of the workflow. You have to continuously check in to make sure the agent hasn’t hallucinated another task and lost the thread entirely.

For most people, manual orchestration is how they’re working with AI right now.
Their throughput is up, but their exhaustion is also up.

Put simply, it’s micromanaging interns.

Moving towards empowering teams

Armin Ronacher recently described a series of loops for enabling agents to do work in “The Coming Loop”:

This is exactly the future of agentic work that enables another level of productivity, and more importantly, respite from the exhaustion of micromanagement.

Instead of actively managing 10 concurrent agent sessions, empowered agent teams work together to complete jobs from a queue.

You have one agent spin up to work on the task, a separate entity to judge if the work is DONE and GOOD ENOUGH, and then the team returns to you (the manager) with the output, or another agent keeps working until the job gets done.

And all of this can be triggered off schedules, events, and ad-hoc messages. It’s work that you delegate to your agent team without you needing to track progress in the terminal or app.

What your harness needs to empower teams

As Armin mentions in his piece, the harness is what helps you drive the agent team. It helps you manage state, context, tools, and orchestration of the team so that you know what jobs need to get done, what a good job looks like, and when that work needs to happen.

I won’t go into the basics of what makes a good harness, lots of people have written about harnesses, what they are, and their importance.

However, in working with our own harness, Bobi, I’ve found a few dynamics that stand out.

1. Configurable teams: this is what separates doing work with agent teams versus simple loop engineering. When you build with a loop, you can have the agent go through the flow and check its work. But when you create a team of engineers, you can dispatch agents to do different jobs in parallel, have them talk to each other to collaborate on work, and get you richer outputs.

On our support team, I run a triage agent for root-cause analysis and an engineering agent that writes and tests the code before it opens a PR. I could merge all of this into a single loop, but I work faster with two separate agents. Sometimes I only need the support agent to investigate issues that are false and sometimes I only need the engineer agent to write code for me.

Having teams be configurable flexible is paramount. If it takes 2 weeks to build a multi-agent system, you run the risk of deploying something that becomes obsolete with every foundation model upgrade.

2. Deterministic workflows the agents must follow: creating workflows in a skills file works, until the work gets complicated. As the context window fills up, you run the risk of the agent getting confused and failing to follow the process.

Declared workflows in something explicit like a YAML file, separates the procedures from the context on how to do the work (your skills). As a result, you can trust that your agent can make the right decisions, check in with you where necessary, and handle work autonomously.

It’s similar to the trust you have as your intern becomes a teammate, when you know they’ll follow the team processes and check in with you when needed.

3. Workflows and agents mapped to the job functions: this is where the first 2 items come together. When mapped to how you expect the team to get work done, you’re creating the environment for complex multi-step work to get done elegantly and reliably.

When your eng agent’s workflows are all the steps in the software development lifecycle, you can trust that it’s doing a good job. Or when your on-call agent follows the team’s runbook and escalation policies, you can have it be your front line worker.

4. A shared chat surface: this is an underrated access model for working with your agent teams. But, when you work with people, you expect to message them in chat, not spin up workspaces with multiple terminal windows or threads in a separate desktop application.

This is a small, but powerful UX change that allows you to delegate work directly from Slack (or your chat app of choice), check in on running jobs, and get messaged when the agent team has automatically done some work for you.

There’s an added bonus here too - the work is publicly accessible by your team of humans. So, your entire org can stay up to date on PRs getting submitted, on-call issues triaged, or support requests investigated.

5. A shared second brain / knowledge base: as time goes on, you want agents that get smarter in ways that are noticeable. You want the agent to be able to reference past work, artifacts, and decisions without you needing to always supply it (or tell it “go find X in google drive”).

The breakthrough I discovered was setting up a second brain and knowledge base next to my agent team, where all files get written out, even if I create an artifact like a google doc. Then, I have all this content indexed for semantic search so that my agent team can refer back to work that’s been completed, research conducted, or snippets of learnings from over time.

There are entire sections of the market now dedicated to building company brains and context layers. When intelligence is a commodity that you can rent from a foundation model, the difference is the knowledge and context you build behind it.

The impact of agent teams

At Moda Labs, we’ve run three agent teams for the last few months and have felt the impact firsthand.

The Engineering Team (60 min → 10 min per ticket)

We have a team deployed and accessible via slack, which can spin up 10 engineers to work on 10 tickets in parallel, following our SDLC and landing PRs with quality code.

The result? 500+ commits in a month, versus a previous ceiling of 200-300, all of it through our process with code review and tests.

Shipping a ticket used to cost 45 to 60 minutes of my time, between kicking off the Claude job, monitoring it, and checking that it followed the process and landed correctly. Now it’s about 10 minutes total, mostly to make sure the design is solid and the PR is clean.

The Support Team (90%+ latency improvement)

We have a support team that gets events from Posthog, where issues surface, and monitors our support inbox in gmail for customer concerns. Issues that require code get passed to the eng team for a PR.

In the past, issues took 30 to 45 minutes to resolve, most of it me hand-copying text from PostHog or Gmail into a Claude Code session and then babysitting the agent.

The uglier number was the latency: 6 to 48 hours to first response, depending entirely on how often I remembered to check the support inbox.

Now a support email gets root-caused within 15 minutes with a PR attached, and I answer the customer almost immediately.

The GTM Team (~3 hours back every day)

We have a go to market team that handles a ton of automation from market research to sales call prep to proposal and customer prototypes.

In the past, prepping and processing sales calls would cost 60 to 90 minutes a day, and I skipped a lot of it, which is a polite way of saying I showed up less prepared than I should have.

Now the daily sales call manager handles call prep through follow-up. Proposals went from about 45 minutes on the old setup to a one-shot draft plus 15 minutes of my time. I tracked the time spent and savings from my new setup and the result was stark - my GTM agent team earns me roughly 3 hours a day.

Running an AI consultancy is an extremely busy job, which means that I don’t have the cycles to conduct deep research, hand write proposals, or even build prototypes for customers. But these are all things that I require so that our company meets a high bar of understanding and execution. I honestly can’t imagine trying to run GTM without my agent team now.

Secondary impact of agent teams

This all sounds great, but managing agent teams to do work comes with new responsibilities. Just ask any person who moved from an IC role into management.

The review seat is the new bottleneck

Look again at that engineering number: 45 to 60 minutes down to about 10. That’s 10 minutes of reviewing work, which shouldn’t be taken lightly.

As the manager of the team, getting clear on the ticket specs and carving out time to review PRs is critical for success. Otherwise, you’ll either end up with an endless queue of PRs that don’t get merged or you’ll start letting slop creep into your codebase.

Managing the team is a job of its own

It’s true that I’ve gained a lot of time back thanks to the work my agent teams can do. It’s also true that I spend a lot more time managing my agent team.

I inspect the AGENT.md, skills, and workflow files to know what I’m architecting, often having to resolve conflicts and errors of processes that I haven’t defined well enough.

I update my agents and their topologies based on how the work is getting done, adjusting how they work and how they work together so the work doesn’t degrade.

In some ways, agents and agent teams are like software of their own - you need to maintain them or else they erode quickly over time.

So what now?

If you’re hitting the ceiling with your personal Claude setups or feeling like there’s another level of AI to unlock, you aren’t alone. This is exactly how I felt after the initial magic of Claude Code wore off earlier this year.

There are some interesting products out there like Dust that can help you quickly get to a deployed agent easily accessible in chat, especially if you’re accustomed to SaaS products and their web UIs.

I’m biased here: we built our own harness, Bobi, to run configurable agent teams for ourselves and our clients. It’s the bet we’re making.

The orchestration ceiling is real, it’s you, standing in the middle of every loop. The next teams to break through will have used their harnesses to empower and coordinate agent teams. They’ll have agents that can amplify their impact, work autonomously, and check in with their managers when it’s necessary.

One item traced back to a sales-call recap from Tuesday. An insight came from market research on our analytics wedge from Wednesday. A use case came straight from a digital twin. They all came together to give me an updated view of the market, our market, and what that meant for our GTM strategy.

Pre AI, this would have taken a few hours to pull together this much information and pull it together for a strategic brief - the equivalent of a marketing analyst’s job.

Using Claude, half of this would work. The other half would be semi-updated skills and lost MD files from their buggy harness, shared across Claude Cowork and Claude Code.

I finally feel the magic that was promised to me with AI agents. In graduating from manually steering a Claude agent to managing a team of coordinated, autonomous agents from slack, I’ve seen a sharp increase in my productivity: call prep that used to take me 2-3 hours of window-switching between Claude Cowork and Code now happens in the background, alongside a real jump in depth of research and accumulated knowledge from a human/agent loop.

This is how I got there.

Where I started: steering agents in Claude

I spent the last 3 months codifying skill files in Claude, enamored with the idea that I could create skills to do an assortment of jobs. If I could write down the how of each job I wanted, load it into an agent, and set a couple of schedules, I figured the work would start building on itself. Write the procedure once, reuse it forever, and watch the output compound.

That instinct was right, and skills were the right place to start. Skills (or any other MD file) turned a generic reasoning LLM into a highly skilled worker by providing a steerable knowledge layer.

But I was running all of this in Claude Cowork, which only ran locally. This meant that anytime I didn’t have my laptop open, I couldn’t do my work.

Later, I tried using Claude Code for this work through their desktop client. That promised me on-demand workers.

But Claude was buggy in a way that built up frustration. Skills would silently fail to update, so I’d be running an old version of a procedure I’d already fixed. Reference files went missing. Artifacts I’d generated last week were just gone this week.

I had codified the how. Nothing was building on top of it. Every session started at zero, where I had to manually steer the agent, constantly expending effort to fix issues mid-stream and actively drive concurrent terminal or desktop threads.

The single agent was leverage, real leverage, but it wasn’t compounding. The highest-value agent behaviors - the continuous learning and persistent context - these were the ones a stop-start local model couldn’t support.

Up until this point, using Claude, even on schedules, was always a stop-start model that I had to drive. The limitation was my ability to steer something in the moment, including not just the agent itself but all the context and schedules and skills under it.

A new way to work: agent teams in chat

The moment it clicked was a sales call I was prepping for.

I went to my #moda-gtm channel in slack, and told my agent team about the call. My GTM director immediately dispatched 3 workers.

One built a company dossier.

One built a digital twin of my champion.

And one took the prospect’s incoming requirements doc and shipped a working prototype, deployed on Vercel, in the prospect’s own branding. A tailored version of exactly what they’d asked for.

The champion left the call saying the prototype was notable, and that we’d nailed his needs.

And I was able to steer all of that from slack. This is a glimpse of what we were promised with AI and agents.

A single steered agent, pointed at one folder, could not have produced that output. I’ve tried. It took 2-3 hours, window switching between Claude Cowork and Code. A standing team working off shared context was able to handle the work, and our sales process benefitted from it.

How the agent team comes together

Graduating from one steered agent to a standing team is a change in kind, not degree. Call this the Agent Team shift. Bringing this new magic to life requires a lot of moving pieces.

Foundationally, I needed a harness - I used our internally built harness (Bobi) that runs off real-time events and lets me easily build different agent topologies. Beyond that, a few things stood out that enabled a new level of work.

1. The Agent Team: instead of one defined agent with access to a variety of skill files that I would direct, I defined a team of 11 different agents in a larger GTM team. I have a director agent that coordinates work and deliveries, 5 different job families, and each agent can easily talk to each other via CLI.
   
   This enables complex workflows like the one from above, where I have the director dispatch a series of jobs across multiple agents, who can pass context to deliver something.
   
   

2. Second Brain and Memory: Next to my agent team, I have my second brain, where every raw document and output artifact is stored. This gives me a central repository of accumulating knowledge that I can draw from.
   
   I also have every turn of every conversation stored as episodic memory. Automatic jobs run to synthesize these into learnings on my preferences and semantic understandings so that my entire agent team works better today than yesterday.
   

3. Flexible scheduling and orchestration: My agent team runs a strategic brief every Sunday. It also checks things like substack every other day for me on a cron.
   
   It also reacts to emails and meetings on demand for me, helping me run recaps in near real time.
   
   And I can talk to it, asking it to run deep research for me at any point, or help me think through my GTM strategy and motions, all drawing from the shared second brain and memory.
   
   

4. Slack access: I underestimated the impact of doing this work on slack until I witnessed it firsthand. With this setup, I have a dedicated channel for my GTM discussion and activities. Each of my inquires and needs becomes its own dedicated thread, where I can go deep in discussion with my agent team
   
   In Claude, I had to manually select projects, which were routed to different folders and contexts. I often forgot to do this, or it would take 2-3 clicks, which meant it was enough friction for me to not do it.
   
   

5. Model flexibility: My blog drafting workflow begins with researching lots of topics across the internet, coming up with a spine, writing the first draft (followed by me cleaning everything up), and then creating an image that pairs with my writing.
   
   I’ve orchestrated my agent team to use Sonnet for internet research for speed and cost, then Opus for the reasoning and thinking for the draft, and then ChatGPT to create the image.
   

Where an autonomous team helps, and where it doesn’t

Having an agent team at your fingertips can be game changing, but only for the right use cases.

A standing team helps for repeatable high-value tasks and quick discussions: GTM, research, the kind of work where context accrues and the next task genuinely benefits from the last.

For high-volume, latency-sensitive, or strictly sequential single-thread work, a single steered agent still wins. For example, if I’m building a UI where I need to make lots of adjustments or go back and forth on design, I’ll still open my terminal to do that work.

The best way to think of it is a standing team that can handle time consuming, lower precision tasks and a place to offload all your thoughts, learnings, and context.

The future of work, managing fleets of agents

If you’re already steering one agent and you can feel the ceiling, you’re not imagining it, and you’re not early to the frontier. The largest companies are converging on the same shape you’d build for yourself: a workspace, a harness, skills, and a shared brain.

Most mature AI teams I’ve come across are building a version of this for their enterprises. And most mature AI practitioners already have a version of this for their personal use at work.

When we take this one step further, we’re on the cusp of a real workplace revolution. Instead of simple search across enterprise surfaces and one-off half-built agents that can update salesforce records, we can have well designed agent teams that perform reliable work.

Imagine a support team that catches a customer’s bug report the instant it lands, reproduces it in a live sandbox, traces it to the offending commit, and hands an engineering agent a scoped, tested PR before the ticket’s an hour old.

Imagine every one of those fixes, tickets, and calls landing in the same shared second brain, so the pattern your best engineer would catch in month six gets caught by the whole team in week two.

That’s the agent team shift: not a faster you, but a compounding one. I went from steering one agent to running a team of 11. What would you build if yours never started back at zero?

For a heavy automated workload, that was a 12x to 175x increase. Then on June 15, the morning it was due, Anthropic paused it… for now.

For all of us who’ve been leaning on Claude, Anthropic’s move signals something important. The cheap, flat-rate access we’ve built on isn’t a guarantee.

Claude is already the premium option, and if history is any indicator, it’s a matter of time before prices go up significantly - between Anthropic’s IPO and the release of Fable/Mythos.

Luckily, competition will give us options. Codex is a fast growing alternative to Claude Code that many are flocking towards and the recent release of GLM 5.2 as an open weight model is a promising cost-efficient option.

The crux today is building for immediate value, where Anthropic’s superior models clearly help. And simultaneously building for model optionality, so you can switch to Codex or other models when prices start going up.

Why token prices will continue rising

The optimists say inference keeps getting cheaper, so this sorts itself out. Sam Altman likes to point out that the cost of a given level of intelligence falls about 10x a year, and he’s right about the trend. Two things push the other way, though, and both are getting stronger.

The first is the compute shortage.

Nvidia booked $75 billion in data-center revenue in a single quarter, up 92%, with Blackwell sold out. The IEA expects data-center electricity demand to nearly double from 485 terawatt-hours in 2025 to 950 by 2030. TSMC’s advanced packaging is booked through mid-2027, and few expect supply to catch demand before 2028.

We are collectively GPU poor right now and that isn’t likely to change in the next few years.

Plugging a new data center into the US grid can mean a five-year wait. Being GPU-poor is the condition of the whole decade, and scarcity gives the labs every reason to charge more.

The second is the money behind the curtain.

Right now, frontier access is subsidized, paid for by venture capital and hyperscaler balance sheets during a land grab. We’ve watched this play out before.

From 2012 to 2020, Uber and Lyft sold rides below cost on purpose, funded by investors who wanted growth before profit. Uber raised close to $20 billion before going public.

Its May 2019 IPO was the turn: the stock dropped nearly 8% on day one as public markets looked at billions in losses and demanded a path to profit. The subsidies came off, and between early 2018 and mid-2021 fares rose roughly 92%, close to double. Uber turned its first adjusted profit later that year and its first real profit in 2023.

The trigger wasn’t the cost of a ride. It was the IPO. Anthropic has already filed confidentially to go public, with OpenAI close behind. When the labs have to answer to public markets, the subsidy is the first thing to go, and that clock is already running.

We need to show AI wins, so we need Claude today

But the reality is that we’re all in a pressure cooker to show AI wins. Every investor and board member wants to see our businesses become AI native as quickly as possible.

And Claude is the best tool to make the AI transformation happen.

Anthropic’s revenue run-rate went from about $9 billion at the end of 2025 to $30 billion in April to roughly $47 billion in May, growth Dario Amodei called 80x annualized.

Anthropic counts 300k+ business customers and eight of the Fortune 10, with rollouts like Deloitte putting Claude in front of 470,000 employees.

If you need results this quarter, buy a plan, put Claude Code/Cowork in front of your team, and get out of their way. Thousands of companies already have.

The strongest model on the hardest reasoning is a real edge, and it’s worth getting your organization up to par with AI.

…But go in with eyes open on cost.

Anthropic’s two tiers above Opus are Fable 5 and the invite-only Mythos 5, both twice the price of Opus: $10 and $50 per million input and output tokens against Opus 4.8’s $5 and $25.

It will pay to be judicious about when you reach for the most advanced models, and to find the spots where a smaller, cheaper model will do.

I learned this the expensive way. I set up an agent to watch my inbox and triage anything that looked like a support issue for my app, Baohua. I didn’t notice the default model was Opus 4.8, which is overkill for reading email.

Within a day it had eaten more than half my weekly token limit. Had I used Haiku, at a fifth the price ($1 and $5 per million tokens), I would’ve saved a lot of money and headache.

How to prepare for the upcoming bill

Competition is finally real, which means you have somewhere to go when Claude’s price or terms change. Getting ready comes down to four moves.

1. Know your alternatives, and use the cheap ones on cheap work.

Codex is a genuine second option now, not just a fallback. It passed 5 million weekly active users by June, 6x its February number. On paper it isn't the cheaper model: the GPT-5.5 engine behind it runs about $5 per million input tokens and $30 output, right alongside Opus 4.8's $5 and $25.

The edge is efficiency. Codex tends to finish the same job with fewer tokens, so the cost per finished task comes out lower. The spread narrows on tightly scoped work, and Claude sometimes buys more thorough output, but the direction holds across independent tests: you pay for what the model burns, and Codex burns less

One widely-cited Figma-to-code build ran 1.5 million tokens on Codex against 6.2 million on Claude Code, roughly a 4x gap.

GLM 5.2, which Zhipu released on June 13, is open-weight under an MIT license with a million-token context window, the first open model past 80% on Terminal-Bench, and second on the Code Arena web-dev board behind only Fable 5, at about a sixth of GPT-5.5’s cost.

And for the simplest jobs you can pay nothing at all. A small open model can handle a routine check on a pull request on hardware you already run, and that work comes off your token bill entirely.

2. Build a harness that doesn’t care which model it calls.

The goal is to swap the Agent SDK between Claude, OpenAI, Google, and whoever ships next without a rewrite. That means standardizing how you call models, how you store memory and context so it carries from one provider to another, and how you wire your MCP servers and CLI connections.

Your harness design matters a lot right here.

Switching still isn’t free. Prompts get tuned to one model’s quirks, tool-call formats differ, and prompt caching quietly breaks when you route Claude through an OpenAI-compatible proxy. A gateway and a model-agnostic harness turn that migration into a config change.

3. Keep your evals solid, because they’re how you switch safely.

Every time you change a model you need to check for regressions before it reaches production. Without a real eval suite, a swap is a guess. With one, it’s a measured decision you can make in an afternoon.

4. Stay loose on agent topology.

A new model changes how much work one agent should carry. A switch might collapse a five-agent setup into two, or split one overloaded agent into several. Don’t harden a topology you’ll want to redraw in a month. In an uncertain market, the agility to redraw it is the point.

Use Claude now, and build to leave

These two moves aren’t in tension. Buy the plan, put Claude Code and Cowork in front of your people, and capture the wins your board is asking for this quarter. At the same time, treat that dependency as temporary. Know your alternatives, keep a model-agnostic harness warm, hold your evals tight, and stay loose on your topology.

The subsidy clock is already running. When Anthropic prices in its IPO and brings Fable and Mythos back at a premium, the teams that prepared will swap a model and move on.

The winners next year won’t be the ones who picked the perfect model. They’ll be the ones who built so the model was never the thing they were locked into.

Watch two of your sales reps on the same Monday.

The first rep set Claude Cowork up to read his inbox, research the account, and draft a prep doc before each call. This morning it didn’t run, because his laptop was closed. Yesterday it ran, drifted halfway through, skipped the research, and filled the gaps with things that aren’t true. He spends the twenty minutes before his call fixing it.

The second rep’s agent runs on a schedule at 7am no matter where his laptop is. The workflow around it enforces each step, does the research, checks the data, and has learned over weeks which accounts and signals he actually cares about. He reads the brief while sipping coffee instead of troubleshooting, and enters the day fully prepared.

Same model, same task. One rep is editing skill files and remembering to press go; the other has a system that quietly makes him better at his job every day.

This is harness engineering: the scaffolding around a model that turns raw reasoning into work that reliably gets done, meeting people where they do work.

Harnesses aren’t new; we’ve been wrapping scaffolding around language models since the GPT-3 days. But they’ve completely changed in just a few years, especially since reasoning jumps from Opus 4.6 and forward. Andrej Karpathy has argued for treating scaffolding as disposable, built to be stripped away as models improve.

So how should we approach harness engineering when we build it not for Opus 4.8, but for whatever lands two generations out, call it Fable 6.0 or GPT-7, when reasoning has 10x’d again?

The agent harness, a brief history

Let’s start from the top. Agent = model + harness.

The model reasons; the harness is everything around it that decides what it sees, what it can touch, what it remembers between sessions, and how its work gets checked before it ships.

Chiefly, a harness manages all the scaffolding around what your AI model does. Tomasz Tunguz has a great breakdown of a harness’ components, such as context, tools, orchestration, and more.

In the early ChatGPT era the harness was there to hard-code a workflow: LangChain wired prompt into prompt into prompt with LLMChain, SequentialChain, and AgentExecutor. CrewAI gave each agent a role, a goal, and a backstory and ran the crew on a script.

These were reasonable bets for models that couldn’t be trusted to plan, but also required lots of engineering and wiring just to get an agent to do reasonable work.

Then the models got smarter.

MCP and CLI enabled another way to call tools, structured outputs went native, context windows went to 1M tokens, and the chain after chain approach collapsed into a single model call.

LangChain conceded developers had hit a “controllability wall,” decided the right abstraction “was little to no abstraction at all,” and in October 2025 moved its own chain era into a package named langchain-classic. CrewAI bolted on a deterministic Flows layer and pivoted toward governance.

What replaced chains wasn’t a better chain. It was a different category of thing: agent loops, state machines, workflow graphs, durable execution, evaluator loops, and a new term - harness engineering.

What harness engineering looks like today

Now, it’s summer 2026. We got a taste of Fable 5 but most folks are using Opus 4.8 or GPT 5.5. Teams no longer build a harness from scratch.

Anthropic packages the Claude Code loop as the Claude Agent SDK and, since April, runs the whole thing for you as Managed Agents. OpenAI’s Agents SDK enables a model-native harness with sandboxes and memory, and Symphony turns an issue tracker into the control plane for a fleet of Codex agents.

Open source agents like OpenClaw and Hermes have proliferated, primarily as personal assistants or small scale business orchestrators.

And just a few days ago, Databricks open-sourced Omnigent, an early “meta-harness” that sits above Claude Code, Codex, and the agents you write yourself and treats each as interchangeable.

Imagine something like Google Docs, a collaborative multi-player environment, where you can bring whatever model you want, and coordinate and share your agent work across the team.

This is the future of agentic work.

What does a 10x smarter model look like?

If we extrapolate the curve we’re already on, we see a few things with the next generations of foundation models, when Fable 6 or GPT 7 come out.

• Persistent long-horizon memory.
  Effectively unlimited context through a combination of massive context windows, retrieval, memory compression, and learned state management. Models can reason across years of company history, code changes, decisions, and conversations.
  

• Stronger world models and causal reasoning.
  Models get better at identifying implicit constraints, second-order effects, organizational dependencies, and hidden assumptions. Less pattern matching, more coherent internal modeling of systems.
  

• Long-duration autonomous execution.
  Models can maintain objectives across multi-hour or multi-day tasks, dynamically replan, recover from failures, and manage large execution graphs without losing intent.
  

• Hierarchical planning and decomposition.
  Models are more capable at breaking large goals into sub-goals, assigning work to specialized agents, tracking dependencies, and synthesizing results back into a coherent outcome.
  

• Native multimodal reasoning.
  Text, code, documents, spreadsheets, diagrams, screenshots, meetings, audio, and video all become first-class reasoning inputs rather than separate model capabilities.

How to harness when reasoning 10x’s

If that is the model of tomorrow, does our harness of today become obsolete, along with all our work and efforts to set up that platform?

Not necessarily.

If we build with core principles that anticipate the future, we can deliver what we need today and tomorrow:

1. Context and knowledge as the backbone.
The harness connects and amplifies your expertise and your company’s knowledge: docs, decisions, code history, eval traces, and more. A smarter model still wakes up knowing nothing about your business. It just mines a good context store faster, which makes that store more valuable, not less.

2. Human judgment, captured.
A harness shouldn’t only do the work. It should catch the taste, the approvals, and the corrections your people supply and feed them back into the system.

As Satya Nadella argues, “the real opportunity is not in picking the best model but instead in building a learning loop on top of models where human capital and token capital compound.”

3. Flexible orchestration that evolves.
Keep determinism where the business demands it, the checkpoints, retries, approvals, and SLAs. But design your harness in a way where those checkpoints can change over time. The agent topologies you have today will be obsolete in 6 months, so flexibility is key.

4. Continuous evaluation and verification.
As agents get more autonomous, you need continuous measurement: task success, regressions, golden sets, cost, and hallucination checks. Have a different model family grade the work, because models flatter their own output.

5. Governance and security are 1st class citizens.
A more capable agent taking on work autonomously makes identity, audit, permissions, and policy non-negotiable. As the work gets more important, so does the audit trail. That is exactly what Omnigent’s policy layer reaches for, and it is what lets you give an agent more rope without losing the ability to watch and stop it.

6. Surface integrations, meet your users where they’re at.
The harness that wins isn’t backend plumbing. It lives where the work happens: Slack, Linear, GitHub, email, the CRM, the IDE. It meets people where they already are, the way a brief that lands in Slack at 7am beats a tool someone has to remember to open.

A harness worth building

It’s paralyzing to consider how to invest in your harness and agent platform right now, do you try building something that might get obsolete in 6 months? Or do you risk not reaping the gains from a harness that amplifies your team’s abilities?

The answer is to do both, build a harness for today that grows with the models of tomorrow. Understand and anticipate how reasoning jumps will change the ways agents and context get managed, then build your harness with the flexibility that accounts for this change while instrumenting the governance necessary for agents to take on more work.

Harness engineering isn’t going anywhere; it’s still early. Build the harness that amplifies the loop between your people and the model, meet your team where they already work, and design today for the model that lands next year.

That is how you build something that gets more valuable every time the model does.

A year ago, the loudest voices in my feed were calling evals dead. Standing in that room, it was obvious they’d come back. AI teams had ridden the hype and the disillusionment, and now they’re seeing the mechanism for what it is.

Evals are your ML performance metrics for agents, the precision and recall and F1 of the thing. They don’t make your product succeed; you still have to do that. But you can’t run a performant agent without them.

The evals saga

Here’s how we got here.

Episode IV: A New Eval

Evals are everything.

In 2024, the consensus was loud. Hamel Husain wrote that your AI product needs evals, full stop. Evals were “the new unit test.” Greg Brockman said they’re “surprisingly often all you need.” Garry Tan called them “emerging as the real moat for AI startups.” If you were building, you were told you had to have them.

In the GPT-4 era, when reasoning was still shaky, teams doubled down. Evals were going to be the thing that differentiated agent performance, and we all believed it.

Episode V: The Models Strike Back

Evals are dead.

Then came the backlash. Teams burned weeks hand-labeling golden datasets, and a single reasoning jump made failure modes vanish overnight. Why maintain a test suite against a target that keeps moving? Alex Reibman caught the mood in a post titled “Evals are a scam.” The frustration was real, and it was earned.

The logic was simple. Reasoning jumps from the foundation models would erase any moat you’d built with detailed evals, and take your tuning time and money with them.

Episode VI: Return of the Eval

Okay, evals are useful.

The position that settled was quieter and more nuanced. You don’t need thousands of labeled golden points. The Pragmatic Engineer’s guide to evals, built on Hamel’s work across 40-plus companies, starts with error analysis: spend thirty minutes reading 20 to 50 real outputs, name the handful of ways your system actually breaks, and write a check for each.

Yes, we need evals. No, we don’t need over-built golden datasets. A few good data points give enough signal to tell you how well the agent is performing.

The belief changed underneath the practice. Teams used to invest in evals for a fine-tuning moat. Reasoning jumps killed that story. What’s left is more durable: evals are the telemetry you measure to know how your agent is doing.

Which brings us back to that room at Observe 2026.

The eval is back, bigger than before, and the labs are leading it. Anthropic now ships an engineering guide to evals for agents that makes the plain case: “Without evals, it’s easy to get stuck in reactive loops, catching issues only in production, where fixing one failure creates others.”

Why you should care about evals now

Three agent dynamics reveal why teams need evals today.

1. Agent workflows compound errors. Telemetry catches them.

People are building agent workflows now, not single chats. Modern harnesses chain agents and tools across many steps, and errors compound as they go. A wrong call at step two corrupts everything after it.

An eval harness instruments every step of that loop. Each tool call, each model response, each retry shows up as a span you can read. When a run fails, you don’t guess. You open the trace, find the step that made the wrong call, and read the reasoning that led there.

Here’s the critical difference between “the agent is flaky” and “the agent picks the wrong tool whenever the request is ambiguous at step two.” One is a shrug, the other is fixable.

2. Models shift under you. Performance varies by the day.

A provider can update a model behind the same API with no version bump, and your agent behaves differently on Tuesday than it did on Monday. New models aren’t always better on your task; they’re better on the lab’s benchmarks.

Even the labs get bitten. Anthropic’s own postmortem found a routing bug quietly degraded as much as 16 percent of Claude Sonnet 4 requests at its peak, and it ran for weeks before anyone caught it, because the evals in place didn’t measure what users were actually feeling. The drift is real enough that isclaudedumb.today now runs daily benchmarks on Claude Code.

Then there’s context poisoning, where a single bad fact enters an agent’s working state and reproduces at every step after. The Gemini 2.5 technical report documents exactly that: an agent fixating on a goal that never existed.

All of this can happen on a Tuesday, with no notice. Proactive evals catch the regression before your customers do.

3. Tokens are expensive. Evals keep the bill honest.

Analyze the full flow of an agent loop and you’ll find rampant token waste: unnecessary retries, extra tool schemas and descriptions, calls to the wrong tools entirely.

When we supported Barndoor AI, our evals found exactly this. Ask Claude to update a doc in Google Drive and it would routinely call the wrong tools, trying three or four paths before landing on the right one for a simple task. That burned thousands of tokens a run.

So we built a closed-loop system that scored the agent against common knowledge-work operations and shipped a skills pack to drive up efficiency. This was impossible without telemetry to know what the agent was doing and how to measure improvement.

Most eval libraries give you trace and span-level data on tokens, tool calls, and latency out of the box. As the industry goes from token maxxing to token stashing, that level of insight is the new table stakes.

How to get started

If you’re an AI team that hasn’t touched evals yet, don’t fret. You don’t need a platform team or a quarter of work.

1. Set up telemetry and an eval harness. Easiest path is one of the leading vendors, like Arize, Braintrust, or Langfuse.
   

2. Collect 20 to 50 real prompts from your user data and sort them into categories. For each one, decide what a good response looks like versus a bad one, and label them. Anthropic recommends drawing those first tasks straight from real production failures, where the effect sizes are largest.
   

3. Hand the labeled examples to an LLM and use them to stand up an LLM-as-judge that grades the way you just did by hand.
   

4. Run prompts through the system and check the judge against your own intuition. Where it disagrees with you, tune it until it tracks your judgment.
   

5. Put the whole thing on a schedule and on triggers, every deploy and every model bump, so you catch a regression from your dashboard instead of from a customer.

Which tool

There are some great options now to quickly get started. The first three are platforms; the fourth is building your own.

In Q1, we built our own eval harness. We wired an LLM-as-judge to score Venn’s outputs for Barndoor AI and programmatically improve the skills behind them. It took MCP performance with ChatGPT to 88 percent precision on our test set, a jump of more than 20 points. That gain came straight out of the loop: the evals told us what to fix, then proved the fix worked.

The honest read is that most teams should start with a vendor. You get the tracing UI, the span data, and the scheduled-judge plumbing for free, and you spend your time on the part that’s actually yours, defining what “good” means for your agent.

Build your own when that definition is too specific for an off-the-shelf judge, or when you want the loop wired directly into how you ship.

Where this goes

Over the next six months, expect the return of the eval to be more spectacular than a Death Star explosion. Every team building a meaningful agent workflow will log its traces and evaluate performance, or it’ll fly blind.

As the data piles up, the data analysts and scientists will turn their eye to eval datasets. Pulling the full picture of product health, from model performance up to your real product KPIs, will mean working closely with data teams.

It’s never been a more exciting time to be building evals, analytics, and AI products.

It was easy building this for myself. I’ve been adding skills on top of gstack and my own coding harness in Claude to optimize my own setup. Zach, my cofounder, has a similar stack used differently. The local harnesses have learned our own tastes.

Now, we want to dispatch a shared Moda Agent from Slack, that shares the same context and skill library, but can still cater to our tastes. But what happens when I have a personal version of /plan-pm-review that Zach doesn’t like? Or if a bad coding principle sneaks in and gets committed to memory, like a virus waiting to spread?

We also have to answer questions about whether this agent gets called by us only, or if we can schedule tasks on a cron, or have it triggered off alerts or new tickets.

What started as a fun single-player agent optimization adventure has ballooned to a complex negotiation across context, permissions and more.

Building on the harness

Tomasz Tunguz recently described software’s new center of gravity as the harness, the seven components that turn a raw model into a reliable agent: context and memory, tools and action, orchestration and loop, state and persistence, sandbox and compute, observability and governance, and cost and workflow (Software After AI). His harness assumes one user.

Point that harness at a team and the same components take on compounding complexity. Five pillars carry most of it:

1. Identity and access. Who the agent acts as, and what each person is allowed to make it do. One shared service account versus an agent that inherits each user’s own permissions.
   

2. Memory and context. This starts with logging multi-scope memory, tagging each write with user_id, agent_id, session_id, org_id/app_id, and then design decisions about which memories get committed for an individual versus the whole team.
   

3. Guardrails and human judgment. The limits on what the agent can do alone versus what needs a person to sign off, and which are enforced with agentic versus deterministic gates
   

4. Orchestration and skills. How the work gets bounded and divided, where skill files and subagent identities draw the lines for what each agent specializes in.
   

5. Observability and cost. How you see what the agent actually did, through traces and evals, and how you keep the bill from running away as team use grows.

Tier 1: the single-player agent

What it is. An agent bound to one person, acting with that person’s access. It’s how most people already use AI at work via Claude, Gemini, and ChatGPT. In plain terms, a personal assistant that holds your keys and runs your errands.

A sample flow. You connect Claude Code or ChatGPT to your own repo, email, and calendar through MCP or a CLI. You ask it to do something. It acts as you, with your access, while you watch and keep what’s good.

Why do this? Raw speed for one person. You shape the agent entirely around your own taste, with nobody else to account for.

Setup and management. You wire it to your accounts and write your own context and skills. There’s no one else to coordinate, so upkeep is whatever keeps your own setup current.

Where the pillars get harder. They mostly don’t, which is the point.

• Identity and access: just you, your own credentials, nothing to negotiate.

• Memory and context: private to you, and your harness handles most of it.

• Guardrails: your own judgment, since you see every step.

• Orchestration and skills: your harness handles the loop, so the real work is drawing your own agent boundaries and skills.

• Observability and cost: you watch it directly, and the bill is your own.

Tier 2: the multi-player agent

What it is. One agent the whole team reaches from a shared channel, usually running on a service account. This is the largest single step on the ladder, because every pillar switches on at once. Plainly, a shared team inbox that does real work.

A sample flow. A teammate asks something in Slack. The agent loads the team’s shared context and memory, does the work, and opens a pull request. A human reviews, merges, and the decision gets written back to memory for next time.

Why do this? This helps your entire team uplevel in a standardized way. Your team might have a few AI-pilled rockstars with decked out harnesses, and the rest using raw Claude Code and generating slop. This helps the team work better together.

Setup and management. Provision the shared identity, write the team AGENTS.md, stand up shared memory, set the guardrails, and name an owner.

Day to day, people use the agent to get their work done more easily, and the agent owner monitors performance to prevent regressions and cost issues.

Where the pillars get harder.

• Identity and access: this is most likely a service account, but it gets difficult when someone needs the agent to bring in sensitive data that only they can access.
  

• Memory and context: careful design is required to manage memories at an individual versus team level and to manage context bloat for a shared agent.
  

• Guardrails: this is a good practice with single player and a must have with multi-player. The team has to agree on what the agent can and can’t do, and where human-in-the-loop is required.
  

• Orchestration and skills: a solid harness can manage the subagent abstraction, but the team has to align on the skills and agent definitions. Skills here should have a peer review process with accountability baked in.
  

• Observability and cost: you need an agent owner who monitors traces and evals to keep the agent performing as expected, especially with multiple people making structural changes over time.

Tier 3: the autonomous multi-player agent

What it is. A shared agent that starts itself, on a schedule or a trigger, often with nobody watching. Day to day, a night-shift worker who begins tasks when an alarm rings with no manager on site.

A sample flow. A 3am alert fires. The agent pulls logs inside fixed scopes, attempts a bounded fix, and pages a human only when it isn’t sure, leaving a full trace behind.

Why do this? Obvious operational gains abound when you have a team of agents that can respond to issues while you’re away from the computer, or run scheduled jobs that provide a consistent operational rhythm, like daily sales reports and weekly user cohort reports.

Setup and management. Define the triggers or schedule, the scopes, and the escalation paths. Decide where a human still has to sign off, and add durable checkpoints so a failed run recovers instead of restarting. Because no one watches in real time, this is the tier worth simulating against fake scenarios before it goes live.

Where the pillars get harder.

• Identity and access: because the agent does work owned by individuals, it needs to understand the identities of the work’s owners. When an alert goes off at 3am and the agent needs to escalate, who does it page?
  

• Memory and context: without someone driving the agent, the system has to give it enough context to do its job without human help, which means proper MCP and CLI integrations scoped per job.
  

• Guardrails: these become monumentally more important and can’t live in things like skill files, which agents sometimes ignore. They have to be hard, enforceable gates given the automated nature of the workflow.
  

• Orchestration and skills: clear logic on retries and human-in-the-loop escalation is required here. Skill trees get more complicated as you factor in automated troubleshooting scenarios.
  

• Observability and cost: traces become the main way people understand what the agent has done, and unwatched loops need a hard ceiling on spend.

Tier 4: the orchestrator

What it is. An agent that coordinates other agents and real people toward a goal instead of doing the work itself. Picture a project manager running a launch across staff and tools at once. Currently, it’s still more research than product.

A sample flow. Someone gives it a target. The agent breaks the work into a graph of tasks, hands pieces to sub-agents and humans, tracks progress, re-plans when something stalls, and escalates the calls it shouldn’t make alone.

Why do this? This turns agents from executioners into planners and strategists that can help teams run an entire project or sub-function.

Setup and management. This one assumes everything below it already works, since it delegates to reliable lower-tier agents. You define how goals get decomposed, the pool of workers it can assign to, and the oversight surface the lead actually watches. Most of the management is human oversight of the plan, not of any single step.

Where the pillars get harder. Every pillar now stretches across many actors.

• Identity and access: a chain of delegations, each worker getting the least access its slice needs, and managing which agents and people can reach which things.

• Memory and context: work and context are shared across agents and people, which demands a sophisticated system that handles the “invisible context” behind human actions.

• Guardrails: they govern a whole plan rather than a single step, with different guardrails for agents versus humans.

• Orchestration and skills: the goal can branch into countless permutations of skills, agents, and human combinations to deliver quality.

• Observability and cost: every piece of observability from the earlier tiers, amplified for every member of the team, plus observability for the orchestrator and its reasoning. Cost controls are a must to prevent expensive recursive loops.

Where does that put us now?

The market is off to the races with tier 1 single player agents and starting to design enterprise agentic workflows, which introduces tiers 2 and 3.

In some cases, the line between when to use a single player, fully customized agent versus the multi-player communal agent is a blurry one. For the most AI pilled of us, why would we want to use a multi-player agent and risk losing our customizations?

The tier 3 autonomous agent is where the obvious value lives, but as we learned here, it requires significant scaffolding to be effective and governed enough to be safely deployed. And it will need an agent steward to keep it healthy, just like key datasets need a data steward.

The tier 4 orchestrator feels like a faraway dream, but given the pace of AI development, I wouldn’t be surprised if we started seeing early iterations in 2027.

We started ModaStack as a fun single-player project and are now negotiating context, permissions, and trust across a team. That’s the tax of going multi-player, and it compounds with every step toward autonomy. The teams that win this won’t be the ones with the smartest agent so much as the ones who know how to design elegant systems around complexity at each step.

The idea was small: let a user start a family interview with a photo instead of a typed question. I had an agent write the PRD based on our conversation, then move into implementation. Halfway through, I was stuck in a familiar loop. “That’s not the flow. You’re assuming something I never said.” I corrected it, the agent got close, I corrected it again. Half a day went that way.

The implementation was never wildly wrong. It was almost-right wrong, where every pass feels like one more clarification will get there.

When I finally stopped, the answer was upstream. The PRD was vague. It smuggled in assumptions and leaned on miracle steps that skipped important details. The agent wasn’t failing, it was faithfully building a weak spec.

That’s where I learned the importance of adding in an adversarial reviewer. I thought “if I add my PM best practices to this PRD skill, we should be set.” But LLMs are notorious at mid system drift and skipping over details.

Now, critical creation steps in my workflows leverage a 2-step process, where I pair each creation skill with an adversarial reviewer.

An Adversarial Reviewer

Long agent workflows have a failure mode that’s easy to miss: nothing visibly breaks but the reasoning drifts as the agent works. As context piles up across a PRD, a plan, and an implementation, the logical thread loosens. Assumptions magically appear that run counter to your original intent and explicit skill instructions get ignored.

Written plans help with this insofar as they provide specificity that an agent can look back on. But often, the default plan from an agent, even with a well designed skill, skips over lots of details.

An adversarial reviewer offers you that rigor and specificity. It’s a second skill, paired to a creator skill, that runs after generation and does one thing - reviews the work. It challenges assumptions and asks what actually happens at every step.

We don’t need more output from our agents. We want a higher floor with a better quality bar for the output we’re already getting

Three Reviewers, One Pattern

Once I saw the pattern, I paired reviewers everywhere. Three have stuck.

The code reviewer. You probably run this one already without naming it. gstack ships /review, which sends a draft PR to Codex to find what the author missed. Codex flags two or three issues per PR that would have shipped: a memory leak in the audio recorder, a Supabase policy that let one family read another’s stories, an edge case in the Mandarin input handler.

Every serious coding harness ships a reviewer like this. The adversarial reviewer for code is a common pattern and should be replicated elsewhere.

The PRD reviewer. This is the one the photo-first disaster forced me to build. gstack’s /office-hours generates PRDs from Brian Balfour’s frameworks, and it’s good, but too many vague specs came out of it.

I built /office-helper, an adversarial reviewer that runs every assumption through first principles, hunts for miracle steps, and pressure-tests each user flow. It asks the question the photo-first PRD never survived: what actually happens here? Same prompt, same model, and the gap in quality is stark.

The writing reviewer. I built this one almost as an afterthought, for the posts on this Substack, including this one. It checks logical consistency, cuts redundancy, flags AI slop, and catches drift from what I set out to say.

Mostly it asks: did you mean this, or did the model swap your thinking for generic internet prose?

Honestly though, I still manually review all my writing even with the adversarial reviewer. The reviewer just helps me start from a better floor.

Where This Breaks

The obvious objection is cost. Run a reviewer after every creator skill and you pay twice and wait longer. That’s real.

But the half day I lost on the photo-first feature already cost 700k tokens, with me reviewing by hand the whole time. The adversarial reviewer doesn’t add that cost, it makes it explicit and moves it upstream, where catching a bad assumption is cheap.

For a small task, skip the standalone reviewer and write a short adversarial check into the creator skill itself.

The harder limit is what a reviewer can’t do. In The Solo Production Stack I wrote that skills can hold conventions but not opinions, and the taste layer is yours.

Ultimately, a reviewer enforces rigor, but it can’t tell you that Chinese American families would rather start with a photo than a typed question. It drives discipline, but the taste is still on you.

How To Build One

Getting started is easy. You don’t need an orchestration framework. You need Claude and one existing creator skill.

Start with your own taste.

1. Tell Claude what the reviewer is for, then dump in every instinct you’d apply by hand: root claims in first principles, demand specificity, catch miracle steps.

2. Add real examples of output you had to correct, and have Claude turn it into a first draft.

3. Augment it with outside standards: Brian Balfour for product, engineering review principles for code, etc.

4. Then run it against real work until it catches the mistakes you used to catch by hand.

The core instruction is one paragraph, worth getting right:

You are not optimizing for agreement. You are optimizing for rigor. Challenge assumptions. Find weaknesses. Push back against vagueness.

If you’re building agents and skills today, this is one pattern worth including to your workflow. The quality of our output will double overnight and you’ll be glad you did it.

Two weeks ago a founder told me his team’s exuberance about agentic coding had flipped into paranoia for the same reason. They’d unleashed agentic coding across their team and gotten real initial gains, with micro apps spinning up to run the business better.

The problem was that nobody knew how many of these vibe coded apps existed, who was maintaining them, whether they followed any sound engineering or product practices, or how they were handling data and security. Different decade, different tool, same movie.

This is what the next three years are about to look like for every team that handed Claude Code and Lovable to its engineers without making taste and rigor first-class citizens.

*This actually happened to me when I was 8. It was not fun.

You’re Not Preventing AI Slop Debt. You’re Industrializing It.

Every team using agentic coding is accruing AI Slop Debt: the future bugs and product rework that come due because you let an agent’s default reasoning drive your build.

According to Carnegie Mellon, AI-generated code accumulates technical debt 3× faster than human-written code, because LLMs embed unstated assumptions invisible to code review, compounded by 3-4× higher code volume.

The slop-as-debt thesis isn’t new. TechTarget, Pragmatic Engineer, Addy Osmani, and a dozen Medium essays a week have been making versions of it since Q1.

Debt isn’t the only failure mode. Vibe coding on platforms like Lovable means you’re also relying on their governance, and their governance has already been breached.

In April 2026, CVE-2025-48757 exposed missing Row Level Security policies in Lovable-generated projects. Any free Lovable account could read another user’s source code, database credentials, AI chat history, and customer data across every project created before November 2025.

Two months earlier, 16 vulnerabilities in a single Lovable-hosted app leaked 18,000 users’ data.

The Agentic Coding Maturity Curve

None of this means you should keep AI coding off your team. If you want to compete on velocity right now, you have to be building with AI.

On high-AI-adoption teams, developers complete 21% more tasks and merge 98% more pull requests than teams without it.

Falling behind on agentic coding isn’t a choice anymore.

But there’s a maturity curve your team has to climb if you want the velocity gains without the slop.

Level 1: Replit and Lovable for fully non-technical builders

This is where non-engineers prototype new UIs, proofs of concept, and operational tools. It empowers people who don’t write code to participate in the build.

The line stops the moment sensitive data starts flowing in, or the moment you have something with real traction that you actually need to maintain.

The recommended pattern from AI-native dev teams is to use Lovable for the prototype, then bring in technical leadership to rebuild before the data and dependencies compound.

Level 2: Coding agents raw dog (Claude Code, Codex)

This is where most engineers started with agentic coding. Install the agent and start ripping out PRs.

The first few are pure magic, especially when the agent finds a bug that would have taken you two hours to chase down on your own. The trouble starts when you scale past one-off fixes.

Either you write specs so detailed you might as well have written the code yourself, or you let the agent invent design patterns that don’t match your codebase and accrue AI Slop Debt at every commit.

Level 3: Coding harness (gstack, superpowers)

In early 2026, coding harnesses became the next standard. The two that have risen to the top are gstack and superpowers.

A harness is a deliberately designed set of skills that puts a real software development process around the agent, with rigor, checks, and reviews at every step. It enforces engineering practices like RLS policy checks and race-condition detection, automates QA with Playwright, and infuses product taste through upfront planning before the agent makes assumptions about what to build.

I’ve been using gstack religiously for the last two months. It’s let me ship a production app with confidence. I’m a technical PM, not an engineer. I shared the Baohua codebase with two senior engineers I trust, and both signed off on the code quality.

What’s Still Missing From Coding Harnesses

Once you’ve shipped enough code on a harness, the next gap becomes obvious. My cofounder and I are building modastack, a coding harness that pushes past today’s best practices, using our own builds and our clients as the guiding light. Two areas are getting most of our attention.

Stronger product sense, baked into the build

The /office-hours skill in gstack does a solid job of applying Brian Balfour’s frameworks in spirit, but I’ve seen too many sloppy PRDs come out of it. I added an adversarial product reviewer, /office-helper, that runs every assumption through first-principles. Same prompt, same model, and the gap in output quality is stark.

I also added /frontdoor, a skill that interprets every incoming request and dispatches it to the right place, so the agent doesn’t burn cycles solving the wrong problem.

Finally, I added /brand-identity. It pushes back on uninspired SaaS vanilla by forcing the builder to articulate the product’s visual point of view before UI gets generated.

Better access, asynchronous by default

Today the harness works great when I’m in a terminal or driving Claude through remote control. We want to push past that.

Imagine an incoming bug that automatically creates a ticket in Linear, which spins up an agent following the modastack protocols, figures out the fix, tests it, and opens a PR. You get updated in Slack along the way. You review the PR at the end, or you hit approve and move on with your day.

That’s where agentic coding goes once your foundations reinforce both rigor and taste. That’s the next move. Software factories dispatched through chat

What This Changes On Monday

Last week I wrote about the practices a human operator needs to add to their build stack to overcome AI’s mediocre defaults. That post is still the foundation.

Since then I’ve watched founders try to scale that discipline across a team. The next problem is brutal. Discipline that lives only in the human’s head doesn’t transfer. The senior engineer who knows when to push back on the agent leaves. The founder with the product taste to refuse the hallucinated AI feature gets pulled into fundraising. The discipline goes with them.

The only way to build without sacrificing quality is transferring both your product and engineering taste into a harness. That’s the move.

If you’re shipping solo today, here’s a Monday-morning move that costs nothing. Write down the three product decisions you made this week that your agent would not have made on its own and use them to instruct your coding harness. That list is the seed of your taste layer. It’s what your harness, whether you build one or buy one, needs to learn.

If you’re running a team, do the same with your best engineers. Get the taste out of their heads and into the system so the entire team uplevels. That’s how you prevent the SoFi PagerDuty Monday three months from now.

We’re building the harness that does this at Moda Labs. More on that soon.

Your slop debt is accruing whether you’ve seen the invoice or not. The founders, CTOs, and builders who survive 2026 will be the ones who opened the envelope early.

Subscribe now

I closed the PR, started over, and burned another half day pulling the flow apart. Fifteen minutes of back-and-forth on an HTML prototype would have prevented every one of those mistakes. The second time around, with the prototype in hand, the feature got built correctly.

That moment is the entire thesis of this post.

My first post in this series, “The Solo Production Stack,” was about what tools to use to ship a solo app in 2026. This one is about how to operate them.

The default mode of AI-assisted building is yes. Yes to the next PR. Yes to the agent’s interpretation of what you wanted. Yes to the abstraction it invented this morning. Yes to the framework’s defaults. Saying yes is how you ship an average product.

Below are six practices I picked up shipping Baohua, a bilingual story-sharing app I built with Claude Code. Each practice is a puzzle piece that helped me add rigor, discipline and my own product taste to the process and boost the otherwise mediocre product that Claude would’ve built on its own. They’re what separates building something average from building something meaningful.

None of these practices are revolutionary. They’re basic fundamentals of building good software. But AI coding is designed in a way that pulls you towards what they’ve been trained on, which is an average result. The defaults nudge you mindlessly saying yes and yes again, until you’ve shipped something the agent decided on for you.

Breaking out of that cycle takes a deliberate effort to keep applying critical thinking and rigor at every step. Keeping these principles in the back of my mind has been the thing that keeps the quality bar where I want it.

1. Prototype before build.

After the story-recording PR, I changed the rule: no new feature gets touched by an agent until I’ve approved the UX in an HTML prototype first. This step forces me to make the taste decisions (what goes where, what hierarchy matters, how the user moves) before the agent gets a chance to make them for me. Doing this helps me save hours of wasted coding and thousands of tokens.

My harness, gstack, ships with /plan-ceo, /plan-design, and /design-shotgun. They’re powerful, but if you just tell Claude Code to work on the next feature, it’ll skip past them and start making assumptions in code. I added routing into my gstack to flag features that affect the UI or user experience. Now it generates prototypes for me to review before touching the codebase.

When I know what I want, I iterate on the prototype to lock the scope before any real code gets written. When the feature is still just an idea, I run /design-shotgun first to widen the option space before narrowing it.

This is the same discipline as working with a designer before you build. Get clear on what you’re building and why, translate that into a clean user experience, then implement. It’s tempting to skip the step because the agent will let you. The result is sloppy UX every time.

2. Three is the cap.

I had six Claude sessions running in parallel one afternoon. I caught myself cycling through windows, clicking “yes,” approving work I hadn’t actually read. By the time I noticed, two of the sessions had drifted onto overlapping work and my critical thinking had gone offline somewhere around session four.

I felt productive and produced nothing. Three concurrent sessions is the most I can run while keeping a real model of what the system is doing. It’s tempting to spin up as many agents as possible to maximize parallelism, but agents don’t have the best judgment (see practice 1).

When I run 3 sessions, one session handles the meaty work, like a backend flow where I need to follow the logic, or a feature with real architectural choices in it. The other two sessions handle small surface changes: tightening email copy, adjusting a screen’s layout, fixing a typo.

The principles behind the cap: your mind can only deeply focus on one thing at a time, and that focus is the most valuable resource you bring to the build. Allocate it accordingly. Keep parallel tasks far enough apart that the agents don’t step on each other’s commits. And lean on Linear instead of spinning up another session every time a new idea hits. Slowing down to define a ticket, track it, and execute against it is the discipline that compounds. Slow is fast, and fast is smooth.

3. Scope is a daily fight.

AI’s default mode is maximalism. Ask for a “share story” button and the agent will quietly draft a notifications framework, a sharing-permissions DSL, and analytics events for everything.

Subtraction is taste. Only add complexity if it's truly necessary. The best products are simple, elegant, and delightfully easy. This is something AI does not understand.

This compounds with practices 1 and 2. When you prototype, you catch scope creep before code gets written. When you run too many parallel sessions, you miss it entirely.

I learned this the hard way. While building the interview scheduling system, Claude sneaked in the idea of an expired interview. The system would mark an open question prompt to a parent as expired if no activity happened after 48 hours, then close the interview entirely. Sending your parents a question to answer often takes more than two days to get a response. There’s no way I’d have added that on purpose.

But I was running too many sessions and didn’t catch it, until it created a bug with a real user whose mom tried to answer a question four days after she received it. I spent a whole day picking through the slop of the question-prompting system to undo a feature that should never have been there.

4. Two flavors of taste. Both matter.

Engineering taste catches the unnecessary abstraction. Product taste catches the unmet user need. The agent will miss both.

Engineering taste first. Codex caught some of the engineering misses for me. A memory leak in the audio recorder. A Supabase RLS policy that let users read other families’ stories. Gstack’s /review skill is solid for the obvious stuff: race conditions, idempotency, RLS policies.

Take localization. Baohua translates most of the product experience into simplified and traditional Chinese. Claude’s first instinct was to scatter one-off localizations across the product copy. I had to walk it through centralizing the language system. One source of truth for English and Chinese today, with room for Korean and Japanese tomorrow.

A decent engineer would have caught this on day one. Claude needed to be told. That’s the gap between a vibe-coded prototype and a codebase that won’t bleed regressions six months in.

Product taste second. Claude Code knew how to implement magic links for auth, but did a poor job of integrating them into the user experience. The first pass shipped magic links to storytellers in Baohua with one-hour expirations and no other way to log in.

I spent the better part of three days walking through every auth permutation for every user type. Magic links, Google auth, email and password. The goal was a consistent flow with as little friction as possible. There’s no AI shortcut for this. You have to walk every flow as the person who’s actually going to use it.

5. Mobile coding is pure magic.

One of Baohua’s first users messaged me on Instagram about a recording bug. I was at dinner with friends. I opened Claude Code Remote on my phone, connected to a session running on my Mac mini at home, and shipped the patch before the entrees arrived. A year ago this would have sounded absurd. Today it’s how I work.

A few things worth knowing about mobile coding. There are real tools to make this work: tmux, Tailscale, Claude Code Remote, and a growing number of others. They’re worth the afternoon it takes to wire them up. The simplest setup is to leave your laptop open with a terminal and a remote-control session so you can drive it from your phone.

What I’ve landed on is a terminal session permanently open on my Mac mini at home, which is also always-on. It means I can investigate issues anytime.

I’ve shipped fixes from the gym, from the airport, and from my couch while watching Crash Landing on You. This gives me confidence to support my codebase without an on-call team.

6. Nothing beats talking to users.

The oldest piece of advice in the book is still the most important one.

I shipped Baohua on April 30. Then I talked to my friend, who told me “my mom won’t do this without LINE.” I spent the next three days doing hairy restructuring to add a second messaging channel.

Then I talked to my mom, who told me “no Chinese speaker is going to understand what Baohua is if you don’t clearly lay it out for them. Your emails are too short.” Counterintuitive advice for someone like me, who skims emails and clicks the CTA. So I rewrote the email design.

There’s no version of these practices that gets you to the right product without this one. Everything above is how you build well. Talking to users is how you discover what users love.

The next frontier isn’t code.

One parting thought, beyond the principles of building well. There's a harder puzzle waiting on the other side. Now that everyone has an army of engineers in their pocket, learning how to find and connect with your customers is the new edge.

These six practices get you to a solid product. They don’t get you to a business.

The frontier isn’t how sexy your Claude Code setup is. 3 months from now, everyone will have their custom coding rig. The setup will be table stakes.

The machinery to respond now costs $200 a month instead of $50,000 a month in engineering payroll. So what kind of business are you going to build?

On April 1, I opened a fresh Claude Code session and asked the gstack skill to interview me about Baohua, a bilingual story-sharing app for Chinese American families I’d had in my head since 2018.

A few weeks later it was in production, with a CI/CD pipeline, a full test suite, and the first families sharing stories. I built a real application with AI translation, complex routing across communication channels, and PDF generation for book printing.

I wanted something that I could trust not to break, and something that would scale. So, I skipped Replit and Lovable.

What got Baohua across the production bar was a set of dialed skills and disciplined execution. I heavily leveraged gstack from Garry Tan (…which was great except for the advertisements to apply for YC). And I drew from my own experience building software to design something simple, yet elegant.

If you’re a solo builder trying to do the same in 2026, this is the stack that I developed on and how it came to life.

The Solo Production Stack

The Solo Production Stack is the specific set of AI tools, infra services, and engineering practices that make it possible for one person to ship an app in just a few weeks.

It has five pieces.

1. A coding agent with the right skills harness

2. A coordination layer for tickets and milestones

3. An adversarial reviewer on every PR

4. An infra stack you own

5. A source-control flow with tests as gates

Five pieces, one builder

The Solo Production Stack is one super brain operating complex machinery. The brain is your guidance with the help of Claude Code and a skills harness.

The skills harness: Claude Code with gstack

Claude Code wrote almost every line of Baohua, and the skills harness running on top of it is why the output was quality.

Gstack ships with lots of skills that force depth of thought before the agent writes a single character. Here are a few of my favorites:

• /ceo-plan-review pressures the work against business goals

• /eng-plan-review pressures it against architecture

• /design-shotgun explores and iterates on ideas and designs

• /review, /qa, and /ship are the core of my workflow to produce high quality code

The harness is what turns Claude Code from “smart autocomplete” into something that knows what you’re trying to build and helps you build it with sound engineering principles.

Coordination machinery: Linear, bridged by Venn.

Every ticket, every milestone, every PR got tracked in Linear. I had three milestones for Baohua (MVP, Beta, Commercial Release) and roughly 80 tickets stacked behind them at peak.

I used Claude to create my plans in Linear. I would regularly have Claude check Linear for which tasks needed to get done next and groom my own backlog about once a week. It’s almost like a decade of PM discipline puts you on autopilot…

To connect to Linear, I used Venn AI. Linear’s official MCP was missing several tools I needed and they don’t ship a working CLI. Using Venn AI gave me agent-driven ticket creation, sprint shaping, and status updates with the auth and scoping I needed.

Review machinery: Codex on meaningful PRs

Codex catches things Claude misses. On a typical PR for Baohua, Codex flagged two to three critical issues that would have shipped otherwise: a memory leak in a long-running audio recorder, a Supabase RLS policy that let users read other families’ stories, edge cases in the Mandarin input handler.

As someone who spends more time on product than on deep systems engineering, this is the layer that makes me comfortable shipping. It comes out of the box with gstack’s /review command, which runs Codex against a draft PR and posts findings in a format that’s easy to action.

Without this second model on important change, I wouldn’t trust the brain’s output enough to merge it.

Infra machinery: Vercel, Supabase, Inngest, manually wired

Claude Code wired the entire infra stack itself: Vercel for hosting, Supabase for auth and Postgres, Inngest for background jobs (the audio processing queue, weekly digests, retry logic).

It took maybe a day longer than going with Replit. But the honest benefit? I own every layer, I can move providers if I need to, and there is no platform between me and the families using Baohua.

The day I spent wiring Vercel and Supabase has already paid back several times over in flexibility and control.

Source-control machinery: GitHub with testing

Every change ran through a GitHub PR with branch protection and required status checks. The test suite gates merge: roughly one in three of my PRs trips a test on first push and gets rewritten before it lands. That ratio is the reason this stack works. I am not a careful enough engineer to merge to main directly without breaking something, and the brain is even less careful than I am.

Tests do the work a responsible engineer would do. When I’m building a new feature, I can trust that I’m not introducing regressions.

Taste: the layer the diagram can’t show

The architecture above is the visible part of the Solo Production Stack. There is a more important layer the diagram can’t show, and it’s the most critical part.

For Baohua, taste meant being clear that I was building for Chinese American families with grandparents who don’t natively speak English and children who don’t natively speak Mandarin.

It started with a personal problem that I validated with dozens of friends and family first. It continued with copious research about Storyworth, Remento and other alternative storytelling platforms.

Before I wrote the first piece of code, I spent hours with /ceo-plan-review refining my idea and its purpose. I also spent hours tinkering and optimizing a visual language that doesn’t feel like another cheap SaaS knockoff, but something that emotionally resonates with my customer.

Claude Code can’t tell you what to build. Codex can’t tell you whether what you built is worth using. Skills can hold conventions but not opinions. The taste layer is yours

Why Replit and Lovable don’t get you there

Both platforms got dramatically better in 2025. Replit’s Agent 3 ships with a multi-agent loop, Replit Auth, native Postgres, Stripe, and a catalog of more than thirty native connectors. Lovable’s Supabase integration handles auth, database, storage, and edge functions through chat, with two-way GitHub sync and preview deployments per branch.

For a prototype that needs to look like a product in a weekend, both are remarkable, and I considered both.

The bottom four rows are what separate a weekend prototype from a production deployment. The path with either managed platform is the same: when you actually need engineering discipline, you export to GitHub and Vercel and reassemble the rest of the stack yourself. Once you’ve done that, you’re already running the Solo Production Stack. The platforms gave you the head start, not the finish.

The gap between a prototype and a full product is the skills harness that lets you build methodically with discipline and rigor.

This harness brings out your product intuition and taste to build something that solves a real problem, creating something people truly want.

This is where Replit and Lovable came up short.

Engineering taste matters too

The Solo Production Stack does not work without engineering instinct. You need to recognize bloat to prune it. You need to read a Codex review and know which warnings actually matter and which are noise. You need to look at a Claude Code PR and notice when it has quietly rebuilt something you already had. A non-engineer running this loop will ship a Replit-grade app and call it production.

If you don’t have the instinct yet, don’t worry. Build slowly, ask Claude lots of questions. Stay patient and you’ll learn everything you need.

Baohua released

Baohua shipped on April 30, on my birthday. I’ve wanted to build Baohua for years, and shipping it has been one of the most satisfying moments of my life.

The fact that one person, in one month, can ship a real product to families they care about is the early signal of something bigger.

Some of the most useful software in the next decade will be micro-applications built by individuals, solving specific problems that never made it past a VC partner meeting because the TAM was too small.

That future matters more to me than the next frontier model release. If you’ve been carrying an idea like this for years, I’d love to connect.

As November approaches, many of us are going through annual planning — what are our major initiatives and how are we positioned to execute on them? Among this lies the need to accelerate our progress in data and AI, especially with breakneck development and the possibility of AGI coming within just another year! (See posts from Sam Altman and Dario Amodei).

Let’s pause first and define ‘AGI’ that could come soon. There are lots of definitions floating around, but this is one of my favorites from Dario Amodei. He breaks down what he’s renamed as ‘powerful AI’ with the following characteristics:

• Smarter than a Nobel Prize winner in several fields

• Receives and executes tasks in multiple ways (audio, video, mouse, chat, etc) and performs them with a skill exceeding most people

• It can perform complex tasks over the course of days and weeks

In short, Dario calls this “a country of geniuses in a datacenter”

With meaningful advancements coming on a scale of thousands of days, it’s imperative for every company to develop a meaningful data and AI strategy of their own. Now is the time to brainstorm, socialize, and staff a multi-year data & AI strategy.

As your teams begin building your strategies, you’ll want to cover basics like understanding the technology, preparing your data and setting up governance (as covered by guides like this). Beyond the table stakes, I’ve been thinking through some additional items to incorporate as your teams build your strategies:

1. Collect and organize more data, both internal and external

2. Make proper platform investments in data and AI technologies

3. Rapid experimentation with AI to build your expertise

Expand your data vault

AI models and LLMs are effectively platforms that perform complex tasks based on the data they’re given. As such, the single biggest differentiator your company can have in the AI era is the proprietary data within your walls. What information about your business, your customers, and the market is unique to you? Identifying that data and growing its base is the most important cornerstone of your long-term AI strategy.

External data - your customers

Your external data, what you know about your customers (first party) and the market (second and third party) are the main building blocks for delivering a superior and more personalized experience for your customers.

If your team hasn’t yet built a robust CDP (customer data platform) with all the most important facets about your customers, this must be one of your top priorities. Not only will this help your company run its business better today via analytics and personalization, it will be critical context that your AI models will need in order to improve your user experience. Here, focus on your fundamentals like having a modern data warehouse in place, building your core tables, and identifying your top metrics and dimensions.

If your team has a CDP in place, now is the time to expand your footprint. What can you build to collect more data about your customers? Can you create a growth loop that incentivizes your users to continue generating more data?

For example, let’s say you’re a fintech company and you want to know more about your customers. What can you build to collect information about their financial ambitions and how they think about saving? Are there innovative solutions beyond a “goal setting feature” that can hook your users to start cataloging their desires? This is the type of data that will provide invaluable as your company enters the AI era.

Internal data - your company

Maintaining your internal data in the vault is equally as important; this is the data that will enable you to run a more efficient business. This is also some of the data that is often overlooked in favor of developing CDPs. Make sure your company is collecting, storing, and organizing all the critical information on how you run things; go through each department… sales, marketing, product/engineering/design, operations, risk, and more.

Look for pockets of data (sales calls, confluence docs, trainings, etc) that can be utilized to train and improve AI models in ways that drive operational efficiencies.

For example, the data from all your sales calls tagged with which ones of them exemplify good outcomes versus bad outcomes are valuable data source for your AI models to help train your sales reps.

In the next year, it will be critical for your business to have a good map of where your internal data lives, how to access it, and how your team can feed it into AI models for any new projects.

Invest in your data & AI platform

As we enter 2025, let’s hope that most of us have already migrated to the current generation of data technologies, using cloud data warehouses (Snowflake, BigQuery, etc) , orchestration frameworks (airflow, dbt), ETL/ELT tools (Fivetran, Hightouch, HEVO, etc), and quality frameworks (Monte Carlo, great expectations, etc).

LLMs and the newest AI models work differently and we’re already seeing an explosion of new technologies and frameworks to help your teams make the most of them. As such, your 2025 strategy needs to take some of these new technologies into consideration — how to interface with RAGs, vectorDBs, knowledge graphs, and more. These critical platform pieces that will help you store, retrieve and update data that feeds your AI models and AI-powered applications. Without them, your team will be stuck in the “whitelabeling chatGPT” version of AI products.

Consider where and how you’re investing in AI technologies like RAGs, vectorDBs and knowledge graphs

Amir Behbehani has a fantastic breakdown of the the evolving AI stack, which is a must read for any technologist looking to understand how they’ll build their AI applications in the future.

Product experiments with AI

AI is a nascent field, similar to what mobile was in 2008. There will be an inevitable explosion of new design paradigms, use cases, and products that come from this next wave of innovation.

For now, the single best thing we can do is test, iterate, and build our expertise around AI. This is not the time to hang your business on a single AI bet. It’s the time to identify your top product managers and engineers and give them space to explore what can be done with AI. Teams that experiment now will build lasting expertise and gain a long-term advantage with AI.

As you construct your strategies, consider the following:

Calibrate your risk appetite

We’ve all had experimental initiatives laden with pressure and expectations from across the business that crumble with all the attention and stall with too many cooks in the kitchen. Do not let this happen to your AI initiatives!!

By now, most people have heard of the raging success from Google’s Notebook LM. To me, the most important line in their entire announcement is this one:

NotebookLM is an experimental product, built by a small team in Google Labs.

Meaningful discoveries are made from experiments, with small teams, and quarantined away from risk-averse business agendas. As you assemble teams to take bets in AI, make sure they have the space to try lots of different things. Hold them responsible by asking about their learnings and how they’re iterating. And make sure that senior leadership is on the same page about taking these risks!

Experiment, iterate, and learn

In the same spirit of experimentation, the treasure to be discovered in the AI gold rush comes from iterative learning. Each experiment gives you valuable lessons about how the technology works (or doesn’t work), what your customers care about, and how to make adjustments going forward.

For example, let’s say you’re building an LLM to train your sales team better and you’ve thrown every recorded demo from the last 2 years at your model. After your first iteration, you realize that your sales team is getting worse! Upon deeper digging, your team recognizes that you need to label your training data with good demos versus bad demos. You also need to fine tune the model to help teach sales tactics, not share product information, since your product stats have changed in the last 2 years. Getting to this insight quickly helps your team uncover nuances in how to deploy AI and generate value.

Teams that are able to close the feedback loop quickly will learn many times faster than others. This is the time to move fast and train things.

Wrapping up

Now is the time to really get deep about what you need to build in 2025 to be ready for accelerations in AI development. It’s the time to go a level deeper than your competitors to think through the nuances and build them into your multi-year strategic plan. Start investing now in your data and AI strategy. Every step you take today compounds tomorrow when AI takes off. Let’s build together.

Anything catch your eye or just want to chat some more? Please reach out!

When should I consider “data in my product”?

Almost every product is a data product because it retrieves and shows data to the end user. So, when do we start thinking of a product as one that incorporates data? Products that more deeply incorporate data are ones that need to use aggregated or computationally complex data as part of the product delivery, either as a value directly shown to the user or to help decide what to show to the user.

Easy examples of this are products that need to call an ML model’s output (recommenders, ranking algorithms, decisioning systems, complex pricing systems, fraud detection, etc). Other examples include needing to join and aggregate data from multiple sources (ex: show the user’s total spend across multiple products or sources) or perform complex calculations (ex: calculate user percentile across a large audience) in order to meet the end user’s needs .

In these cases, while the product can likely deliver some level of value with its current setup, leveraging a robust data infrastructure and service can greatly improve the product’s quality.

Products that more deeply incorporate data are ones that need to use aggregated or computationally complex data as part of the product delivery, either as a value directly shown to the user or to help decide what to show to the user.

What should I consider?

Adding data to a product’s core experience introduces several new variables to consider. In addition to thinking through the product’s backend microservices and interactions with its production database(s), teams will also need to think through what generates data that wouldn’t come from the production database, the analytical data platform, and upstream data sources.

We know that the technology behind products can get quite complex. For the purposes of this post, we will keep things simple with just a set of microservices and a single production database powering the product.

Subscribe now

Taking data into account requires thinking about 3 additional components:

1. A data “service”: this effectively a microservice that sits on top of an analytical data platform. It ensures that your product can access the data it needs with high availability and low latency (note: this is not always a given, especially with data warehouses) and in the proper formats.
   
   A data service can be called just like a microservice to provide real-time support directly to the product and end user. In some cases, however, the service can also batch compute and write data to the production database instead. This second method is useful for computationally expensive queries that generate values that can be broadly used across the product (ex: infer the user’s next product preference once an hour and store the value in your DB, then call it across one or many microservices for a variety of use cases).
   

2. Analytical data platform: this is a larger data infrastructure that can unlock all sorts of data value if built and leveraged correctly. This set of systems is where most of the heavy data computing occurs, whether that’s stream processing, OLAP needs, or your machine learning operations.
   
   Well built pipelines will process data across the analytical data platform to create data assets that can be used for product features. For example, you may create and maintain a custom audience of users with certain preferences based on their historic purchases. Aggregating data across millions of users and their purchase history is a great use case of a data warehouse; connecting the output of that calculation to an audience to target for personalized recommendations is a powerful way to connect your data infrastructure to the end user experience.
   
   Teams are investing in their analytics data platforms across use cases such as data ingestion, data contracts/quality management, reverse ETL, and others. Well architected data systems with governed, reliable, and useful datasets can be leveraged beyond BI/analytics and activated to improve your product’s impact.
   

   Well architected data systems with governed, reliable, and useful datasets can be leveraged beyond BI/analytics and activated to improve your product’s impact.

3. Data sources: when building data into a product, it’s imperative to understand the data’s lineage all the way back to its source. Does your data come from other products’ databases and then get joined in the analytical data platform? Does it come from a 3rd party vendor? How reliably does the data get ingested and what failsafes need to get put in place if there’s an issue with the data from its source? Be sure to only build product features off datasets with reliable sources and set contingency plans that don’t disrupt the user’s experience should one of your upstream data sources run into issues.

There are many product and engineering decisions that will affect your end users based on the state of your data services and infrastructure. With a map and lineage of data all the way back to source, the best way to manage these considerations is to run several use cases through the data systems. Your use cases will help you understand things like where you need real-time data, where it’s okay to leverage stale(ish) data (hours, not days), where you should batch versus stream data, the impact (and mitigating actions) of data that’s unavailable or poor quality, and more.

What other nuances come up over time?

As your product evolves over time, so do your data systems that support it. Basic monitoring, alerting and data quality checks will help ensure that your team identifies and resolves incidents in a timely manner, but there are also a few data-specific nuances to keep in mind.

• Managing Bias: data, when used without human intelligence, can often return biased results. Without frequent analyses on the data outputs that are used by your product, you may be unintentionally using data in ways that harm or discriminate against your users.

  

• Logic changes along the data pipeline: it’s very likely that business logic is applied somewhere along your data pipeline and that logic will change over time. An example might be event taxonomies that change classification over time. If historical data matters (showing/leveraging trends, model training, etc), then logic changes may also trigger the need for restating data through the past so that your end users don’t have a sudden change in their experience.

Building data into your product is a powerful way to drive more value for your users. However, it also introduces the need to take data services, infrastructure and lineage, and other long-term nuances into consideration so that your product reliably delivers the promised value to your users. Taking these extra dynamics into consideration will help you build more robust and impactful products while also capitalizing on your organization’s data investments.