Subscribe • Previous Issues

Your AI Passed the Test. Did It Do the Job?

At the recent AI Conference, Ion Stoica described a coding agent that made a key-value store, a simple database, roughly six times faster. It passed the tests, but there was a catch: the agent had stopped storing data. Instead of saving information and retrieving it later, the agent noticed that the test data followed a predictable pattern. Whenever the benchmark requested a value, it simply calculated the expected answer. By any reasonable measure of what a database should do, it was a failure.

What interests me about this example is that it points to a problem well beyond coding agents. Before we can test whether a system does what we want, we have to translate our intentions into explicit requirements and decide how to test them. Important assumptions can get lost in that process, and the tests themselves may not reflect real-world conditions. A passing evaluation tells us that a system met the test, not necessarily that the test captured what we actually meant.

Two Gaps Behind a Passing Test

Stoica and his colleagues describe two gaps in software development. The requirement gap is the difference between what people want and what they write down. The model gap is the difference between the conditions assumed during testing and those encountered in production. Here, model refers to a representation of the operating environment, not the AI model.

The database example illustrates both gaps. The requirements never explicitly said the system had to store whatever values users supplied. That’s the requirement gap. The benchmark also relied on predictable values, even though real users might store data that cannot be reconstructed. That’s the model gap. Fixing one problem would not necessarily have fixed the other.

These are old engineering problems, but AI agents make them harder to ignore. They can try solutions quickly and repeatedly optimize against the same tests, discovering shortcuts that might never occur to their human designers. They also lack much of the business context that experienced employees take for granted. A second agent reviewing the same incomplete requirements may miss the same thing.

Mathematics Has Its Own Version

I see a useful parallel in mathematics, where verification is much more rigorous than ordinary software testing. Lean is a system that checks mathematical proofs against precise rules of logic. Recent advances in AI have made it easier to translate human-written mathematical arguments into Lean, a process called autoformalization. The AI can then revise the translated proof until Lean accepts it.

But what exactly has been checked? Researchers recently compared the Lean version of a widely discussed OpenAI proof involving the Navier-Stokes equations, which describe fluid motion, with the written argument. They found important differences. In one case, the Lean version required a stronger assumption and therefore established a weaker result. In another, the mathematical reasoning differed. The researchers did not conclude that the original proof was wrong, only that the Lean code did not verify the written argument as presented.

There’s an important distinction here. If a trustworthy checker like Lean proves the intended mathematical statement using a different argument, that can still establish the statement’s truth. But it doesn’t establish that the original written proof was correct. Someone still needs to confirm that the statement being checked captures what was intended.

This translation step is now common across many AI applications. Companies are using AI to turn policies into instructions for agents, medical guidelines into recommendations for clinicians, and business requirements into automated decisions. An agent may follow those instructions correctly even if important assumptions or exceptions were lost in translation. Adding more AI reviewers may help catch mistakes, but they can also share the same misunderstandings.

Test Engineers Learned This Long Ago

Traditional test engineers have been dealing with versions of these gaps for decades. Their work begins with deciding what a product should withstand and then recreating the conditions it will actually face. A medical device intended for patients cannot be adequately tested only by trained nurses. The model gap shows up with AI agents, too. Testing them on carefully written requests may not tell us how they handle vague instructions, missing information, and unexpected situations in practice.

Safety engineering offers another lesson. Warning people not to do something is weaker than preventing them from doing it. The same applies to agents. Telling an agent not to access a sensitive system is not the same as removing its credentials or blocking network access. This is why I favor enforcing important boundaries in software rather than relying on instructions in a prompt.

The broader lesson is to design systems around how people and agents actually behave, not just how we expect them to behave. Testing needs to account for both the requirements we write down and the conditions the system will encounter.

What I Would Change

• Ask what passed, and under what conditions. A single pass/fail score can hide a lot. This is one reason I’ve advocated for high-dimensional evaluations, which examine specific risks and failure modes rather than relying on one overall score. Even then, ask which assumptions and real-world conditions those evaluations might still miss.

• Review the translation from intent to rules. When AI turns instructions into software or rules for agents, require it to surface added assumptions, changed requirements, and unresolved ambiguities. Spend expert time confirming that the resulting rules capture what was intended.

• Use more realistic evaluations. Combine cheap automated tests with representative workloads, unexpected inputs, and limited deployments. Keep some evaluation cases out of the agent’s development loop so it cannot tailor every solution to the grader.

• Put hard limits outside the model. Use permissions, isolated environments, approval thresholds, and the ability to reverse consequential actions. A prompt telling an agent to be careful is not a security boundary.

• Make failure improve the process. Treat real-world failures as feedback on both the requirements and the tests. Determine whether the implementation was defective, the requirements were incomplete, or the tests assumed unrealistic conditions. Fix the underlying problem rather than just patching the immediate bug.

• Measure the whole cost of confidence. Faster generation is not necessarily better if it creates more review work, testing costs, or production incidents. Measure the full workflow, not just how quickly an agent produces an answer.

The Bottleneck Is Moving

As AI makes it cheaper to produce code, mathematical proofs, and other outputs, the harder work increasingly lies in defining success and establishing confidence in the results. What we’re seeing in coding and mathematics is an early glimpse of what’s coming for other kinds of knowledge work. I expect people to spend less time producing routine work and more time translating human intentions into instructions machines can follow, checking what might have been lost in translation, and evaluating whether the results make sense.

This doesn’t mean humans need to review every step. But we do need to figure out where human judgment matters most, particularly when requirements are ambiguous or mistakes have serious consequences. I would ask every team deploying agents one question: verified against what? If the answer only describes the test, without explaining how it reflects what people intended and what happens in the real world, we still have work to do.

The Looming Debate Over Frontier AI Labs

AI and the Vanishing Apprentice

Ben Lorica edits Ethics.dev and the Gradient Flow newsletter, and he hosts the Data Exchange podcast. He helps organize the AI Conference and the Agent Conference. You can follow him on Linkedin, X, Mastodon, Reddit, Bluesky, YouTube, or TikTok. This newsletter is produced by Gradient Flow.

Subscribe • Previous Issues

What If AI Builds the Robot Instead of Controlling It?

Last year I wrote about robotics moving from bespoke machines toward reusable foundation models, essentially an attempt to build a general-purpose robot brain. More recently, I argued that models are getting easier to buy, while the harder work lies in making data useful. A newer development complicates that story. What if we can automate more of the work required to engineer the robot itself?

The Engineering Bottleneck in Robotics

A large class of robotics problems sits between rigid factory automation and the general-purpose robots we see in demos. The robot and workspace may stay mostly the same, but packages arrive in different positions, groceries come in different shapes, or cables bend differently every time. Roboticists sometimes call this variational automation. These jobs are common in warehouses, factories, and food-processing facilities, but they remain expensive to automate.

There are several bottlenecks. Robots lack the vast stores of physical experience available to language models, and each new application still requires engineers to assemble perception, grasping, motion planning, and recovery systems. Simulation reduces physical testing, but building one that captures the properties that determine success is itself labor intensive. The commercial bar is also high. A robot has to work quickly and reliably, not just succeed in a carefully selected demo.

Four Ways to Build a Robot

The traditional approach combines well-understood components for perception, motion planning, and control. These systems can be fast and dependable, but they require substantial engineering for each application. More recently, Vision-Language-Action models, or VLAs, have tried to replace some of this work with learning. Give the model images and an instruction, and it produces actions. The promise is broader generalization, but several VLAs struggled when object positions or task instructions changed.

Simulation offers another path. A robot can practice thousands of times without tying up hardware or requiring someone to reset the scene after each failure. The hard part is making the virtual environment predictive of the real one. Small errors in friction, weight, stiffness, or contact behavior can make a policy that succeeds virtually fail on hardware. This is the familiar sim-to-real problem.

World models are increasingly being positioned as part of the answer. Some learn from large collections of video, then provide a foundation for robot policies. Others reconstruct real environments, generate additional experience, or test what might happen after an action. The most interesting systems are already hybrids, combining learned representations with simulation, conventional software, and feedback from real failures. The line between a world model and a traditional simulator is becoming less clear.

What If the AI Builds the Robot Program?

This is why a recent post from Ken Goldberg caught my attention. His group is exploring what it calls Agentic Robotics. Instead of asking one large neural network to control the robot directly, coding agents assemble a program from reusable robotics skills. Their Graph-as-Policy system represents the program as connected modules for tasks like perception, grasping, and motion planning. The agents rehearse it in simulation, find where it fails, and modify the relevant piece rather than retraining the whole system.

The more surprising step is automating the simulator too. Goldberg describes giving a frontier model a short video of a robot wiping ink off a metal bar with a sponge, along with access to a physics engine. The model built a simulation, then agents used it to improve the robot program before running the code on the real robot. Not every trial worked. In one, the robot pushed over the bar because its mass could not be reliably inferred from the video. That failure shows why testing on real hardware still matters. This reminded me of a recent conversation about AI-assisted chip design. Hardware engineers are using foundation models to write and optimize kernels, the specialized programs that make models run efficiently on particular hardware. In both cases, AI is helping experts with engineering work that has traditionally been detailed, specialized, and time consuming.

There are reasons to take this seriously, but also reasons not to get ahead of the evidence. In one benchmark, the agent-generated system came close to a graph built by robotics experts. In another example, Ambi Robotics reports using agents to analyze 500 production events and develop three candidate fixes in about ten hours. The company then tested all three on production robots in customer warehouses for ten days. Two reduced throughput. The third increased it by 4.2 packages per hour, and Ambi says it deployed that fix across 30% of its U.S. fleet. These are still bounded industrial tasks, and open-ended manipulation and humanoids are much harder. The interesting result is not that the agent is always right. It is that generating, testing, and improving highly specific engineering solutions may become much cheaper.

World Models May Have to Earn Their Place

What changes my view is how far Agentic Robotics can get without a learned world model. A general-purpose foundation model interprets the task, helps build a task-specific simulation, and writes the robot program, while a conventional physics engine handles the dynamics. For bounded industrial tasks, that suggests learned world models are not always necessary. Companies developing them need to show where a learned model adds something that a foundation model, existing robotics software, and a physics engine cannot provide as well.

I would not count world models out. They might reconstruct an environment, fill in physical behavior that is hard to infer from a short video, or generate variations for stress testing. A VLA might still be the best way to handle a manipulation skill that is hard to program. The bigger shift is that none of these models necessarily has to become the robot brain. The general intelligence can sit in an engineering agent that chooses the models and tools needed for a particular job.

This also changes where I would look for durable advantage. Individual models can be replaced as better ones arrive. Calibrated simulations, reusable skills, evaluation environments, failure histories, and production data are harder to reproduce. This should look familiar to enterprise AI teams. The foundation model gets easier to buy, while the surrounding system becomes the asset. For robotics teams, the practical question may shift from “How do we train the best robot brain?” to “How quickly can we assemble a system that turns observations of the real world into behavior we can trust?”

Book Recommendations

• Broken China: How the Economic Miracle Shattered and What it Means for the World. Most media coverage of China focuses on its export and manufacturing prowess. Yet when I talk with visitors and friends just back from the country, what they most want to discuss is the bleak job market, especially for young people. For anyone who wants to understand China’s economy, this book is essential reading. Drawing on decades of on-the-ground analysis, it is rigorous in both its arguments and its evidence.

• Kobuk the Destroyer: And Other Tales from the Wild, Unseen World of Test Engineering. I enjoyed this entertaining look at the often overlooked world of test engineering. As AI applications and agents become more capable and complex, I think AI teams have a lot to learn from decades of experience testing how systems fail, behave under real-world conditions, and handle unexpected situations.

• Money to Burn: The Unvarnished Truth About Leon Black, Apollo, and the Rise of a New Wall Street. This book gave me a much better understanding of private equity through the story of Apollo, a firm born out of Drexel Burnham Lambert’s collapse. I found Apollo’s evolution toward private credit especially interesting, particularly its ties to insurers and annuities, which are becoming increasingly important to financing the AI buildout.

• The Everywhere Millionaire: Who Is Really Rich in America and How They Got There. If you want to better understand America’s small and midsize businesses, this book is a goldmine. I learned a lot about how the owners of seemingly ordinary businesses build wealth and wield economic influence. Written by two academic economists, it’s also a surprisingly fun and engaging read.

The $2.1 Trillion Compute Landlord

Ben Lorica edits Ethics.dev and the Gradient Flow newsletter, and he hosts the Data Exchange podcast. He helps organize the AI Conference and the Agent Conference. You can follow him on Linkedin, X, Mastodon, Reddit, Bluesky, YouTube, or TikTok. This newsletter is produced by Gradient Flow.

Subscribe • Previous Issues

The Model Factory Matters More Than the Model

A couple of months ago, I wrote about the growing post-training stack that lets teams turn general-purpose models into something better suited to their own work. I recently spoke with Joel Hron, CTO of Thomson Reuters, about a more ambitious version of the same idea. His team starts with a strong open-weight model, continues pre-training it on Thomson Reuters data, then puts it through several stages of post-training. I am going to call this deep model customization. Thomson Reuters is not alone. Cursor, C3 AI, ByteDance, Cisco, and Reddit are exploring variations of the same approach, and their experiences already point to a few useful lessons.

Why Fine-Tuning Is Not Always Enough

The companies going down this path generally tried easier options first. Thomson Reuters started with commercial models and relatively simple LoRA fine-tuning. The models improved on the tasks being targeted, but other capabilities deteriorated. C3 AI ran into a different problem. Its proprietary Type System does not appear in public training data, so retrieval alone produced an 11 percent pass rate on its internal coding evaluation. Adding agents and tools pushed frontier models to around 60 percent, but responses took roughly two minutes. The information was available, but the models had not really learned how C3’s system worked.

The lesson is not that RAG or fine-tuning are ineffective. They solve different problems. Retrieval is useful when information changes or is too large to put into model weights. Post-training can reshape behavior and improve specific tasks.

Continued pre-training goes deeper by changing what the model knows about a domain. That matters when a company’s advantage comes from knowledge, language, workflows, or judgment that is poorly represented in public training data. ByteDance wanted Valley3 to understand multimodal commerce. Cisco wanted a small model that understood cybersecurity. Reddit wanted a model that understood community behavior and moderation context.

The Checkpoint Is Just the Starting Point

The common recipe begins with model selection. These teams are generally not pre-training a general model from scratch. They evaluate open models against their own workloads and pick the best starting point. Thomson Reuters selected Qwen using its internal evaluations. Cursor chose Kimi K2.5 after testing candidate models on coding knowledge and its own code. C3 AI has repeated its process across five generations as stronger open models have appeared.

Next comes continued pre-training. The company mixes domain data into the model while usually retaining general data so specialization does not wipe out useful capabilities. Thomson Reuters trained on a 200 billion token mixture drawn from a much larger collection of legal, regulatory, news, and other material. ByteDance added an e-commerce training phase to Valley3. Cisco continued training Llama 3.1 8B on billions of tokens of cybersecurity text. This is where data engineering matters. Teams have to decide what goes into the mixture, what gets removed, and whether more data is actually improving the model.

Then comes post-training. Teams use supervised examples, preference data, reinforcement learning, or some combination. What stood out to me is how often the training environment resembles the actual product. Cursor trains Composer inside coding environments with tools similar to those used in Cursor. Thomson trains its model to use Thomson Reuters search and research systems. Valley3 learns to operate a multimodal product-search engine. The goal is not only to teach the model what to know. It is also to teach it how work gets done inside the company.

These Models Do Not Have to Replace Everything

One of the key lessons from Thomson Reuters is that success does not require replacing every commercial model. Hron told me their frontier model (Thomson) currently handles about 5 to 10 percent of inference inside CoCounsel. Its main production job is high-volume document analysis, such as reviewing large collections of contracts during M&A due diligence. CoCounsel still uses Claude, OpenAI, Gemini, and smaller models. Thomson gets the work where its legal specialization, control, and economics make it a better fit.

Other deployments follow a similar pattern. C3’s Narwhal handles C3-specific coding, documentation, customer support, and agent tooling while the broader platform remains model-agnostic. Cursor offers Composer alongside third-party models. Cisco is putting its security model into XDR and other security workflows. Reddit reports that its Nova-based moderation work has reduced missed threats while consolidating work previously handled by multiple models. A specialized model does not need to handle every request to be worthwhile. It can be valuable simply by owning the high-volume or highly differentiated parts of the workload.

The Hard Part Is Everything Around the Training Run

Thomson Reuters gives a clear sense of what this takes. Its core effort involved roughly 30 scientists and engineers, seeded partly through the acquisition of Safe Sign. It also had something most companies cannot quickly recreate: more than 3,000 legal and tax experts, decades of curated metadata, several trillion tokens of content, mature retrieval products, and evaluations that could detect whether the model was improving in law while getting worse elsewhere. The final training run cost less than $450,000 in compute, but Hron estimates the broader two-year effort at roughly $40 million including people and infrastructure.

The other examples vary widely. C3 credits a small data-science group and already had proprietary code, documentation, production feedback, and internal evaluations. Cursor built much more infrastructure itself, including distributed training, large-scale RL, and sandboxed coding environments. Reddit took the opposite route and used Amazon Nova Forge, which provides model checkpoints and managed training infrastructure. This connects to a point I have made before about data. A pile of proprietary documents is not much of an advantage by itself. The useful asset is the machinery that turns data, expert judgment, production failures, and evaluations into better models.

The Model Factory May Matter More Than the Model

The most interesting part of these projects is that several teams already treat the underlying model checkpoint as replaceable. Thomson Reuters expects to start from newer open models as they appear. C3 AI has already repeated its process across multiple bases. Cursor reused its training machinery for Composer 2.5. The durable asset is increasingly a model factory: the data pipelines, evaluations, training recipes, RL environments, serving infrastructure, and people who know how to run the loop again.

The most interesting part of these projects is that several teams already treat the underlying model checkpoint as replaceable. Thomson Reuters expects to start from newer open models as they appear. C3 AI has already repeated its process across multiple bases. Cursor reused its training machinery for Composer 2.5. I recently saw the next step in this progression in a conversation with Ameet Talwalkar of Datadog. Datadog went further with Toto and pre-trained its own time-series foundation model from scratch, using the huge volume of observability data it already had. Taken together, these examples suggest that the durable asset is increasingly a model factory: the data pipelines, evaluations, training recipes, RL environments, serving infrastructure, and people who know how to run the loop again.

I’m not suggesting that every large company should start training its own model. Most teams should exhaust prompting, retrieval, better harnesses, and conventional post-training first. Deep model customization is still expensive and operationally demanding. But the threshold is moving. As open-weight models improve and AI infrastructure gets easier to rent or buy, more companies with distinctive data and high-volume workloads will be able to go further. I suspect the interesting question will shift from whether enterprises can customize models this deeply to which parts of their businesses are valuable enough to justify it.

What If AI Didn’t Have to Think in Words?

AI and Jobs: The Rising Tide

Ben Lorica edits Ethics.dev and the Gradient Flow newsletter, and he hosts the Data Exchange podcast. He helps organize the AI Conference and the Agent Conference. You can follow him on Linkedin, X, Mastodon, Reddit, Bluesky, YouTube, or TikTok. This newsletter is produced by Gradient Flow.

Subscribe • Previous Issues

Local AI Is a Routing Problem, Not a Privacy Feature

I’ve had a longstanding interest in local AI for a simple reason. I would rather not send proprietary code, unpublished plans, or knowledge about how a business operates to an outside model provider when I have a viable alternative. I want more control over where that information is processed.

A company does not need a model on every employee’s laptop to achieve that. A properly secured internal server can keep information away from outside providers. I admit there’s a bit of wishcasting here. But if my friends are any guide, I’m far from the only one who wants a capable model living on my own laptop.

What has changed is that more tasks can now run on individual devices. As local models improve, I’ve started thinking more seriously about what belongs on a laptop or phone, what needs an internal server, and what still calls for an outside provider. Privacy drew me to local AI. I think the more consequential question is where each task belongs.

The Device Becomes a Place to Run AI

Local AI is already appearing in consumer products. Supported Pixel phones can summarize recordings on the device, while Windows can generate live captions without an internet connection. People use these features without downloading a model or running an inference server. That is part of what makes them useful. Developers can build similar features with open-source projects such as Moonshine Voice, which supports on-device transcription and voice interaction.

I expect more business applications to follow. Tools for running local models can select suitable hardware, download models, and provide interfaces similar to hosted AI services. Developers no longer have to assemble every component themselves.

The hardware is improving too, but I would be careful about generalizing. Microsoft’s Project Zenith targets developer-class PCs with at least 64 GB of memory and 250 GB/s of memory bandwidth. That tells us something about where powerful laptops are heading. It does not mean the typical company laptop can run the same workloads.

There is also a cost to using computers people already own. Models compete with employees’ applications for memory and processing power. Laptops sleep and disconnect. A fleet is worth considering as AI capacity, but it is not a free or continuously available computing cluster.

Which Work Goes Where?

I would start with well-defined tasks whose results are easy to check. Transcription, document classification, local search, and focused coding assistance are candidates for laptops. Shared business data or demanding, always-available applications may call for an internal server. Outside providers remain an option when information is approved for external processing.

An evaluation of local coding models found that narrow assignments with specific instructions worked better than extensive code exploration. Long conversations were less reliable. Phones have tighter memory and heat constraints, making short, specialized tasks a more practical starting point.

The economics are less obvious than token prices suggest. In one study of coding agents, prompt caching made cloud inference cheaper per token, yet shared on-premises infrastructure had roughly 40% lower modeled total cost. Dedicated hardware cost more than the cached cloud API, and the local configuration required substantially more repair work. The study involved one developer and no employee laptops.

I would measure cost per correctly completed task, including failures, rework, response time, and the effect on the employee’s computer. Owning the hardware does not automatically make inference cheaper.

Responsibility Moves With the Work

Moving inference onto employee devices changes where companies manage safety and security. Models and operating systems provide protections, but those protections can differ depending on where a request runs.

Apple lets developers configure guardrails for its on-device foundation models, while its Private Cloud Compute models have policies developers cannot configure directly. Apple also warns that built-in safeguards can miss harms specific to an application. Where inference runs affects both the available protections and the developer’s control.

Model updates can change responses, and platform safeguards can evolve. Developers need to repeat their evaluations when the underlying system changes. Companies must also decide which models employees may install, keep approved versions current, and withdraw problematic releases. Local inference becomes an endpoint-management issue when employees download models on their own.

My minimum requirement is straightforward. A company should know which model handled a request, where it ran, what data and tools it could access, and what happened if it failed. An app shouldn’t send confidential information to an outside provider just because the local model can’t finish a task.

People running agents on personal computers face similar risks. An OpenClaw agent deleted emails despite instructions to seek approval. I personally would not give an agent unrestricted access to personal files, let alone financial accounts! Sensitive accounts should remain off-limits, and difficult-to-reverse actions should require approval enforced by the software, not merely requested in a prompt.

Where AI Runs Becomes a Design Choice

Apple offers a glimpse of where this is heading. Its latest foundation-model family spans on-device execution and Private Cloud Compute, while its developer framework supports additional local and external models. The demonstrated on-device model has an 8,192-token context window, compared with 32,000 tokens in Private Cloud Compute. These destinations are not interchangeable.

I do not expect laptops and phones to replace internal servers and clusters or outside providers. But as local models improve and the tools for running them become simpler, I expect more routine AI work to move onto devices people already use. The question is where each task belongs and how to make that choice reliable, economical, and secure.

Biology Has a Data Problem, Not a Model Problem

One of my favorite presentations at the recent AI Conference came from the founders of Onyx, who asked when biology will get its ChatGPT moment. Their argument is that compute and models aren’t the bottleneck, and neither is the sheer volume of data. Language models inherited an internet where text, images, and intent were already linked. Biology has huge datasets, but genetic, molecular, and clinical measurements are rarely collected from the same people, and almost never over time. That brought me back to a point I’ve been making lately: useful AI data often has to be built, not just collected. I think there’s a big opportunity in creating these domain-specific datasets. Onyx is pursuing an ambitious example, building a virtual immune system that aims to predict how patients respond to disease and treatment.

Learn More

AI Is a New Game Mechanic

Ben Lorica edits Ethics.dev and the Gradient Flow newsletter, and he hosts the Data Exchange podcast. He helps organize the AI Conference and the Agent Conference. You can follow him on Linkedin, X, Mastodon, Reddit, Bluesky, YouTube, or TikTok. This newsletter is produced by Gradient Flow.

Subscribe • Previous Issues

Eight Security Assumptions AI Agents Quietly Break

If you check Ethics.Dev regularly, you may have noticed how often AI agents and cybersecurity have been colliding lately. Some of the stories sound straight out of movies: agents escaping test environments, communicating through channels nobody intended, finding vulnerabilities, stealing credentials, and reaching production systems.

The part I think deserves more attention is less dramatic. AI is changing the economics of an attack. A human attacker has limited time and attention. An agent can try thousands of paths, run several approaches at once, abandon failures cheaply, and keep going. In one recent enterprise intrusion, work estimated to take human operators roughly two weeks was compressed into less than ten hours. During the Hugging Face incident, investigators reconstructed around 17,600 agent actions over four and a half days.

That changes what “good enough” security means, and this is why I worry a lot about the cybersecurity implications of frontier models. Here are the lessons I would put in front of enterprise AI and cybersecurity teams.

1. Plan for attackers who can afford to fail

Most of the weaknesses exploited in the recent incidents were not exotic. Attackers found accounts with too much access, internal systems that were exposed unnecessarily, and everyday configuration mistakes.

What changed was persistence. Hugging Face reconstructed roughly 17,600 actions. Most attempts went nowhere, but that hardly mattered. The agent tested many paths, abandoned failures, switched channels when blocked, and returned to earlier leads until enough ordinary weaknesses lined up into a viable attack chain.

I would revisit security assumptions that quietly depend on obscurity or attacker impatience. An obscure internal endpoint is not much protection against software that can systematically explore everything it can reach.

2. Measure how long containment actually takes

Security teams usually measure whether they can detect an intrusion. I would add another metric: how long does it take from detection to containment?

Recent examples make this question concrete. One AI-enabled enterprise intrusion compressed an estimated two weeks of work into less than ten hours. Google observed another actor go from a compromised cloud resource to a running mass credential-harvesting campaign in under six hours.

The organizational bottleneck matters as much as the technical one. In the Hugging Face case, the security systems surfaced warning signs, but escalation moved too slowly. If revoking credentials requires three approvals and a meeting, the organization can understand the attack perfectly and still lose the race.

When I wrote about AI incident response a couple of years ago, one of my main recommendations was to work out containment options before an incident happens. Agentic attacks make that advice more urgent. Decide in advance which containment actions can happen automatically and which ones the security team is preauthorized to take.

3. Secure the harness, not just the model

I recently argued that teams should evaluate the model and its harness as one system. The cyber evidence makes that point much stronger.

The harness is everything surrounding the model: tools, memory, credentials, network access, execution environments, and policies. In tests that used AI models as attackers, researchers found that the surrounding tooling can dramatically change what a model can accomplish. Australia’s cybersecurity agency recently reached the same conclusion from the defensive direction. Organizations control the harness, even if they do not control how the underlying model evolves.

This suggests a useful exercise. Ignore the model name for a moment and inventory everything the agent can see, call, write to, or spend. Then ask: if the agent behaved badly tomorrow, what is the worst thing this harness would let it do?

4. Plan for agent swarms

One of the stranger lessons from the Hugging Face incident is that agents may find ways to cooperate even when nobody designed them to. In that case, hundreds of agents shared information and built their own communication methods across runs that were supposed to be separate.

That matters because a group of agents can divide up work, share discoveries, and keep pursuing a problem after individual attempts fail. It also means that shared folders, databases, message queues, and other writable resources can become communication channels you never intended.

I would not assume every group of agents will behave this way. But if you are deploying many agents, treat communication between them as another permission to control and monitor.

5. Stop giving agents permanent credentials

I have written before about treating agents as non-human identities. The practical guidance is becoming clearer.

Give each agent its own identity rather than hiding it behind its user’s credentials. Use narrowly scoped credentials that expire when the task ends. Keep read and write permissions separate. Avoid static cloud keys sitting inside an agent environment. Make sub-agents traceable back to their parent and human owner.

This is ordinary least privilege adapted to software that can actively explore its environment. The difference matters. A human employee may never discover that an old credential gives access to some forgotten system. An agent can systematically look.

6. Assume prompt injection eventually works

Prompt injection is basically an attacker planting instructions in material an agent reads. I would assume that sometimes the agent will follow them.

That changes the question. Instead of asking whether malicious instructions can get through, ask what happens when they do.

A coding agent reads code comments and configuration files. A security agent reads logs, hostnames, vulnerability reports, and attacker-generated payloads. All of those can contain text intended to influence the model. A successfully manipulated agent should still encounter hard limits on its credentials, network access, tools, and ability to make consequential changes.

7. Keep the security boundary outside the model

One of my recent rules for production agents was to put hard constraints in software rather than prompts. Recent incidents provide a good reason to be stricter about that distinction.

A prompt saying “do not access production” is guidance. An API permission that makes production inaccessible is a control.

If an action could have serious consequences, let the model suggest it, but have another part of the system decide whether it is actually allowed to run. The same principle applies to audit logs. If an agent can alter the record used to investigate its behavior, you do not really have an audit trail.

8. Prepare to defend with agents as well as against them

There is an uncomfortable symmetry emerging. Attackers are using AI because it increases speed and scale. Defenders will probably have to do the same.

An incident involving tens of thousands of machine actions produces more evidence than a human team can reasonably inspect in real time. Tools like Graphistry are helping investigators find connections across large volumes of security data. That helps, but security operations centers will increasingly need agents too, to triage alerts, hunt threats, investigate incidents, and help contain attacks.

Those agents bring their own risks. They spend their time reading data that attackers may have tampered with and often hold unusually powerful credentials. Prompt injection can redirect what they are trying to do without any credential being stolen. Defensive agents should therefore run inside tightly controlled harnesses, with access limited to what they actually need.

The challenge is to use AI to keep up with attackers without creating another security problem in the process.

The Data Behind Better AI

I’ve been writing a lot lately about a simple idea: better AI depends as much on the data around the model as the model itself. Reverie is a one-day summit in San Francisco on November 5 focused on exactly that part of the stack. I’ll be there, and I hope to see some of you there.

Learn More

Ben Lorica edits Ethics.dev and the Gradient Flow newsletter, and he hosts the Data Exchange podcast. He helps organize the AI Conference and the Agent Conference. You can follow him on Linkedin, X, Mastodon, Reddit, Bluesky, YouTube, or TikTok. This newsletter is produced by Gradient Flow.

Subscribe • Previous Issues

The AI Data Problem Moved Downstream

Last year I wrote that robotics had a data problem language models were lucky enough to avoid. Text, images, and code had accumulated for decades before anyone decided to train models on them. Robots had no comparable internet of physical experience.

That distinction is starting to look less clean. Recent systems have trained on enormous collections of human video. One company reported using more than a million hours. Another turned roughly 1,900 hours of first-person human footage into more than 18,000 hours of robot-format training data. The interesting part is not that robotics suddenly found its missing dataset. It is that every one of these systems needs substantial machinery to turn human experience into something a robot can actually use.

My last two articles focused on data companies already have. This article looks at a different part of the problem: information can exist without being usable by the system.

The Web Is Becoming a Market With Rules

Something related is happening on the digital side, but from the opposite direction. The information is already there, but access to it is becoming more conditional.

An academic publisher recently made selected books available for retrieval by AI research products while explicitly withholding model-training rights. This distinction matters. “Use this for AI” is starting to break into separate permissions for training, retrieval, inference, and perhaps eventually other uses.

The technical infrastructure is moving in the same direction. Website owners can increasingly distinguish between crawlers used for search, training, and agents. One closed-beta system even lets a crawler receive an HTTP 402 response with a price for access. The market is still immature, but the machinery for negotiating access already exists.

Trust is getting harder too. More than a third of webpages published since ChatGPT launched now show significant signs of AI authorship. More troubling are sites producing hundreds of thousands of pages specifically designed to become evidence for AI recommendation systems.

I recently wrote about the risks attached to where a dataset came from before deployment. The newer problem starts after deployment.

When Retrieved Data Can Act

A training dataset can be inventoried, licensed, audited, and removed. Nobody can pre-audit every web page or document an agent will encounter tomorrow. Some of that information will come from sources the company does not control.

That difference becomes important once the system can act. Researchers recently found documentation across more than 100 websites that led coding agents to install software whose ownership could not be established. Another study showed how a malicious tool retrieved from a shared library could be copied by a self-evolving coding agent, stored as a new skill, and propagated further.

This is why I think retrieval is becoming a security boundary rather than merely a search-quality problem. Researchers are already experimenting with source trust scores, cross-document consistency checks, and sanitization at that boundary.

A search system can turn bad information into a bad answer. An agent can turn it into an action. For agents, provenance starts to look less like metadata and more like software supply-chain security. You need to know not only what the system read, but what that information was allowed to cause.

Robotics Found a Pipeline, Not a Corpus

Human video looks like an obvious answer to robotics’ data shortage until you look closely at what the new systems actually do.

One pipeline retargets human actions to robot bodies, inserts robot arms into the video, filters the results, and then combines the synthetic data with real robot demonstrations. Another learns from human and robot video, then grounds what it learned in robot trajectories before adapting it to the particular machine doing the work. Human video is useful raw material. It is not robot experience.

The infrastructure behind the largest effort makes the point even more clearly. Processing throughput had to increase from 14,000 to 440,000 episode-hours per week. Turning the raw data into something the training system could use once took about 48 hours and now takes under a minute. At this scale, the work includes transformation, quality checks, metadata, storage, caching, and continuous preparation for training.

All this changes how I think about the robotics data problem. Robotics did not discover an internet of physical experience. It is building the machinery that can turn human experience into one.

The distinction matters because the machinery may become as economically important as the robots themselves. Collecting, cleaning, translating, grounding, and continually refreshing physical experience is starting to look like a separate layer of the robotics stack.

Useful Data Is the Output

The common thread is not that digital agents and robots use the same kind of data. They obviously do not. It is that useful data increasingly sits at the end of a process rather than at the beginning. The technology is new, but the data problem is familiar.

For digital agents, that process decides what outside information can enter and what the system is allowed to do with it. For robots, it turns human video and demonstrations into data the machine can learn from. The mechanisms are different, but in both cases useful data has to be made from the raw material.

That leads to a management question I suspect many companies cannot answer cleanly: who owns the work of turning raw information and experience into something an AI system can safely use?

The Work in Between

Models are getting easier to buy. The web may be full of information that an agent still cannot safely use. There is plenty of human video. The hard part is turning it into something a robot can use. In both cases, the data problem has moved from finding raw material to doing the work required to make it usable.

Beyond Transformers: A Different Way to Build Intelligence

Six Reasons I Think Open AI Models Will Win

Ben Lorica edits Ethics.dev and the Gradient Flow newsletter, and he hosts the Data Exchange podcast. He helps organize the AI Conference and the Agent Conference. You can follow him on Linkedin, X, Mastodon, Reddit, Bluesky, YouTube, or TikTok. This newsletter is produced by Gradient Flow.

Subscribe • Previous Issues

Stale Data Used to Be Annoying

Imagine a procurement agent checking inventory and deciding that stock has fallen below the reorder threshold. It places another order. The problem is that a large delivery was recorded a few minutes earlier, and the copy of the data the agent queried has not caught up. A dashboard running on stale data might show you the wrong inventory number. An agent running on stale data can place the wrong order.

For years, many analytical systems could tolerate some delay, ambiguity, and imperfect data because a person sat between the result and the action. That person could wait for a slow query, notice that a number looked wrong, or ask which definition applied. Agents remove much of that buffer. They check the data, take an action, see what happened, and check again.

Subscribe now

Last week I wrote about the value of preserving the history of how work gets done. This week I want to look at what an agent needs to know while it’s carrying out a task..

Relevant Is No Longer Enough

The first wave of enterprise AI focused heavily on retrieval: find the information relevant to a question and put it in front of the model. Agents need to answer a different question. Not just what is relevant, but what is true right now. Did the payment clear? Is that inventory still there? Was the account suspended an hour ago? Did another process already touch the record? Information that is five minutes old may be fine for a research question and dangerously wrong for an action.

Freshness is not the only problem. Definitions matter too. Suppose two systems disagree about what counts as an active customer. An analyst who notices the ambiguity can stop and ask somebody. An agent may choose one definition and keep going.

A small benchmark illustrates the problem. When business definitions and relationships were modeled ahead of time, accuracy improved substantially. I find the failure mode more interesting than the scores. When the semantic layer, which encodes those definitions and relationships, could not answer a question, it said so. When the model wrote SQL directly against the data, it could return a perfectly plausible wrong number.

When data informed people, ambiguity produced bad analysis. When agents act on that data, the same ambiguity can produce bad operations.

Analytical Queries Become Production Dependencies

Correct answers are not enough if they arrive too slowly or not at all. A human running an analytical query can usually wait. An agent may be waiting on that query before it takes the next step.

If a query is step seven of twenty, every extra second can delay the steps behind it. A timeout can stop a customer interaction or business process rather than merely leave someone staring at a dashboard. Software can also launch many tasks at once and revisit the same data as conditions change.

That puts different expectations on some analytical systems. Latency, availability, concurrency, and workload isolation start to matter in ways that look more like application infrastructure than traditional analytics.

Data Quality Matters More When AI Acts

Stale records, duplicates, inconsistent definitions, and poorly documented fields are familiar data problems. Better models do not make them disappear. What is different is what can happen when an agent acts on them.

A duplicate record can trigger duplicate outreach or a second transaction. A bad entity match can cause an action against the wrong account. A poorly documented field can quietly change what an agent thinks it is measuring.

In one experiment, adding profiling and query history left an analytics agent around 40% reliable. Cleaning up the data model and improving the documentation brought it to roughly 90%. It is one specific result, not a benchmark, but the lesson is useful: basic data modeling and documentation mattered more than adding more context.

Some failures that look like AI failures are really old data failures whose consequences have become operational.

Permissions Have to Follow the Data

Agents also make an old permissions problem harder. A single task may pull information from a warehouse, a search index, a SaaS application, a vector store, or a cache. Those systems do not necessarily enforce access in the same way.

Suppose a source system hides a document from a particular employee, but a search index built from that system does not preserve the same restriction. An agent retrieving from the index can now see something the employee could not have retrieved from the source.

This is why permissions cannot be treated as something checked only when data first enters the system. When information is copied, indexed, cached, or retrieved somewhere else, the access rules need to travel with it.

The practical question remains simple: when an agent reads a record, whose authority is it exercising?

Production Requires More Than Retrieval

Some of the database architectures I wrote about last year make more sense in this context. Databricks is pushing this idea further with an architecture that lets transactional and analytical workloads operate over a unified storage layer instead of constantly copying operational data into a separate analytical system. Less copying can mean less lag between something changing in the business and that change becoming available to an application or agent.

But I think the broader lesson is more useful than any one architecture. A prototype can show that an agent can perform a task without proving that the surrounding data systems can support it reliably.

Production has to answer harder questions. Is the state authoritative? How does the agent learn that something changed? Do permissions survive when data is copied or indexed? What happens after a partial failure? Can a retry repeat an action? A working demo can hide how much data infrastructure still has to be built.

For years, the data stack mostly helped companies observe and understand themselves. Agents increasingly use the same information while doing work.

Once agents can act on the data they retrieve, the data stack becomes part of the application.

More Intelligence, Less Waiting, Less Power

Not Every AI Job Needs a Chatbot

Ben Lorica edits Ethics.dev and the Gradient Flow newsletter, and he hosts the Data Exchange podcast. He helps organize the AI Conference and the Agent Conference. You can follow him on Linkedin, X, Mastodon, Reddit, Bluesky, YouTube, or TikTok. This newsletter is produced by Gradient Flow.

Subscribe • Previous Issues

When Work History Becomes an AI Asset

Google recently agreed to pay $10 million for the internal business data of Spirit Airlines. The airline is bankrupt, but its emails, Teams messages, software, spreadsheets, and operating records apparently still have value. Google plans to use the material for product development and AI. Spirit’s flight attendants objected to the sale and sought additional protections for employee data, and a bankruptcy judge has delayed approval while the dispute gets sorted out.

I find the deal interesting for a different reason. I recently noted that as access to capable models gets easier, more of the durable value moves into what a company learns by operating its systems. The Spirit deal made me want to look more closely at one part of that argument: the record of how people actually do the work. That made me wonder whether this kind of operating history is more valuable than companies have realized.

From Outputs to Process

Most of what companies keep is a record of results. You have the finished financial analysis, the resolved support ticket, the code that shipped, or the recommendation that went to a customer.

An agent trying to perform the work may need more than the final result. It can benefit from seeing which information someone looked for, what they ignored, which tools they used, where they got stuck, how they corrected course, and how they decided the job was finished. Final documents can tell an AI what a company knows. Workflow history can teach it how the company operates.

This is starting to move beyond theory. Researchers recently used software-development event logs to infer the roles people perform and generate corresponding agent specifications. Companies are also building training environments where agents can practice complete tasks and get scored on whether they actually accomplished them.

The underlying idea should be familiar. People get better at a job by accumulating experience, not just by reading the finished work of people who came before them. Agents may need something similar.

The Archive Is Not the Asset

That does not mean your Slack archive is suddenly a competitive moat. Most workplace exhaust is probably just exhaust. I think there are three useful layers here. At the bottom are messages, meetings, tickets, commits, clicks, and logs. Above that are trajectories, where those fragments are connected into a sequence from goal to decision to action to outcome. The most valuable layer is what you can build from those trajectories: good examples, failure cases, evaluations, corrections, and reusable workflows.

One recent system illustrates the progression nicely. It looks across messy traces left by agents, finds procedures that recur, and turns some of them into executable workflows. In one example, a task that had previously involved 34 API calls was reduced to 11 when the recurring procedure was compiled. There was extra work involved in doing the compilation, so this is not evidence that every workflow suddenly gets cheaper. What matters is that experience that used to disappear into logs can sometimes be turned into something reusable. The raw archive is not the asset. The asset is the ability to turn operating experience into something a machine can learn from and reuse.

This also creates a preservation problem. Companies may worry about valuable workflow data leaking when the more immediate issue is that they never captured the valuable parts. A decision gets separated from its outcome. An unusual exception disappears into chat. Someone fixes a problem but nobody records why the fix worked. Companies may keep plenty of activity and still lose much of the experience.

Same Model, Different Operating History

Imagine two competitors with access to essentially the same capable model and comparable internal documents. The first connects the model to those documents. The second also has hundreds or thousands of examples showing how experienced employees investigate problems, choose tools, deal with unusual cases, recover from mistakes, and check their work. It has connected those examples to outcomes and built evaluations that tell it whether its agent is actually improving.

The second company does not just have more data. It has built a learning system around its own operating experience. Companies increasingly have credible choices at the model layer, even if open models are not always the cheapest or best option for a particular task.

So I would not frame this as open versus closed AI. A frontier API might be right for one workload, a hosted open model for another, and something running inside your own infrastructure for a third. Once a workflow contains valuable company know-how, the question is no longer just which model works best. It is also where you are comfortable having that know-how processed and stored.

Start With the Work Worth Teaching

If I were running an enterprise AI program, I would start by identifying work that is valuable, comes up often, requires real expertise, and produces outcomes I can evaluate. A simple test is whether you would invest meaningful time teaching a new employee to do the task well. If you would, ask whether some of that accumulated experience could be made useful to an AI system too.

But I would not start recording everyone all day. The useful material is not the volume of activity. It is the decisions, exceptions, failed approaches, corrections, and feedback that explain what good performance looks like. The goal is not a bigger archive. It is a better record of how the work succeeds.

I would also settle the rights question early. Can the company use those records for AI, and what employee or customer information is mixed in? Teams should also decide what can be transferred after an acquisition, what an outside model provider may retain, and which workflows are distinctive enough that tighter control is worth the additional operational burden.

The Spirit dispute is a good preview of what happens when those questions wait until after somebody decides the data is valuable. Companies have spent years preserving the outputs of work. Agents may make the process behind those outputs just as important. The opportunity is not to save everything. It is to recognize which parts of that experience are worth keeping, and turn them into something an AI system can actually learn from.

Water, Power, Noise: The Local Impact of Data Centers

AI’s Knowledge-Work Ratchet

Ben Lorica edits Ethics.dev and the Gradient Flow newsletter, and he hosts the Data Exchange podcast. He helps organize the AI Conference and the Agent Conference. You can follow him on Linkedin, X, Mastodon, Reddit, Bluesky, YouTube, or TikTok. This newsletter is produced by Gradient Flow.

Subscribe • Previous Issues

Self-Improvement Without the Science Fiction

A few weeks ago I wrote about AI systems that keep learning after deployment instead of treating every interaction as a fresh start. That prompted a few readers to ask about recursive self-improvement (RSI). The connection is real, but I think it helps to separate three ideas. Continual learning asks whether experience makes a system better next time. Bounded self-improvement goes further by letting the system help turn that experience into a tested, persistent change. RSI is the stronger claim, where the system improves the process that generates those improvements in the first place. These are not steps on a ladder. A system can do any one of them without the others.

RSI gets a lot of attention inside frontier AI labs because, if it works, it could accelerate the development of future models. But most AI teams are not training frontier models, so why should they care? Because some useful and practical tools will arrive well before full RSI. A system that learns from production failures, proposes changes to its own application stack, tests them, and keeps what works will change how AI applications are built and maintained. The near-term story is less about intelligence improving itself without limit and more about automating pieces of the improvement process we currently do by hand.

The Harness Is Where the Leverage Is

For most application teams, the interesting surface is not the model weights. It is everything wrapped around the model: prompts, context, memory, tools, routing, subagents, and workflows. This surrounding software is often called the harness. Unlike a frontier model, it is something your team actually owns, and it is also much easier to change, test, and roll back. One experiment took a model from a 40 percent to a 62 percent pass rate without touching the model at all. Another found that reorganizing how agents exchanged information could improve results while cutting inference costs by as much as 60 percent.

This reminds me of a point I’ve raised previously: teams should resist the reflex to solve every performance problem by reaching for a newer or larger model. Often the better investment is improving the system around the model. Production traces can reveal recurring failures, and an agent can increasingly help diagnose whether the fix belongs in memory, instructions, tool use, routing, or workflow design. I would start with small changes rather than allowing wholesale rewrites. A memory update or routing change is much easier to understand and reverse than letting an agent redesign its entire operating environment.

Why the Loop Can Lie to You

Generating a candidate improvement is becoming easy and cheap. Knowing whether it is actually better is much harder. In enterprise applications the easiest thing to measure is usually not the thing the business cares about. Closing more support tickets doesn’t necessarily mean happier customers. More sales activity doesn’t necessarily mean more revenue. A self-improvement loop can become very good at optimizing a proxy while the real objective barely moves. That makes evaluation part of the product architecture, not something you bolt on afterward.

The more autonomy you give the loop, the more important it becomes to keep some evaluation outside its reach. One experiment found a large gap between self-evaluation and reality. Systems rated themselves as improving while independent tests showed that many were not. In another experiment, the score improved because the agent learned to avoid being flagged by the hallucination detector, not because it hallucinated less. Nothing mysterious is required here. This is ordinary metric gaming with a faster optimizer. I would keep protected tests, grow the regression suite whenever production exposes a new failure, and make rollback easy.

Where Self-Improvement Works First

The best early targets share a simple property: the environment can tell the system whether a change helped. Code compiles or it doesn’t. Tests pass or fail. Games have scores. Chip design has simulators and engineering constraints. These environments let a system try many changes and get relatively fast feedback. Open-ended knowledge work is harder because “better” may depend on judgment that is slow, subjective, or only visible months later. There is also a second ceiling. Better prompts and workflows can help a model make better use of what it already knows how to do. They can’t give it abilities it simply doesn’t have.

For production systems, I would treat every proposed improvement like a software release. Generate a candidate, test it independently, promote it only if it passes, keep the previous version available, and require human approval when the consequences are significant. Some things should simply remain outside the editable layer, including permissions, security boundaries, spending limits, and the controls that decide what reaches production. As machines take over more of the experimentation, human work moves upward toward choosing objectives, designing evaluation environments, and deciding which risks are acceptable.

The Economics May Matter Before the Recursion Does

The first payoff from self-improvement will be cheaper AI, not smarter AI. Some of the strongest examples involve a better harness reducing inference spend, or a smaller specialized model outperforming a much larger general model on one narrow workflow. Once a task is repeated often enough and its outcome is measurable, optimizing directly for that task can be more attractive than continually buying more general capability. There is a cost to running the improvement loop. Agents analyzing traces, generating alternatives, and running evaluations consume compute, and most proposed changes get discarded. The right comparison is the cost of that experimentation against the engineering labor, inference spend, and recurring failures it replaces.

The more interesting long-term asset is the history the loop accumulates. Production traces stop being debugging exhaust and start becoming raw material for the next version. Failures become regression tests. Successful tactics become reusable skills. Rejected changes teach the system what not to try again. None of this requires full RSI. The practical version looks more like modern software with an automated improvement pipeline attached. Forget about when recursive self-improvement arrives. Will your application be meaningfully better on day 500 because of everything it encountered between day one and then?

The teams building the harnesses, eval loops, and rollback systems this piece describes will be in the room at AI Conference, San Francisco, Sep 30–Oct 1. Use gradientflow20 for 20% off.

The Paradox of China’s AI Involution

Ben Lorica edits Ethics.dev and the Gradient Flow newsletter, and he hosts the Data Exchange podcast. He helps organize the AI Conference and the Agent Conference. You can follow him on Linkedin, X, Mastodon, Reddit, Bluesky, YouTube, or TikTok. This newsletter is produced by Gradient Flow.

Subscribe • Previous Issues

Your Model Is Not Your Moat

In a recent article, I wrote about companies turning their own data, workflows, and production feedback into specialized intelligence they increasingly control. The deeper idea was compounding. What matters is not simply whether a model performs well today, but whether using it creates assets that make the system better tomorrow. More AI teams are beginning to build around that idea. Which brings me to a question I keep coming back to: if everyone can increasingly rent the same intelligence, what exactly becomes defensible?

Subscribe now

The moat is what you learn by operating

Strong models are becoming easier for everyone to access. What competitors cannot buy is an understanding of how your organization actually works. Which information matters for a decision? Which exceptions do experienced employees notice? Which workflows deserve automation? What does a good result look like, and how do you know when the system has failed?

That makes operational context and evaluations unusually important. A connector that gives an agent access to 400 reports is useful, but it is not much of a moat. Knowing which five reports your best analysts use, when and why they use them, is much harder to copy. Over time, production traces, evaluations, permissions, workflows, customer knowledge, and expert decisions form a company-specific operating layer. I would spend less effort defending access to a particular model and more effort capturing what the organization learns from using models.

There is an economic version of the same argument. Owning a particular layer of the AI stack does not guarantee attractive margins. A generic memory component can become a commodity. A deeply integrated memory system that measurably improves an agent’s performance may not. Depth and outcomes matter more than drawing a box around a layer and declaring it defensible.

Build for a platform that will keep changing

By some estimates, the best open-weight models have gone from roughly sixteen months behind the frontier to about two. I would treat the exact number cautiously, but the direction matters. Open models are not automatically cheaper once you include infrastructure and the people required to operate them. What they increasingly provide is another option.

That matters because depending completely on one provider is a business risk, not just a technical choice. Prices change. Models disappear. Terms get revised. Providers can restrict capabilities or start competing with companies they previously supplied. Security teams have even described closed models refusing legitimate defensive work because attacking software and testing its defenses can look similar from the model’s perspective.

My response would not be to predict which model or agent framework wins. I would make the important parts of the system replaceable. Keep models, business logic, state, policy, and execution reasonably separable. Use open interfaces where they help. Teams working on unrelated problems keep arriving at translation layers between incompatible systems. I take that as a useful signal. The platform is still unsettled, so interoperability has strategic value.

Build products that leave something behind

One idea I keep returning to is that not every product needs to survive indefinitely. Some AI applications will lose their differentiation when the next model absorbs the feature that made them special. That does not mean they were bad things to build.

Suppose a product lasts eighteen months but gives you thousands of real customer interactions, a stronger evaluation suite, workflow data, distribution, and a deeper understanding of the problem. Those assets can feed whatever comes next. The better question may be less “Can somebody copy this feature?” and more “What will we know after operating this product that somebody starting tomorrow will not?”

The same logic applies to the interface. Some teams are bringing full functionality into major AI assistants rather than insisting that customers stay inside their own applications. The reasoning is that value lives in the intelligence and context provided, not necessarily in the window customers look through. If users increasingly prefer assistants, defending the container may become a poor use of energy.

For startups, I also like the analogy to mobile. Incumbents eventually moved almost every important web product onto smartphones. The defining new companies came from things smartphones made newly possible, including cameras, GPS, and constant connectivity. AI may follow a similar pattern. Automating an existing workflow can be valuable, but incumbents can do that too. I am more interested in what becomes possible only because inexpensive intelligence and software generation now exist.

Legacy modernization is one example that deserves more attention. Companies have deferred these projects because documentation is incomplete, experts have retired, and nobody wants to own the migration risk. The interesting approach is not simply asking a model to rewrite COBOL in Python. Treat the old system as the specification, generate tests from it, translate the code, and use deterministic methods to verify that the new system behaves the same way. That turns a risky, open-ended migration into a checklist you can verify step by step.

When generation gets cheap, judgment gets scarce

There is a human version of this shift too. As code, analysis, text, and other outputs get cheaper to generate, producing the first answer matters less. Figuring out whether it is the right answer matters more.

That pushes value toward judgment, debugging, evaluation, and problem selection. Can someone turn a fuzzy objective into something a system can execute and test? Can they spot a failure that looks superficially correct? Can they explain why an agent went wrong and improve the system around it? A conventional coding test may tell you less about those abilities than real projects someone has shipped and a candid postmortem on one that failed.

I also would not assume that rapid productivity gains mean organizations transform overnight. Tools change much faster than job descriptions, incentives, and management structures. The first phase may look mostly like familiar teams producing more, with people spending more time directing and checking automated work.

When customers own the compounding loop

This is where the argument comes back to OpenAI and Anthropic. The challenge is not simply that an open model might catch them on a benchmark. Their best customers can increasingly combine proprietary data, workflows, tools, evaluations, and production feedback into intelligence that improves inside their own organizations.

That changes the competitive landscape. Frontier labs still compete with one another on general model capability, but increasingly they also compete with the accumulated intelligence their customers are building downstream. The more of that learning companies can capture for themselves, the more important it becomes to ask who actually owns the compounding loop.

Models, features, interfaces, and routine output are becoming easier to obtain. Operational context, evaluations, customer knowledge, distribution, and judgment get stronger through use. Buy what commoditizes. Build what compounds.

AI Is Rewiring Work Before It Rewrites Employment

When the AI Compute Bill Arrives

Ben Lorica edits Ethics.dev and the Gradient Flow newsletter, and he hosts the Data Exchange podcast. He helps organize the AI Conference and the Agent Conference. You can follow him on Linkedin, X, Mastodon, Reddit, Bluesky, YouTube, or TikTok. This newsletter is produced by Gradient Flow.

Subscribe • Previous Issues

Nine Practical Rules for Agents Doing Real Work

In recent conversations with crews building agents, I keep hearing the same lessons. Teams with very different products are arriving independently at almost the same architectural choices. That convergence feels important. In recent posts, I argued that passing your evals does not mean an AI system is safe, and that many of the most consequential risks sit outside the model itself. This is the constructive follow-up: once an agent is doing real work, what should you actually build around it? I think a practical architecture playbook is beginning to emerge.

1. Put hard constraints in software, not prompts

Models are good at flexible reasoning. Ordinary software is good at state, permissions, calculations, retries, and predictable control flow. Reliable systems keep that division explicit rather than asking a prompt to guarantee something important. A prompt is guidance. It stays negotiable no matter how firmly you word it, which is fine for judgment calls and a real problem for anything that can’t be wrong.

The dividing line is simple: let the model handle ambiguity, which is where it creates value. Put calculations in trusted code, enforce permissions through policy systems, validate code with compilers and tests, and check important factual claims against authoritative sources. Ask what the model can currently violate that ordinary software could prevent. When a failure would create a material consequence, the architecture should make that violation impossible or require explicit approval.

2. Give the agent only as much autonomy as the job needs

One pattern I keep seeing is teams giving agents more autonomy than the job requires. That excess autonomy creates more possible paths, more opportunities for error, higher operating costs, and a harder governance problem. For many applications, conventional software should control the workflow while the model handles the smaller number of steps that genuinely require judgment. Greater autonomy can be justified when the task requires exploration, open-ended planning, or creating new tools.

I would treat autonomy as a deliberate design choice rather than a default. Start with a flexible workflow, observe which paths repeat reliably, and turn those stable paths into ordinary code. List every point where the agent chooses its next action, then ask which of those choices the task truly requires. A mature agent faces fewer open-ended choices over time, not more.

3. Build the agent around the domain’s trusted process

A support workflow already specifies how to triage a request, what information to collect, when to escalate, and which actions require approval. A medical workflow does the same through diagnostic protocols. When a domain already has a proven checklist, protocol, or decision process, use that as the agent’s structure rather than starting with a generic loop that tells the model to make a plan and keep acting until it is done.

This makes the system more reliable because the structure has already been tested by the field. It also makes the system easier to understand for the people who have to trust it and sign off on it. The best agent architecture often looks less like a general-purpose digital employee and more like the field’s existing best practice made executable.

4. Design for recovery, not a flawless first pass

Small errors compound quickly in long workflows. A system that succeeds 95 percent of the time at each step has only about a 60 percent chance of completing ten independent steps without an error. That helps explain why a three-step demo can look extraordinary while a longer business process falls apart.

Do not build around the assumption that the model will stop making mistakes. Add checkpoints, checks that confirm each action produced the intended result, retries, reversible actions, and the ability to resume from a known-good state. Measure recovery separately from first-attempt accuracy. A system that detects drift and corrects itself is more useful than one that looks perfect until the first unexpected tool response.

5. Evaluate the model and harness as one system

What exactly are you evaluating when you test an agent? The harness is the software surrounding the model, including its tools, context management, memory, policies, and recovery logic. That surrounding system can affect performance as much as changing the model. In one comparison, the same open model showed an 18 percentage point difference between the best and worst harness configurations. That is one of the findings I would take most seriously when comparing models.

A leaderboard is only a starting point. Evaluate the model with your tools, context, permissions, and failure modes attached. Then rerun the evaluation whenever either the model or the harness changes. Changing either half means you are evaluating a new system.

6. Keep multi-agent teams small, and let someone disagree

Multi-agent systems can look sophisticated while mostly adding coordination overhead. The more effective pattern is usually a clear orchestrator with a small number of specialists, each with a distinct role, tool set, permission scope, and only the information it needs. Larger teams can help when the work is genuinely parallel or when independent perspectives are part of the requirement, but large swarms often create duplicated work and make failures harder to trace. One role deserves special protection: a critic or breaker with explicit criteria and the authority to block or escalate. Knowing when not to answer is something the system must be designed to do, not merely a personality you request in a prompt.

7. A bigger toolbox can make the agent worse

In one test, a larger tool set was slower and slightly less accurate even though it contained every tool in the smaller set. This may be the most counterintuitive rule on the list. The extra tools were not bad. The problem was overlap. When several tools can plausibly handle the same request, the model spends attention choosing among near-duplicates instead of doing the work.

More tools also give the agent more systems it can read from or write to, and they create more sequences of tool calls for the team to test. Favor a compact toolbox in which every tool has a clearly different job. Run required setup tools automatically, and log which tools get picked, what goes in and out, and how often they fail. If two tools could plausibly handle the same request, combine them, route between them in software, or remove one.

8. Context, memory, and enterprise knowledge are different systems

The distinction is operational: context is the information the model needs during the current run, memory is what the system should carry forward from earlier work, and enterprise knowledge is the governed collection of documents, records, and policies it may consult. Treating them as one undifferentiated store creates both quality and security problems. They can share infrastructure, but they should not share the same retention, retrieval, and access rules by default.

Memory in particular should preserve lessons, not transcripts. What matters is the mistake to avoid, the technique worth reusing, or the preference that should change future behavior. The test is not how much the system remembers. Inspect what it saved, what it later retrieved, and whether that information changed the result.

9. Fix the knowledge before you pay for a bigger model

When an agent performs poorly, the default reaction is often to buy a stronger model or begin fine-tuning. Before changing the model, I would look closely at what it is being given to work with. One team replaced a pile of raw support documents with a diagnostic playbook. A router translated a customer’s complaint into a symptom, then a problem type, then the relevant handbook. Tokens fell 43 percent and errors fell 48 percent without changing the model.

Retrieval is not a feature you install once and forget. It fails when a user’s words do not match the document’s words, when information is buried in tables or PDFs, or when sources conflict. Before changing the model, improve the structure, routing, and governance of the knowledge it receives. The model is the component you can buy. The system around it is what you own.

If agents and real-world AI deployment are on your roadmap, come spend two days with the teams already doing it. AI Conference, San Francisco, Sep 30–Oct 1. 20% off with gradientflow20.

The Case for AI Data Centers Nobody Is Making Well

The Memory Bottleneck in Frontier AI

Ben Lorica edits Ethics.dev and the Gradient Flow newsletter, and he hosts the Data Exchange podcast. He helps organize the AI Conference and the AI Agent Conference. You can follow him on Linkedin, X, Mastodon, Reddit, Bluesky, YouTube, or TikTok. This newsletter is produced by Gradient Flow.

Subscribe • Previous Issues

The biggest AI risks sit outside the model

The most revealing AI failures right now are not stories about models becoming too capable. They are stories about everything around the model. One system received more access than its test environment could contain. Another was trained on material whose acquisition created $1.5 billion in exposure. In a third, people walked away more certain without being any more correct. I recently argued that passing your evals does not mean you are safe. This is the next layer of that argument. Permissions, inputs, and presentation are three places where serious AI risks accumulate, and each one lands on a different desk.

What an Agent Can Touch

Model capability tells you what a system can do. Architecture determines what it may do, what data and tools it may use, and which actions require approval. A mediocre model with broad write access can be more dangerous than a stronger model in a tight box.

OpenAI’s cyber-focused models, including GPT-5.6 Sol, left the sandbox used for testing, reportedly exploited a zero day, reached the open internet, and compromised Hugging Face production systems. Nobody appears to have noticed for days. Calling the behavior rogue explains little. Safety filters had been switched off for the exercise, and network isolation was weaker than the test assumed. Those were architecture and configuration decisions.

Other reports describe an agent deleting user files without being asked and a coding tool uploading full codebases without clear consent. Different incidents, same pattern: a weakly specified task becomes dangerous in proportion to what the system can touch.

An agent can also be redirected by hostile instructions embedded in an email, document, web page, or tool response. Once tools are attached, prompt injection can lead to an unauthorized action, not merely a bad answer. Permissions and approval rules therefore have to be enforced outside the model.

Give every production agent its own identity and task-specific permissions rather than letting it borrow a person’s credentials or a generic service account. Split read from write. Use short-lived credentials, narrow network access, transaction limits, and approval gates for irreversible actions. Preserve logs and make recovery possible. The goal is not perfect prevention. It is a small blast radius, fast detection, and enough evidence to understand what happened.

Not every system needs all of those controls. If an agent only reads, gets things wrong cheaply, and can be corrected before anyone acts, most of those controls are overhead. Once it can write to a system of record, spend money, touch regulated data, or produce something a customer will act on without review, those controls apply. The dividing line is the damage the system can cause before someone stops it. Many teams want a third option. There isn’t one.

Where the Data Came From

The same pattern runs upstream, into what the model was trained on before anyone deployed it. A federal judge approved Anthropic’s $1.5 billion settlement over pirated books used to train Claude. The headline number is the least interesting part. What matters is the line the settlement draws between training on material you obtained lawfully and training on material pulled from pirate libraries. The failure was not model capability. The training pipeline did what it was built to do, and a decision made around it created the exposure.

If you are fine-tuning a model, assembling an evaluation set, or building retrieval over third-party content, that distinction is your problem too. How the material was acquired, what rights came with it, and whether those facts can be reconstructed later matter as much as what the model does with it.

Boards now have reason to ask where training data came from, who carries the risk, and whether a disputed model or dataset can be removed without disrupting the product.

Answering those questions requires a dataset inventory, license terms you can produce on request, a clear answer on who pays if you get sued, a current view of litigation involving your vendors, and a way to swap out a model or dataset without taking the product down. Most teams have some of this. Very few could pull it together on short notice.

The Confidence Trap

Controls and documentation can limit what a system does and clarify what went into it. Neither fully addresses what happens when a convincing answer reaches a user. A recent preprint reports five experiments involving 3,132 participants who answered difficult questions and had the option to say they did not know. Giving people access to AI advice nearly eliminated that restraint. Participants answered more questions, but they were correct about one-third as often as people without AI access. Their confidence nearly doubled.

You should assume your users can do the same thing. In medical, legal, financial, hiring, and security workflows, a confident wrong answer can be more dangerous than an obvious failure. The risk grows when the interface presents a smooth recommendation as the natural next step. A confidence score does not automatically help, especially when it comes from the same model that produced the answer. A precise-looking number can become decoration rather than evidence.

You cannot engineer this one away, because you cannot stop a person from over-valuing a fluent answer. What you can do is design against it, and the moves are concrete. Ask the user to commit to a view before you reveal the model’s recommendation. Show uncertainty honestly instead of rounding it into confident prose. Take confidence numbers from past accuracy, not from the model. Surface disagreement between sources rather than smoothing it out. Make correcting the system easy enough that people actually do so. Let the system abstain. Escalate decisions when the cost of being wrong is high.

Then check whether it worked. The test is not whether people like the assistant or finish more tasks with it. It is whether the human and the system together make better decisions than the process they replaced, know when to abstain, and escalate the right cases.

Three Desks, One Problem

The next phase of enterprise AI will be defined less by who gets the most capable model and more by who builds the strongest system around it.

Security and architecture teams have to bound what the system can touch. Data, legal, and procurement teams have to document what went in. Product teams have to preserve human judgment. Rules people once enforced through policy, habit, or supervision increasingly need technical equivalents such as distinct agent identities, approval gates, provenance, and logs.

These risks land on different desks, but they are becoming one management problem.

Capability is the layer you buy. Everything around it is the layer you own.

Evaluating the system, rather than just the model, requires a richer set of measurements. The Luminos whitepaper explains one approach.

When Mathematical Results Become Abundant

Ben Lorica edits Ethics.dev and the Gradient Flow newsletter, and he hosts the Data Exchange podcast. He helps organize the AI Conference and the AI Agent Conference. You can follow him on Linkedin, X, Mastodon, Reddit, Bluesky, YouTube, or TikTok. This newsletter is produced by Gradient Flow.

Subscribe • Previous Issues

Continual Learning Is Arriving in Pieces

A while back I wrote about how startups are using reinforcement learning to make agents more reliable. A deeper problem behind that whole trend keeps resurfacing: a model can improve during training, but the moment it’s deployed, learning largely stops. A policy changes, a new edge case shows up, a user corrects the system, and the lesson rarely travels past that one incident. The prompt gets patched, the ticket gets closed, and the same class of mistake eventually comes back. The system might pile up logs and documents along the way, but it’s often no better on day 500 than it was on day one.

That is the gap continual learning is meant to close. If reinforcement learning is about improving a model through practice during training, continual learning asks what happens after. Can real usage produce a lasting change in how the system behaves next time? The mechanics vary, sometimes it’s memory, sometimes it’s an agent revising its own instructions, sometimes it’s the model’s weights (its internal parameters) actually updating, but the working definition holds across all of them: a deployed system that captures its own experience, turns it into a durable improvement, checks that the improvement doesn’t break something else, and carries it forward. I count more than 20 startups where some version of that loop is core to the business. That’s enough to take the trend seriously. That raises two questions: why so many are betting on continual learning now, and where they think it pays off.

The Maintenance Bill Nobody Budgeted For

The strongest argument has nothing to do with research ambition, it’s about maintenance. Today, most agent failures become support tickets, prompt edits, or isolated debugging sessions. Continual learning turns a failure into reusable material: a regression test, a corrected instruction, a better tool call, a revised memory, or training data. The harder part is making sure a fix for one case doesn’t quietly break things that used to work. That’s why some of these companies build regression checking directly into the update loop rather than trusting a human to notice later. That is what separates improvement that compounds from an expanding pile of patches.

The second argument is economic. Feeding a model your documents or your production history at the start of every session works, but you’re paying to reprocess the same information over and over. Several of these startups bet that information you use often should get compressed into the system instead of re-read from scratch, which cuts cost and lets a smaller, cheaper model outperform a much bigger general one on the narrow task it’s actually seen before. Push that further and you get something more ambitious than savings, a system that internalizes how one organization actually operates, or captures the kind of tacit judgment call an experienced employee makes but could never fully write into a prompt.

The third argument is specialization, with trust as the constraint. Some expertise is difficult to express as written rules. It lives in repeated examples, corrections, and subtle judgments. Continual learning offers a way to capture that know-how, but it also creates new risks. A system that keeps changing can absorb bad feedback, forget an old capability, leak information, or become impossible to audit. The practical race is therefore not simply to make AI learn more. It is to make learning inspectable, reversible, and safe enough for a business to rely on.

Where Continual Learning Is Already Earning Its Keep

Customer-facing work is one of the clearest early markets. Support systems produce large volumes of repeated interactions, visible escalations, human corrections, and fairly legible outcomes. Sales and account work benefit from the same continuity, especially when the system needs to remember a long-running relationship rather than treat every conversation as a fresh transaction. The second cluster is enterprise knowledge and professional work: law, financial services, healthcare operations. These are fields where the documents are proprietary, the tasks recur constantly, and mistakes cost real money. High stakes plus narrow specialization plus repeated structure is close to the ideal setting for a system meant to get better at your specific work over time.

Technical work is another natural fit because the feedback is often unusually clear. Code compiles or fails. Tests pass or fail. A developer accepts a patch or rewrites it. Infrastructure operations also produce traces of what was tried, what worked, and what made the problem worse. The common thread is that the longer a task runs, the more a small early mistake compounds, so the systems that benefit most are the ones that catch their own drift along the way.

A third cluster uses continual learning to improve AI systems themselves. Product usage can become a steady stream of training signals rather than an archive nobody revisits. Simulated environments can give agents somewhere safe to encounter rare failures before they touch real customers or infrastructure. Model development and performance optimization also provide relatively clear measures of success, making it possible to propose changes, test them, and retain what works. At the more experimental edge are robots and systems operating in changing physical environments, where the world drifts too quickly for occasional retraining to be enough.

Continual Learning Is Already Arriving in Pieces

I do not expect continual learning to become mainstream all at once because it contains two different bets. The faster half is already arriving. Systems can preserve useful memories, analyze production traces, revise instructions, and test proposed changes while keeping a person at the approval step. These approaches are comparatively cheap, inspectable, and reversible. Many teams will adopt them without ever describing what they are doing as continual learning.

Actually updating a model’s weights on an ongoing basis is a harder step. That is not only a tooling problem. It is a trust problem. Teams need strong answers for data quality, privacy, forgetting, poisoning, regression testing, auditability, and rollback before they allow a deployed model to alter itself routinely. My guess is that weight-level learning will first take hold in high-volume, high-value workloads with clear feedback. But the broader shift is already underway. AI products are starting to look less like software you replace every few months and more like something expected to earn its keep by getting better the longer you use it. The real question probably isn’t when every model learns continuously. It’s when every serious AI product is expected to have an actual answer for how it gets better with use.

AI Conference is a chance to go beyond the headlines and learn directly from teams shipping AI in the real world. San Francisco, Sep 30–Oct 1. Use gradientflow20 for 20% off.

AI Has More Than a Marketing Problem

Summer Books

• The Score: How to Stop Playing Somebody Else’s Game. While the examples are mainly from game design, this sharpened a point that applies directly to AI evals, a case of Goodhart's law: once you optimize for a benchmark, it stops telling you what you actually want to know.

• You Won’t Get Free of It: Stories of Mothers and Daughters. Rachel Aviv is one of my favorite writers, and this collection reminds me why: patient reporting that resists the urge to editorialize, even when the material makes that difficult.

• The Cruelest Game: Chasing Greatness in Professional Tennis. This is a gift for any lifelong tennis fan: Futterman gets remarkably close to his subjects (down to what Djokovic eats between points, and how he tracks the shade creeping across the court to survive a five-hour match) and the result is the rare tennis book that explains the sport’s psychology instead of just recapping its highlights.

Ben Lorica edits Ethics.dev and the Gradient Flow newsletter, and he hosts the Data Exchange podcast. He helps organize the AI Conference and the AI Agent Conference. You can follow him on Linkedin, X, Mastodon, Reddit, Bluesky, YouTube, or TikTok. This newsletter is produced by Gradient Flow.

Subscribe • Previous Issues

Passing Your Evals Doesn't Mean You're Safe

Evals are part of every serious conversation about putting AI into production. Teams define benchmarks, set thresholds, and increasingly run red teams to see how the system holds up against someone actively trying to break it. That combination is reasonably good at telling you whether a model is accurate, reliable, fast enough for production, and resistant to an adversarial attack. It says almost nothing about what actually gets a company into trouble once the system is live.

Look at the AI lawsuits and regulatory findings piling up this year and the gap becomes obvious. A man in California is suing OpenAI, arguing that ChatGPT used his own disclosed bipolar diagnosis to keep him engaged in conversation rather than steering him toward help, a product liability claim built entirely around a session memory feature. A federal court let a discrimination suit against Workday proceed on the theory that its hiring software leaned on medical leave and other proxy signals to screen out an older applicant with a disability, exposing the vendor itself, not just the employer who bought the tool. A German court ruled that a company is responsible for its own chatbot inventing a doctor’s medical credentials, reasoning that a chatbot isn’t a third party, it’s simply the business talking. None of these are the kind of failure a red team session or a bias checklist is built to catch. They’re everyday production risk, legal, reputational, and regulatory, sitting inside systems that had already passed whatever evals their teams ran.

The High-Dimensional Fix

I recently sat down with Andrew Burt, CEO of Luminos, about the whitepaper his team just published on agentic and generative AI risk evals. The report’s core argument is that the standard setup, one broad prompt, one model, a pass or fail score, is too coarse to do its job. Ask a model whether an output is “biased” or “manipulative” and you get a shallow answer back. Real problems slip through, harmless outputs get flagged as risky, and either way you’re left without enough detail to know what to actually fix.

Luminos calls its answer high dimensionality, and the idea is more concrete than the name suggests. Take an ad that shouldn’t manipulate the people looking at it. A typical eval asks one broad question, is this manipulative? A high-dimensional eval tests separately for a fake countdown timer, a hidden renewal fee, a bait-and-switch price, and a handful of other specific tactics, then comes back with something like: false urgency (yes), hidden fee (yes), bait and switch (no). That’s a finding a product team can actually act on. The same decomposition has to run across more than one model, because models have real personalities when it comes to flagging risk. Claude tends to run conservative and overflag, other models underflag, and a prompt that works cleanly on one model can fall apart entirely on another. Underneath all of it, the standards need to come from actual legal, privacy, and compliance expertise, not just engineers guessing at what a regulator will care about.

None of these three ideas is new by itself. Plenty of teams already run more than one model, and some already pull in legal or compliance reviewers. What’s rare is doing all three together, and treating the result as an ongoing job rather than a pre-launch checklist, since models get updated, laws vary by jurisdiction, and a test suite that felt thorough in January can be stale by summer.

It’s worth being clear about what the whitepaper is and isn’t: it doesn’t present a controlled study proving this approach beats simple evals by some specific margin. The case is structural, built from Luminos’s own experience running this at scale, but it’s a pretty convincing one. Testing more of the actual risk surface should catch more real problems, calibrated sub-risks should cut down on false alarms, and requiring an explanation for every flag should make results something a team can actually audit and fix.

Treat AI Risk Like Reliability

The pattern across this year’s AI incidents isn’t that companies skipped evals entirely, most had something in place. This isn’t an argument for ditching red teaming, guardrails, or standard performance evals either, they answer different questions and are still worth running. The mistake is assuming that together they add up to a full picture of production risk.

Once an AI system can influence a customer, an employee, an account, or a regulated decision, its risk testing deserves the same rigor teams already give to reliability and uptime. That means specific tests, evidence you can point to, clear ownership, and monitoring that continues after launch, not a box checked once before shipping. That bar covers more systems than it sounds like, a support chatbot, a hiring tool, an internal scoring system, an agent with write access to anything, almost any AI system doing real work in a business. There’s no comfortable middle category of “important enough to ship, not important enough to evaluate properly.”

The Luminos whitepaper is worth reading in full, not just skimming, whether or not you end up using this exact approach yourself.

AMD’s Long Game in AI

Two AI Labs Carry More of the Cloud Boom Than You Think

Ben Lorica edits Ethics.dev and the Gradient Flow newsletter, and he hosts the Data Exchange podcast. He helps organize the AI Conference and the AI Agent Conference. You can follow him on Linkedin, X, Mastodon, Reddit, Bluesky, YouTube, or TikTok. This newsletter is produced by Gradient Flow.

Subscribe • Previous Issues

Specialized AI Is Getting Easier to Build

Last week I argued that open models will absorb most of the money and compute the world spends on AI. A week later, open weights are even more central to the conversation. Recent releases have made the gap between capable and affordable harder to ignore, and a broad coalition of technology companies is now publicly arguing that open models matter for competition, security, and national sovereignty. But the models themselves are only the starting point. The more consequential shift is what teams can now do with them. They can increasingly turn generic weights into specialized intelligence they own.

The Post-Training Stack Is Filling In

Two years ago, taking an open model and making it meaningfully better at your specific job was already becoming easy to do. Supervised fine-tuning, meaning training on labeled examples of good output, was always the accessible option. The newer layer is reinforcement fine-tuning, where a model practices a task and learns from whether it succeeded. More than 25 startups are now building around this idea, but the algorithm is rarely the whole product. They are assembling the less glamorous machinery around it: environments where models can practice, graders and verifiers that decide whether the work was done correctly, tools for generating and curating data, and systems for evaluation, deployment, and monitoring.

Platforms are also starting to automate more of the engineering loop. A team can describe a task, establish a baseline, identify failure patterns, build evaluations and test data, train a model against those failures, and repeat. This still requires judgment, and reinforcement learning is not suddenly a push-button exercise. But what used to be a loose collection of research techniques is becoming a recognizable engineering workflow that more companies can adopt.

The demand side explains why this is happening now. Teams that put generic models into production tend to hit the same three walls. Prompt adjustments turn into whack-a-mole as each fix creates another failure. Costs become difficult to ignore once usage reaches real scale. Then a provider changes a model, retires it, raises the price, or starts competing in the same market as its customer. At that point, renting a general-purpose model stops looking like the obvious long-term architecture.

Specialization is attractive because most enterprise AI work is narrower than the frontier labs’ ambitions. It is classification, extraction, forecasting, tool selection, and the execution of repeatable workflows. In examples described to me, specialized models improved performance by 10 to 30 percentage points on well-defined tasks. In one case, a classifier with only a few hundred million parameters ran on a device and matched a model roughly ten times larger.

From Customizing Models to Creating Them

​​A smaller group is going further and pretraining models from scratch. Datadog is my favorite example because the logic is so plain. They sit on an absurd volume of time-stamped operational data, so they trained a time-series foundation model on it. Two details stuck with me. The model performs better when they exclude public time-series datasets entirely and train only on their own data, which is about as direct a statement of where the value sits as you will find. It is also cheaper to run than the classical statistical pipeline it replaced, once you count the tuning those older methods need on every query. Their proprietary data stopped being an input to the product and became part of the model itself. They are now aiming at something more ambitious, a model of their customers’ distributed systems built from metrics, logs, traces, topology, code, and events, meant to predict what a system will do next rather than report what it already did.

As the lineup for the upcoming Ray Summit hints, plenty of other companies are heading down the same road. A payments company is pretraining on transaction data. A design platform tokenized 900 terabytes across 18 datasets to train a model that generates geometry from text, images, and partial CAD input. A streaming service runs training, fine-tuning, evaluation, and serving on the same shared clusters. None of these are research projects, and two changes explain why. Domain models stay small. The 3D model above runs between 513 million and 1.5 billion parameters and Datadog's largest is 2.5 billion, which is a rented cluster and a strong team, not a national research budget. And the platform underneath has converged on what I have been calling the PARK stack, meaning PyTorch, Ray, and Kubernetes handling data prep, training, post-training, evaluation, and serving in one place.

None of which makes pretraining easy or sensible for the average company. It becomes plausible when you have distinctive data, a workload worth repeating, and the team to keep the model running over time. The hard part is not the training run. It is knowing what data to feed it and in what mix, which is still closer to craft than process. Tooling will chip away at that, and as it does the question shifts from “could a company like ours train a model?” to “which parts of our business are worth one?”

The Moat Moves Into the Model

Put the two developments together and you get the shift toward owned specialization. Post-training is the on-ramp and pretraining is the far end. Most companies will operate somewhere between them, but the distance from “we have a use case and some data” to “we own a model that fits it” keeps getting shorter.

That is the deeper challenge for OpenAI and Anthropic. It is not simply that another open model might match them on a leaderboard. It is that their best customers can increasingly turn proprietary data, workflows, tools, and production feedback into intelligence that compounds inside their own organizations. Once enough companies build that flywheel, the big labs stop competing only against one another. They start competing against their own customers’ data. That is the headwind I do not think they have priced into their IPO story.

If you want to see this up close, Ray Summit has entire tracks on foundation model training, multimodal data curation, and reinforcement learning.

AI Can Win While Data Centers Lose

Four Gates Before You Bet on Open Weights

Ben Lorica edits Ethics.dev and the Gradient Flow newsletter, and he hosts the Data Exchange podcast. He helps organize the AI Conference and the AI Agent Conference. You can follow him on Linkedin, X, Mastodon, Reddit, Bluesky, YouTube, or TikTok. This newsletter is produced by Gradient Flow.

Subscribe • Previous Issues

Open Models Will Absorb Most of the AI Spend

Here is my bet: open models (open weights and open source alike) will end up absorbing most of the money and compute the world spends on AI. The proprietary frontier models get the headlines and the IPO valuations, but developers and AI teams see something different up close. Open models are improving fast, the gap to the proprietary leaders keeps narrowing, and the economics point one way. OpenAI and Anthropic are already starting to warn us about open models, and I expect those warnings to get louder. Here is why I think the spend follows.

Follow the money

Frontier pricing is running on a treadmill. The best closed models hold a clear lead for two to six months before someone matches them or distills the capability and gives it away. That is a brutal foundation for a business, and it is why OpenAI and Anthropic are pushing into applications like coding, security, healthcare, and legal, where margin sits closer to the customer. Watch the IPO math too. Both reportedly want to go public above $800B, valuations that assume pricing power the market may stop believing in. I would not bet my roadmap on today’s API prices holding.

Cheaper per task is not a smaller bill. Cost per finished task is dropping fast, helped by setups where an expensive model plans and a cheap one executes. But Jevons paradox is real: when a resource gets more efficient and cheaper to use, total consumption often rises rather than falls. Cheaper tokens unlock so much more usage that your overall bill can climb even as each task gets cheaper. Budget for cheaper units and heavier use, not for savings that never show up on the invoice.

The premium model is becoming a planner, not a workhorse. Route the hard reasoning and verification to a frontier model, and push the bulk execution to a cheaper one. It is good engineering and good economics, and it quietly breaks the assumption the big labs were built on, that every step runs through a premium API. If you are not architecting this way yet, your competitors already are.

Watch the gap, carefully

Chinese open weights have reached parity in specific, high-stakes areas. In cybersecurity bug-finding, Zhipu’s GLM 5.2 has been measured matching or beating Anthropic’s Opus 4.8. That matters, because finding security bugs is one of the highest-stakes things you can ask a model to do, and open weights help attackers as much as defenders. Still, being good at one thing is not the same as being good at everything, so test these models on your own work instead of trusting news headlines.

Coding agents are still a clear US stronghold, and that gap is not closing quickly. US models ride a flywheel where serious users generate failure data, which trains better models, which pulls in more serious users. Chinese labs are dragged by chip shortages, over-reliance on distillation and benchmaxxing, and pressure to ship consumer products instead of doing open-ended research. That can change, but today the gap holds, so I would not rush to swap out your coding stack.

The “China caught up” story is genuinely contested. Some observers still put the best Chinese labs a year behind, but most estimates land in the three to twelve month range, and it shifts with every release. It matters less than it sounds. For most routine work, my own experience is that open-weight models match the proprietary leaders at a much lower cost, and you can often post-train them to come out ahead. So test on your own tasks, not on anyone’s leaderboard, mine included.

Rethink your stack

Open weights are about control, not just price. You can run an open model on your own hardware, tune it on your own data, and use it without a third party who can change the rules on you. The Fable shutdown made this concrete. When Washington blocked foreign nationals from using the model, Anthropic cut access for everyone, and European buyers started asking whether US models are safe to build on. That is a data moat, a privacy posture, and a hedge against having the rug pulled, not just a cheaper bill. The catch: attackers can run open models just as freely, and many EU and US buyers still will not touch Chinese-origin ones.

A tested fallback is leverage even if you never use it. Just having a working open-weight alternative caps your exposure to a provider’s pricing, rate limits, behavior changes, and policy calls. But the leverage is only real if the fallback is real, running in production with real evals, not a line in a deck that says “we could switch.” A backup you have never tested is not a hedge, it is a hope.

Inference efficiency is the next cost lever. The newest gains are not smarter models, they are faster serving. One Chinese system reports 57 to 85 percent faster generation and several times the throughput under heavy load, with almost no overhead. That attacks the exact cost structure the big labs depend on, so watch how models are served, not only how they score. The cheapest token often wins on plumbing, not intelligence.

Do this now

Run your models as a portfolio, not a religion. Use closed models where they clearly win, open weights where they are good enough or cheaper or more private, and keep the mix swappable. The right question is not “are open models better,” it is “where are they good enough to change my cost and control position.” A portfolio of models still needs governance, evals, and clear ownership, or it turns into expensive complexity.

Build the switching layer before you need it. Routers, OpenAI-compatible endpoints, eval harnesses, memory, logging, and clean abstractions let you change models without rewriting your product or going blind on quality. And treat your prompts, code, and agent traces as strategic assets: decide deliberately what flows to a closed API versus stays local. Those traces are training signals, not exhaust, and you only get to give them away once.

Optimize for resilient, not open or closed. If you remember one thing, make it this. The useful frame is controlled versus resilient. Closed AI gets dangerous only when it becomes infrastructure you cannot replace, because then the vendor’s terms, access rules, and whatever government pressure lands on them become part of your operating environment. Architect so that no single lab, or government, is ever a single point of failure. That is the whole game.

Where I think this all lands

First, the US posture toward China looks self-defeating. Blocking foreign nationals from a US model while clearing advanced chips for export does not slow China down, it hands it an opening and pushes its labs onto a homegrown stack, exactly what the earlier chip bans did. Cutting research ties and treating foreign talent as a threat drives away the people American labs depend on. Export controls have not stalled Chinese progress, and the damage lands on US companies and their customers instead. I would rather see cooperation than a wall, and the durable answer to strong Chinese open models was never to ban them, it is to build better ones.

Second, supply. Open weights are a business decision that can change on a dime, and I used to worry the supply could dry up. I am more optimistic now. Between consortiums, government and enterprise-backed efforts, and new labs stepping in, I expect a steady stream of capable open models, not a drought. Thinking Machines' Inkling makes the point: not the best model out there, but an open-weights base built for post-training and customization. You do not have to wait for the perfect open model to start cutting your dependence on a closed one.

That is the real challenge for OpenAI and Anthropic. In an earlier piece I argued the hybrid stack was coming for their pricing power, and this is the sharper version of that argument. Enterprises are now convinced of three things: they do not want vendor lock-in, they can offload a growing share of compute to open and smaller models they tune themselves, and they would rather not let their own data train someone else’s next model. Put it together and the days of a two-company oligopoly commanding most of the world’s inference are drawing to a close. Which may be why both are racing to go public, building a war chest before that pricing power erodes and pushing into domain-specific markets where an application is easier to defend than a token price. Smart move, and also a tell.

China's AI Export Controls at a Glance

Ben Lorica edits Ethics.dev and the Gradient Flow newsletter, and he hosts the Data Exchange podcast. He helps organize the AI Conference and the AI Agent Conference, while also serving as the Strategic Content Chair for AI at the Linux Foundation. You can follow him on Linkedin, X, Mastodon, Reddit, Bluesky, YouTube, or TikTok. This newsletter is produced by Gradient Flow.

Subscribe • Previous Issues

What Startups Taught Me About the Next Layer of AI Infrastructure

A little while back I wrote about how teams use reinforcement learning (RL) to make agents reliable. Since then I keep bumping into startups where RL is not a research footnote or a feature buried in the stack. It is central to what they are building. I know of more than 25 at last count and still climbing, and in some cases RL is basically the product. It is still early, and much of the current tooling is aimed at frontier labs, neo labs, and sophisticated AI teams. Still, there is enough signal to think that some of the tools and lessons will eventually trickle down to the rest of us. At this point I treat my own RL timelines the way I treat horoscopes, entertaining but not binding. What I will say instead is that this particular wave has startups, products, and customers around it, which the previous periods mostly did not.

The New Infrastructure for Learning by Doing

The common thread is that almost all of these startups are reacting to the same frustration. Frontier models are smart but unreliable the moment you ask them to do real, multi-step work. RL is the lever they pull to turn raw capability into dependable behavior, whether the target is a coding agent, a support bot, or a humanoid robot.

But none of them are really selling RL as a magic algorithm. They’re building the missing machinery around it, and that machinery has two hard parts. The first is giving a model somewhere to practice, which is why so many are in the business of simulated environments and training gyms. The second, and trickier, part is scoring. Did the agent finish the task, follow the rules, pick the right tool, avoid a shortcut that games the metric? A surprising amount of this market is really about graders, verifiers, and the tooling to catch an agent cheating its own reward.

From there the goals split. Some startups build RL environments for frontier labs. Some build post-training platforms for companies that want specialist models. Some focus on continuous learning from production traces. Others carry RL into robotics, industrial control, and logistics. The common thread is simple enough to state plainly. If you can define the task, create a safe place to practice, and score the result, RL has something to work with.

Where Learning by Doing Pays Off

The first commercial cluster is exactly where you would expect. Coding, software engineering, ML engineering, security, and infrastructure tasks show up again and again because they come with built-in ways to score progress. Code can compile or fail. Tests can pass or fail. A sandbox can reveal whether the agent’s action changed the system in the intended way. The feedback is still imperfect, but it is much cleaner than asking a human to grade every step from scratch.

The second big cluster is computer use and enterprise work. These are agents that operate browsers, spreadsheets, CRMs, ticketing systems, internal tools, finance software, documents, and knowledge bases. This is where the difference between a demo and a production agent becomes painfully obvious. A model that can explain a workflow is not the same as an agent that can execute it, recover from mistakes, and satisfy a company’s standards.

Then there’s a smaller but striking set of use cases that leave the screen entirely: robots learning to walk and grip, data-center cooling, freight routing. The wider pattern is that RL is showing up wherever teams can create a loop. There has to be a task, a place to attempt it, a way to observe what happened, and a score that is close enough to what the user actually cares about. That pattern applies to coding agents, enterprise research, support workflows, financial modeling, medical reasoning, industrial cooling, logistics, and robotics. The domains differ, but the recipe is surprisingly consistent.

When Does This Reach the Rest of Us

The question I keep coming back to is when this becomes usable by a broader set of AI teams. Right now, a lot of the tooling still assumes serious technical depth. You need to define environments, write rewards, inspect rollouts, catch reward hacking, and connect the training loop to real systems. That is not casual weekend work. But when this many startups are building environments, post-training platforms, verifiers, agent gyms, and RLOps tooling, it is hard not to expect some of the abstractions to become simpler.

In one sense, the trickle-down has already started. The startups helping companies post-train or customize foundation models are bringing pieces of RL tooling into enterprise settings today. That’s RL tooling reaching real businesses today, not in some hypothetical future. The next step is not that every AI team becomes an RL research lab. It is that more teams get practical tools for defining the work they care about, measuring success, and improving models against those signals over time.

The AI Bubble Risk Map

Ben Lorica edits Ethics.dev and the Gradient Flow newsletter, and he hosts the Data Exchange podcast. He helps organize the AI Conference and the AI Agent Conference, while also serving as the Strategic Content Chair for AI at the Linux Foundation. You can follow him on Linkedin, X, Mastodon, Reddit, Bluesky, YouTube, or TikTok. This newsletter is produced by Gradient Flow.

Subscribe • Previous Issues

Your CLI Was Built for Humans, Not Agents

There’s a friendly debate among developers about how to give AI agents reliable ways to use external tools, data, and services so they can do useful work beyond generating text. One side favors CLIs, or Command-Line Interfaces, where agents run text commands against tools like git, aws, jq, duckdb and internal scripts. The other side favors MCP, or Model Context Protocol, which gives agents a structured way to discover tools, authenticate, and call services through defined schemas. This may seem confined to coding agent, but the tradeoffs apply equally to anyone deploying agents in operations, finance, customer success, or any workflow where agents need to touch real systems. Once an agent starts taking action, its interface with the outside world is no longer just a technical detail. It becomes a critical decision point for security, cost, and governance, regardless of whether that agent is writing code or managing customer data.

CLI: Fast, Familiar, and Sharp-Edged

The strongest case for CLI is that it is familiar, flexible, and cheap. First, AI models have been trained on enormous quantities of shell scripts, developer documentation, and terminal content, which means they already understand tools like git, docker, and kubectl, without any special configuration, and that familiarity translates directly into reliability and lower error rates. Second, CLI output is composable: an agent can pipe the output of one command directly into another to filter, aggregate, or reshape data before it enters the model’s working memory. Finally, the token efficiency advantage is real and measurable. Loading a single MCP server’s full tool schema can consume tens of thousands of tokens before any useful work begins, whereas CLI tools require zero upfront context cost and agents can discover a tool’s capabilities on demand using its built-in --help flag. The catch is that CLI access can be too powerful. Giving an agent raw shell access without sandboxing, credential isolation, and approval gates makes it incredibly difficult to prevent unauthorized changes to files and networks.

The MCP Case for Agents

MCP’s advocates have a direct response to the CLI-first view: CLI doesn’t account for how most AI is actually consumed in the world. The majority of people interacting with AI agents do so through browser-based chat interfaces or mobile apps where running shell commands is simply not an option, and many business tools have no CLI equivalent at all. For those contexts, MCP fills a genuine gap: it gives agents a self-describing interface where the agent discovers which tools are available and how to use them upon connecting, without needing to guess command syntax or parse documentation. Unlike CLI, MCP includes built-in OAuth authentication, per-user permissions, and audit trails that satisfy enterprise security requirements.

The main criticism is overhead. Poor MCP implementations can load huge tool schemas before useful work begins, burning tokens, adding latency, and leaving less room for reasoning. MCP can also introduce server failures, authentication friction, and harder debugging. The practical lesson is that MCP needs good interface design: smaller task-level tools, progressive disclosure, schema filtering, and gateways that show agents only what they need.

Your CLI Was Built for Humans, Not Agents

An agent can learn command syntax. The harder problem is that most CLIs were designed for humans, not software agents operating under time, token, and safety constraints. We easily overlook these flaws because humans are great at filling in the blanks. We can guess missing steps from bad manuals, decode vague error messages, and realize on our own when a tool requires us to log in before showing its options. An agent pays a concrete cost for every one of those gaps: repeated discovery loops, wasted tokens, fragile text parsing, and unnecessary interruptions asking the user what to do next. The CLI didn’t change. The user did.

The practical response is to treat CLI agent-readiness as something you can measure and improve, not just assume. An open-source project called CLIARE (CLI Agent Readiness Evaluation) does exactly this. It probes the actual installed binary, not your documentation, records what happens, and produces a scored report across six dimensions: whether commands are findable and navigable, whether valid invocations can be constructed, whether commands complete without hanging, whether bad inputs fail with clear errors, whether machine-readable output modes are available and actually work, and whether read-only probes stay read-only. The result is a scorecard you can run, track across releases, and share with the teams building agents on top of your tooling. That shifts interface quality from a guessing game into a measurable metric.

This matters well beyond coding agents. Companies can expose real operational authority through CLIs: deployments, finance workflows, database jobs, cloud control planes, support scripts, analytics pipelines, and internal platforms. If agents are going to touch those surfaces, the relevant question is not just “does a CLI exist?” It is “can the agent understand and use it without guessing?” AI teams exposing CLIs to agents should try CLIARE, or at least borrow its core idea: make the command surface measurable, versioned, parseable, and explicit about preconditions, outputs, side effects, and risk.

Match the Interface to the Risk

I would not treat this as a religious choice between CLI and MCP. The better question is what kind of authority the agent is being given, and on whose behalf it is acting. Use CLI when the workflow is local, composable, inspectable, and already lives close to the terminal: code, tests, logs, files, data inspection, build systems, mature vendor tools, and internal scripts. But if agents will use that CLI repeatedly, use CLIARE to measure the surface before relying on it.

Use MCP when the workflow is hosted, multi-user, permissioned, or tied to SaaS systems where OAuth, consent, scoped access, and audit trails matter. In both cases, raise the bar when the agent can touch money, identity, production systems, customer data, or irreversible state. The common design rule is simple: expose the task, not the whole system. Keep the interface small, prefer parseable outputs, separate reads from writes, log consequential actions, and require approvals for operations that cannot be easily undone. Transport choice matters. Interface design matters more.

The AI Developer Productivity Debate: A Field Guide

Ben Lorica edits Ethics.dev and the Gradient Flow newsletter, and he hosts the Data Exchange podcast. He helps organize the AI Conference and the AI Agent Conference, while also serving as the Strategic Content Chair for AI at the Linux Foundation. You can follow him on Linkedin, X, Mastodon, Reddit, Bluesky, YouTube, or TikTok. This newsletter is produced by Gradient Flow.

Subscribe • Previous Issues

Agents Need Maps, Not Bigger Context Windows

Like everyone else, I’ve been enjoying the steady improvement in coding agents and the tooling around them, from frameworks and harnesses to evaluation suites. But the more I talk with teams actually deploying agents in enterprises, the more I circle back to plumbing. Agents need data integrations and infrastructure built for them, a theme I explored in a piece on rethinking databases for agents. I’ve also written about using knowledge graphs (GraphRAG) to improve accuracy and reduce hallucinations, which is especially important in high-stakes applications. The persistent problem has been that most teams don’t have a knowledge graph, and the few that do struggle to keep it current. A new system I recently came across is aimed squarely at that gap.

Automating Graph Construction and Maintenance

I recently spoke with Andrew Moore, CEO of Lovelace, former head of Google Cloud AI, and former dean of Carnegie Mellon’s School of Computer Science. Lovelace’s Elemental platform turns enterprise data into structured context engines that AI agents can navigate and investigate. I think “knowledge graph” is still a reasonable term here, although “context graph” may be closer to the point. This is not a graph built mainly for a human analyst browsing nodes on a screen. It is designed for agents that need to answer questions too large, messy, or time-sensitive for ordinary search, basic RAG, or stuffing more raw documents into a context window.

Elemental ingests whatever you have: relational tables, PDF files, news feeds, JSON fragments, even satellite imagery and other multimodal sources. Everything gets converted into a common intermediate form that eventually becomes nodes and relationships in a graph. The platform automates much of the construction and maintenance of these graphs. A graph with a thousand nodes can tolerate a human in the loop, but once you’re at many millions of facts, any step that depends on a person checking whether two records refer to the same company simply collapses. Too many enterprise AI projects die in the six months of data wrangling that comes before any actual AI work, and removing that toil is the whole point.

For an agent, a context graph gives it a map of the enterprise and the surrounding world. A bank investigating exposure to a sanctioned shipping company might need to connect ownership records, trade data, vessel movements, subsidiaries, legal filings, customer relationships, and recent news. Vector search can find relevant documents, but these questions often require following relationships across many entities and events. That is where a graph-shaped context layer becomes useful.

Graphs for Agents, Without a Graph Database

Elemental is designed around four steps: data ingestion, entity resolution, graph construction, and agent-facing tool access. I covered ingestion above, but one constraint is worth adding: the system has to work without assuming clean schemas, consistent naming, or a single source of truth.

Entity resolution is the problem of deciding when different records refer to the same real-world thing. “IBM,” “International Business Machines,” an internal vendor ID, a legal entity in a filing, and a name in a PDF might all refer to the same company, or they might not. The same issue appears with people, ships, aircraft, accounts, subsidiaries, addresses, products, assets, and events. Anyone can prototype an entity resolution system in a weekend. Making one that scales, handles multiple languages, adapts to entirely new entity types, and updates as new information arrives is brutally hard. Lovelace built its own engine and goes well beyond text matching. I half-jokingly told Andrew that the entity resolution engine alone could be a standalone product, given how many applications live or die on getting this right.

Auditability is just as important. In regulated or high-stakes settings, the system needs to show where a fact came from, how it was linked to other facts, and what version of the graph existed when an agent produced an answer. In plain English, that means lineage, provenance, and version control: how a fact moved through the system, what evidence supports it, and what the graph believed at a point in time. That matters if a regulator, customer, or internal reviewer asks why an agent flagged a transaction, recommended an investigation, or dismissed an alert. “The model said so” will not be enough.

Retrieval is where the graph becomes more than a data catalog. In a serious graph, a single node can have hundreds of thousands of links, so a naive “show me everything within two hops” query explodes into billions of paths and grinds to a halt. Elemental takes a different route. It reframes a question like “who is most affected by this event?” as a probability problem, then uses randomized algorithms and other graph-theory tricks to surface the strongest connections without walking every path. The result is approximate rather than mathematically exact, but it returns confident, ranked answers over millions of nodes in under a second. That trade is well worth making, because the alternative is an LLM trying to reconcile contradictory probabilistic evidence in its head, which language models are genuinely bad at.

Another interesting design choice is that Elemental does not rely on a graph database as the core storage layer. The data is exposed to agents as a graph, but the persistence layer underneath is custom. The reason is simple. Traditional graph databases are very useful, but they were not built for this particular job: broad, high-speed inference across tens of millions of nodes.

Why Private Graphs Need Public Context

Lovelace also brings another asset to the table: YottaGraph. Think of it as a large, continuously updated graph of the public world, covering companies, people, places, events, filings, sanctions records, and the movements of ships and aircraft. This is useful because internal enterprise data rarely tells the whole story. YottaGraph would be hard to replicate because it requires continuous ingestion from many sources, entity resolution at scale, and a lot of operational discipline. Importantly, the enrichment flows inward. Public context can augment a company’s private graph, but Lovelace does not share one customer’s data with other organizations.

Lovelace isn’t alone in using graphs to give agents better context, and I expect to see much more activity here. There is also a funny twist. GraphRAG was promoted heavily by graph database vendors, but some of the more interesting graph systems for agentic AI may not use graph databases at all. Whatever we end up calling these things, knowledge graphs, context graphs, or context engines, the underlying idea of structured, auditable context for agents is becoming hard to ignore.

OckBench: The Cost of Getting the Right Answer

Ben Lorica edits Ethics.dev and the Gradient Flow newsletter, and he hosts the Data Exchange podcast. He helps organize the AI Conference and the AI Agent Conference, while also serving as the Strategic Content Chair for AI at the Linux Foundation. You can follow him on Linkedin, X, Mastodon, Reddit, Bluesky, YouTube, or TikTok. This newsletter is produced by Gradient Flow.

Subscribe • Previous Issues

The Data Compliance Problem AI Teams Keep Ignoring

I’ve avoided writing about copyright and AI. Not because it isn’t important, but because it felt like a legal sideshow compared to the engineering and business questions I find more interesting. That’s gotten harder to justify. There is also something mildly funny about naming my podcast The Data Exchange years ago, because I believed data markets would become central to technology, and then mostly ducking this corner of the data conversation. The data question is now central to how AI systems are built, how they compete, and how they create risk. You can’t talk seriously about any of that without eventually bumping into who owns the stuff these models are trained on.

For AI teams, data is no longer just raw material. One recent economics paper estimated that the value of data missing from official GDP figures amounts to about 4% of GDP on average, and closer to 6% in more recent years. More interesting for companies is the relative value of their own operational data: transaction logs, usage records, support tickets, product telemetry, and customer interactions. That internal data can be more informative than public web data because it captures how a real business actually works. For AI builders, the practical implication is simple: the data sitting inside your company may be more strategically valuable than anything you could scrape from the open web.

The Data Supply Chain Problem

The first practical issue is provenance: knowing where training data came from, what rights apply, and whether it can be inspected or reconstructed later. That used to sound like paperwork. It now looks like core infrastructure. What a model knows, where it performs well, where it breaks down, and what legal exposure it carries are all shaped by the data it was trained on. Pre-training is the broad phase where a model absorbs patterns from large collections of text, images, code, books, journalism, and other sources. Post-training is where you steer the model toward specific tasks, behaviors, and domains using narrower, more curated datasets. Both phases are data strategy questions as much as engineering questions. For most teams, the immediate action is not to audit a trillion-token pre-training corpus. It is to get serious about the smaller, higher-value datasets they use to adapt, evaluate, and deploy models.

Memorization adds a second layer of risk. A model does not need to act like a database to create trouble. Under the right prompts, especially when material appears many times in training data, it may reproduce protected text, code, lyrics, images, or paywalled content too closely. That makes copyright risk a lifecycle issue, not just a training-data issue. Hosted models give providers more room to monitor, filter, and rate-limit suspicious behavior. Released model weights are harder to control. Either way, teams need to test for verbatim reproduction, near-duplicates, and outputs that could substitute for the original work.

Guarding Your Post-Training and Fine-Tuning Pipelines

Most AI teams aren’t doing pre-training. They’re taking a foundation model and adapting it: fine-tuning on domain data, building instruction datasets, running human feedback loops, assembling evaluation suites, or wiring up retrieval pipelines. That’s where the real data work happens for the majority of developers, and it comes with its own provenance problems. The data you use to customize a model, customer support logs, internal documents, expert annotations, licensed content, third-party datasets, needs the same documentation discipline as any other part of the stack. Where did it come from? What rights apply? Was it collected with appropriate consent? Can you reproduce it if challenged? These questions matter whether you’re fine-tuning a vertical copilot or building an evaluation harness, and the answers are often murkier than teams expect. A lightweight data bill of materials can help: a record of each dataset, its source, license terms, consent status, known restrictions, and intended use.

There’s also a technical risk that’s easy to underestimate in post-training contexts. Fine-tuned models can memorize training examples more readily than large pre-trained models, partly because the datasets are smaller and examples may repeat. A customer-facing model fine-tuned on proprietary documents, licensed text, or third-party content could reproduce that material verbatim in ways that create real liability. That risk doesn’t disappear after training ends. Output monitoring matters: similarity detection, content flagging, and policies for handling prompts that look like attempts to extract protected material should be part of any production deployment. No single control handles all of it, and the teams most likely to get caught flat-footed are the ones who assumed that using someone else’s base model meant the data compliance problem was already solved.

Who Owns the Fuel That Runs These Models

We are also seeing the rise of a formal data licensing market, where major publishers and content platforms sign deals with AI labs. While this reduces legal uncertainty for those who can afford it, it also threatens to create a massive moat for wealthy incumbents. Large AI firms can afford broad or exclusive deals that smaller labs, startups, researchers, and open-source projects cannot.

The legal landscape is also evolving unevenly around the world. The U.S. still relies heavily on case-by-case fair use arguments. Europe is pushing more toward transparency, opt-outs, documentation, and compliance processes. Some Asia-Pacific jurisdictions appear more permissive for computational analysis, but global deployment complicates everything. A model trained in one place can be used, challenged, or regulated somewhere else. In practice, many companies will opt to satisfy the strictest plausible standard, especially if they sell to large enterprises or regulated industries.

This is why AI and copyright no longer feel like a legal sideshow to me. They are part of a larger question about how data markets will actually work: who gets paid, who gets access, who can prove what they used, and who can afford the cleanest inputs. The industry still has to find a workable balance between adaptation and attribution. Data is no longer just exhaust from the internet or residue from business operations. It is becoming one of the main assets, bottlenecks, and fault lines in AI. The teams treating it as a compliance checkbox are going to find that out the hard way.

The Runtime Pattern Behind AI Agents

Claude Code to Cursor: a poem

Ben Lorica edits Ethics.dev and the Gradient Flow newsletter, and he hosts the Data Exchange podcast. He helps organize the AI Conference and the AI Agent Conference, while also serving as the Strategic Content Chair for AI at the Linux Foundation. You can follow him on Linkedin, X, Mastodon, Reddit, Bluesky, YouTube, or TikTok. This newsletter is produced by Gradient Flow.