Associated with this newsletter, I posted my video on Youtube (Note it is Chinese). Welcome to watch it:

Read more

Associated with this newsletter, I posted my video on Youtube (Note it is Chinese). Welcome to watch it:

Read more

Associated with this newsletter, I posted my video on Youtube (Note it is Chinese). Welcome to watch it:

Read more

Welcome to my newsletter. Here is ZeroFutureTech。I will share what I learned, researched and did in AI-related domain. Welcome to follow my X.

Associated with this newsletter, I posted my video on Youtube (Note it is Chinese). Welcome to watch it.

We have learned to think of AI as a chat partner that writes, explains, and answers questions. But many software workflows do not need a polished response. They need a decision: Which team owns this email? Is the customer asking for a refund? Should this alert be escalated now? In many cases, the decision model can make that first call, then hand selected cases to a conventional LLM when a written explanation or reply is actually needed.

Jev and Laya explore a different interface. Define the question and its allowed answers first; then have the model return a judgment the application can use directly. The appeal is not simply that the answer is shorter. It is that software can use the result to route the work immediately—and call a generative model only for the cases that need language, explanation, or broader reasoning.

That does not make conventional large language models (LLMs) obsolete. Writing, explanation, open-ended questions, and multi-step reasoning remain generative models’ strengths. The more useful question is: does every AI call in an application need a model that can write an essay?

Start with what the application receives

A conventional LLM usually starts with text and a prompt, predicts the next token one step at a time, and produces a string. You can ask for JSON:

{"team": "billing", "urgent": true}

The application still has work to do: parse the JSON, check that fields are present, confirm team is allowed, and decide whether the semantic judgment is trustworthy. Even valid JSON can contain a wrong value. Function calling or JSON mode can improve the output format; they do not automatically eliminate mistakes in meaning.

A typed decision model takes a state—the context needed to make a judgment—and questions whose answer spaces are defined in advance. A department might be limited to billing, technical, and sales; urgency might be scored as “can wait,” “this week,” or “today.” The model returns a typed result such as Choice, Score, or Noul, which the application can pass into its next rule.

The key difference in the diagram is not which model is “smarter.” It is which output contract fits the job. A conventional LLM has a broad answer space: it can explain, expand, and handle questions nobody anticipated. The trade-off is that the application must parse the text and check its content. Typed decisions narrow the answer space, making results easier for software to consume. The trade-off is that they are not designed for free-form writing or open-ended exploration.

TaskNatural fitWhyWrite a reply, explain a policy, summarize a long documentGenerative LLMThe output needs natural language, not just one labelRoute, score, or classify within a fixed setJev / Laya and similar typed modelsThe software needs explicit fields and candidate valuesScreen first, explain laterA combinationA decision model filters; a generative model handles the few complex cases

A useful pattern: decide first, generate second

A support system can first ask a typed model to identify the intent, urgency, and relevant team. Straightforward cases go to the right queue. If the customer needs a tailored explanation, or if the issue depends on several policy details, the application can pass the relevant context to a conventional LLM to draft a reply. Uncertain or high-impact cases can go to a person instead.

This is a routing pattern, not a promise that every combined system will be cheaper or faster. The benefit depends on how many cases can be resolved at the decision stage, how often the larger model is still called, and the cost of human review. Measure the whole workflow, including both model calls and review time.

What do Choice, Score, and Noul mean?

The names sound specialized, but they map to familiar business questions:

• Choice picks from a defined set. For example, should a ticket go to billing, technical support, or sales?

• Score rates something on a defined scale. For example, is the customer calm, frustrated, or very angry?

• Noul estimates the probability that a statement is true. For example, “Did the customer request a refund?”

One request can include several independent questions about the same state. The application does not need to ask the model to write a long answer, then extract intent, sentiment, and risk from it. But “several questions can be asked together” is different from “the model reasons from each answer into the next.” The research summary says Laya handles multiple questions in one forward pass. That does not mean every model uses the same mechanism, or that well-written questions guarantee correct answers.

Inside Laya: how text becomes a typed answer

Laya’s public model card describes a ModernBERT-large encoder and a decision head. It helps to picture these as a pipeline from context to candidate answers:

1. The input is a state plus a question definition

A state can be an email, support ticket, or business record in JSON. Context quality matters as much as the question: if the model sees only “refund this” but not the transaction, relevant policy, or earlier messages, it has little basis for a useful judgment. Missing context is not something a decision head can guess reliably. The developer also defines what each question asks, which answers are allowed, and what a score scale means. The candidate set is part of the business interface. If a ticket could be about billing, technical support, sales, or something else, leaving out “other” for convenience silently excludes a real possibility.

2. ModernBERT encodes the context

BERT-style encoders read context in both directions: when processing a word, they can use the words before and after it. Given “I don’t want a refund; I just want to know why I was charged again,” a model can consider both the negation and the explanation, rather than treating one keyword as the whole conclusion. The encoder compresses the text into contextual representations for the decision head.

This differs from the main path in a common autoregressive generator, which predicts the next token and continues until it has produced a string. Laya’s model card describes a non-autoregressive decision model; it does not need to generate free-form text token by token. This is a description of architecture and output format, not a claim that every BERT model is automatically more accurate or faster.

3. The decision head scores the allowed answers

The decision head uses the encoded representation to score the allowed answers for each question. For “Which team owns this?” it can compare billing, technical, and sales. For “Did the customer request a refund?” the answer space might be yes and no. The summary says Laya can handle multiple questions in one forward pass, which can put a set of related judgments into one model call.

4. Softmax turns scores into a probability distribution

For a multiple-choice question, the candidate scores can be converted by softmax into probabilities that sum to one. That normalization is a mathematical step; it does not by itself make the probabilities calibrated. A score of 0.7 describes the model’s relative assessment of the options. It does not automatically mean the model will be right 70% of the time on your company’s data. Calibration requires a separate check against labeled examples.

The application then decides what happens: automatically route a high-confidence, low-risk ticket; ask a follow-up or send it to a person if the signal is uncertain; and check transaction records, policies, and permissions before any action involving refunds, payments, or access changes.

Jev and Laya: similar goals, different deployment paths

Both systems make decisions for defined questions, and both publicly refer to RLCD (Reinforcement Learning for Calibrated Decisions). The practical distinction a team encounters first is how each model is delivered:

The distinction is not “Jev has RLCD and Laya does not.” Nor should we infer identical internals just because both can produce several decisions. TypeSafe’s public description mentions a parallel sampler for Jev; Laya’s model card describes non-autoregressive processing of multiple questions in one forward pass. The available research summary does not establish that the two use the same sampling or computation mechanism.

RLCD tells us the target, not the complete training recipe

TypeSafe describes RLCD as training for decisions with calibrated probabilities, rather than optimizing for text that people prefer to read. Calibration can be explained this way: across a large group of predictions assigned confidence around 0.8, roughly 80% should be correct over time.

That is not quite the same as saying “this particular answer with a score of 0.8 has an 80% chance of being right.” An individual prediction can be wrong. Calibration can also change with the customer population, language, workflow, or candidate options. Probability is a measurable signal, not a product guarantee.

The research summary also notes that public materials do not disclose a complete, reproducible RLCD recipe: details such as the exact reward function, training data, feedback sources, policy-optimization algorithm, and calibration loss are not specified. We can explain the stated training objective; we should not present undisclosed mechanics as established fact.

Put it into a duplicate-charge ticket

A customer writes: “I was charged twice. Refund the extra payment today or I’m cancelling my subscription.” The application can use the message and account context as its state, then define four questions:

1. Which team owns the issue? Options: billing, technical, sales, other.

2. Did the customer request a refund? Yes, no, or unclear.

3. How urgent is it? Rate it as can wait, this week, or today.

4. Is there a cancellation or churn risk? Yes or no.

The program—not the model—then applies the business rules. If billing is the strongest department prediction and urgency is high, move the ticket up the queue. If the refund request is unclear, ask the customer to confirm. If the model is uncertain, route it to a person. Even a clear “yes” to the refund question still requires checking the transaction, refund policy, and approval permissions.

A team might use a top-choice probability above 0.5 as an initial review rule for a three-way routing decision: below that, none of the options is a strong enough signal, so a person checks it. Treat 0.5 as an example to test, not a universal threshold. The right cutoff depends on the model’s calibration, the available options, and the cost of sending a case to the wrong place. A high-stakes action may need a much stricter policy—or no automatic execution at all.

That is the boundary of typed decisions: they help software read useful signals from language. They do not own authority or replace verifiable business facts.

How to evaluate whether the probabilities and accuracy are useful

Start with a set of historical records and have people label the right outcomes. Keep question wording and candidate options fixed. Record the answer, every candidate probability, latency, and any failure or fallback. Then measure at least:

• Classification accuracy: overall and by category. Frequent categories should not hide errors in a less common but important one.

• Calibration: group predictions by confidence range and compare the average probability with the actual accuracy in each group.

• Human-review rate: after low-confidence cases are escalated, is the remaining workload manageable?

• Cost of errors: what does a missed urgent complaint, a misrouted ticket, or an incorrect refund cost?

• Workflow impact: did latency, handling cost, backlog, or customer wait time actually improve?

Set thresholds according to the cost of errors. Ticket routing may tolerate a lower bar than a money movement. Decisions involving funds, compliance, or permissions deserve additional checks. At first, use the model to recommend rather than execute. Expand automation only after historical evaluation and live monitoring support it.

What my small routing experiment did—and did not—show

I built a simple script to explore model routing: given a task, choose which model from a candidate list should handle it. I tried three kinds of work—writing a newsletter, designing a TypeSafe introduction page, and designing a complex multi-agent system—and compared the Jev API, DeepSeek-Flash through an API, and Laya running locally. I looked at elapsed time, repeatability for the same input and candidate set, and what changed when I changed the candidate list. I did not measure whether the selected model produced the best work.

Across those three tasks, average elapsed time was about 0.537 seconds for Jev and 4.621 seconds for DeepSeek-Flash. Laya’s warm local calls averaged about 0.225 seconds; its first call, including model loading, took about 6.453 seconds. Those figures describe this setup, not a general speed ranking. API latency varies with network and server conditions, and a warm local call is not comparable to a cold start.

I then fixed one task—designing a TypeSafe introduction page—and ran three rounds for each of three candidate lists containing seven, six, and five models. With each list held constant, Jev picked the same model in all three rounds for all three lists (nine of nine calls). Laya did the same. DeepSeek-Flash matched the repeated choice in eight of nine calls, with one different choice in the five-model list. In this small run, Jev and Laya were more repeatable within a fixed candidate set.

But repeatability is not the same as invariance to the options. When I changed which models were available, Jev and Laya each selected two different models across the three lists; DeepSeek-Flash selected three. Removing one candidate or adding another changes the choice environment and can change the decision. Three repeats per list are far too few to establish long-term reliability.

Here is the selection pattern for the fixed webpage-design task. Each entry summarizes three consecutive runs; the one arrow marks the only within-list variation:

The names are the model choices returned by the script, not a human evaluation of which model was actually best. In this small experiment, Jev and Laya repeated their choice within each fixed list, then changed their choice when the list changed. That is useful evidence about repeatability and candidate sensitivity; it is not evidence of routing accuracy.

The experiment did not record prices, a ground-truth “right” model, full probability distributions, or downstream task quality. So it cannot tell us which router is most accurate, capable, or cost-effective. A useful follow-up would preserve every raw selection and probability, track the candidate set and fallback events, and have people rate the work produced by the selected model. Only then could latency be weighed against quality and cost.

Choosing the interface for the task—and validating it with real data

Jev and Laya make model calls into explicit decision interfaces: give the model a state and questions, and it chooses from a set of candidate answers. Jev is closer to a hosted API; Laya’s open weights make local deployment and task-specific adaptation more accessible. Conventional LLMs remain a better fit when a task calls for long-form text, explanation, writing, or open-ended reasoning.

Karpathy’s comment can be read as a product question: if many software systems need only one good-enough judgment, would a team trade some open-ended generative ability for lower latency and shorter outputs? The answer depends on the real workflow, not just the model page.

When the task is frequent, the answer space is clearly defined, and errors can be reviewed or rolled back, typed decisions are worth considering. Support-intent routing, retrieval relevance, invoice triage, and alert prioritization all fit that profile.

For work that requires explaining complex policies, synthesizing evidence across multiple steps, or writing a tactful response, a generative LLM remains a more natural fit. Many systems can combine them: use Jev or Laya to screen a large volume quickly, then send a smaller number of complex or low-confidence cases to a stronger generative model or a person.

Conventional LLMs are good at making things clear in language; Jev and Laya are designed to give software a predefined judgment it can consume directly. A constrained output shape does not guarantee a correct answer. Measure decision quality in your own workflow and against your own cost of errors.

References

• TypeSafe: Introducing System One Models & Jev

• Laya model card on Hugging Face

• Andrej Karpathy on X

Associated with this newsletter, I posted my video on Youtube (Note it is Chinese). Welcome to watch it.

Most teams can build an agent demo in an afternoon. The model gets a prompt, calls a few tools, and returns something plausible.

Production is where the easy story ends.

An agent that investigates an outage, works through a codebase, or processes a queue for hours has to carry state forward. It has to find the right tool among a large internal catalog, recover after partial failure, keep credentials and files in the right environment, and avoid asking a model to do work that ordinary code could perform more cheaply and reliably.

That collection of concerns is the agent harness. OpenAI’s new Agents API puts that harness—the one behind Codex—behind a managed API.

The important shift is simple: OpenAI is no longer just offering the engine. It is offering more of the vehicle.

What the Agents API actually provides

The basic API shape is deliberately compact. A developer specifies a task, model, tools, knowledge resources, and an execution environment. OpenAI operates the loop that coordinates model calls, tool use, context, and subagents.

That does not make application design disappear. It moves a particular layer of it. Instead of every team independently maintaining prompt loops, tool-selection glue, context summaries, and orchestration code, they can start with an evolving managed harness.

This is why “another API” is the wrong mental model. A model API gives you inference. The Agents API gives you a runtime for an agent that can continue working across a complex task.

The distinction matters because a capable model is necessary but insufficient. A model can reason about a deployment failure. A working agent also needs access to logs, metrics, files, a safe place to run code, a record of what it has already checked, and a way to divide independent work without losing the thread.

The three layers worth separating

It helps to draw a sharp line between three things that are often thrown together under the word “agent.”

LayerJobExample responsibilityModelReasoning and generationDecide which hypothesis best explains an incidentHarnessCoordination and continuityMaintain context, select tools, delegate work, retrySandboxExecutionRun code, store files, install packages, access compute

OpenAI hosts the harness. Developers can choose where the agent executes: OpenAI-hosted sandboxes, their own infrastructure, or supported ecosystem providers. The separation is more than architectural neatness.

For an enterprise, it can mean keeping data locality, network controls, secrets, and execution policy inside its own boundary while delegating the general-purpose mechanics of agent coordination. The harness handles how work proceeds; the environment determines where work is allowed to happen.

Long-running agents need context management, not a longer prompt

Context is the first place a real agent system becomes messy.

As a session investigates, reads files, calls tools, writes outputs, and revises its plan, its working history grows. Eventually it approaches the model’s context limit. At that point the system has to decide what to retain, what to summarize, and how to restore the important state on the other side of a context window.

Teams have built their own summary and checkpoint systems for this problem for years. The Agents API introduces context compaction: it automatically compresses earlier context as a session nears the limit, preserving information needed to continue.

That does not remove the need for good state design. Critical business facts, approvals, and durable artifacts should still live in systems of record, not only in a generated summary. But it makes a common piece of long-running-agent plumbing a platform concern rather than a bespoke project in every codebase.

The practical outcome is important: an agent can span multiple context windows without the application author hand-rolling compaction logic before they can even test the workflow.

Tool search is a response to the enterprise tool catalog

Tool calling was never the whole problem. Tool discovery is.

An agent with five tools can see every schema in its context. An enterprise agent may face hundreds or thousands of actions distributed across ERP, CRM, observability, finance, knowledge systems, and MCP servers. Stuffing every definition into the prompt wastes tokens, makes selection noisier, and reduces the useful room left for the task itself.

Tool search loads relevant tool definitions only when they are needed. For a spike in 5xx errors, the agent might surface log search, deployment history, and metrics tools first—not the entire company catalog.

This changes the design question from “Can our agent call tools?” to “Can it find the right safe tool, with the right scope, at the moment it needs it?” That is a much more useful question for enterprise AI.

Use code for data movement; reserve the model for judgment

The second efficiency lever is programmatic tool calling. Rather than forcing a model through a serial pattern—reason, call a tool, reason again, call another tool—the agent can call tools in parallel, chain related operations, and filter or combine results in code before returning the useful subset to the model.

Consider 100,000 log lines. Asking a model to inspect all of them is a poor system design. Filter by time window, group error signatures, correlate them with deployments, and return the twenty anomalies that deserve interpretation. The model then does the work it is good at: weighing competing explanations and proposing a next move.

The operating rule is straightforward:

• Let the model make judgments and trade-offs.

• Let code filter, calculate, aggregate, and move data.

• Let deterministic tools execute governed business actions.

Besides cost and latency, this division makes systems easier to inspect. A human can understand a query, a filter, and an approval policy far more easily than an opaque chain of model turns.

Subagents are becoming a runtime primitive

The API also provides native multi-agent support. A primary agent can split an incident investigation into independent tracks: one agent checks recent deployments, another examines error logs, and a third investigates dependencies. Each keeps its own context; the primary agent brings the evidence together.

Frameworks such as LangGraph and AutoGen made this style of orchestration familiar. The significance here is not that every workflow should now use a swarm. Splitting work creates coordination overhead, and poorly scoped subagents can multiply confusion. The point is that subagent coordination is moving from framework-level assembly toward a managed runtime feature.

Use it when tasks are genuinely separable, results can be checked, and the fan-out has a clear budget. Do not use it as decoration.

Does this make agent frameworks obsolete?

No. It does, however, move the boundary of what is worth building yourself.

Generic agent infrastructure is steadily becoming part of model platforms: tool calling, web and file search, computer use, context management, long-running sessions, tool discovery, subagent orchestration, and sandbox integration. The return on recreating every one of those primitives will shrink for many teams.

The work that remains stubbornly application-specific is more consequential:

• the business objects and vocabulary the agent must understand;

• the quality, ownership, and freshness of the data it can use;

• the actions it may take and the permissions around them;

• approval gates, audit trails, evaluation, and recovery paths;

• the product experience for the person who has to trust the result.

That is where durable differentiation lives. Not in having a more elaborate loop around a model, but in encoding a company’s real operating context well enough that the agent can act safely within it.

The architectural question to ask now

If you are evaluating the Agents API, start with three questions.

First: is your workflow long, tool-heavy, or parallel enough to benefit from a managed harness? A short, deterministic workflow may still be better as ordinary software plus a single model call.

Second: are your tools ready for agent use? A tool catalog needs clear descriptions, narrow permissions, useful error responses, and an explicit approval model. Exposing an internal API is not the same as making it agent-ready.

Third: what remains uniquely yours after the runtime is managed? The answer should be your data, business semantics, policies, and user experience—not merely a custom orchestration diagram.

OpenAI’s Agents API is currently in public beta. Its deeper signal is already visible: the harness behind an agent is becoming infrastructure. Models remain the engine, but the harness is what keeps the vehicle on the road for the next hundred miles. The destination—and the guardrails—are still the application’s job.

Key terms: OpenAI Agents API, Codex harness, agent harness, long-running agents, context compaction, tool search, programmatic tool calling, multi-agent systems, subagents, sandbox, MCP, enterprise AI agents.

Sources and further reading: OpenAI: Introducing the Agents API · Agents API documentation · Codex harness source code

Associated with this newsletter, I posted my video on Youtube (Note it is Chinese). Welcome to watch it.

If your team is moving from AI chat to coding agents, research agents, or business automation, the question is no longer just whether an AI system can finish the task. The more important question is: where do the prompts, code, tool calls, and execution traces created along the way actually go?

In its September 2026 report, Detecting and countering misuse of AI, Anthropic uses illicit distillation to describe the large-scale, unauthorized, and covert extraction of frontier-model capabilities. The report’s company-specific attributions and figures are Anthropic’s allegations and investigation findings; they are not admissions by the named companies or independently verified conclusions. Still, the broader shift is clear: valuable AI assets now extend beyond model weights and training corpora to model behavior and full agent trajectories.

The short answer

AI security must increasingly protect model behavior, agent trajectories, and user data moving through routing, integration, and reseller layers—not model weights alone.

An agent trajectory is the full record of task completion: reading code, choosing tools, editing files, running tests, observing failures, and iterating. The final answer is the outcome. The trajectory is the reusable training and evaluation asset.

What does the Anthropic report cover?

This is not a report about distillation alone. Anthropic says its September 2026 report brings together cases it identified and disrupted between December 2025 and August 2026 across seven harm areas: cyber operations, influence operations, surveillance, scams and fraud, biological misuse, conventional-weapons development, and illicit distillation.

The shared theme is the shift from AI as an assistant that offers advice to AI as an agent that can run a task chain. In a conventional copilot interaction, a user asks a question and receives an answer. In an agentic interaction, the user supplies an objective and the system can break down work, call tools, inspect results, revise a plan after failure, and continue.

That capability increase does not automatically create harm, but it changes the unit of risk. Instead of evaluating one response, teams may need to evaluate a continuously running workflow that can access tools and data and adapt its behavior over time.

Anthropic notes that the cases it publishes are not intended as typical usage; they are examples of notable or novel activity it observed. The company says it responded to policy-violating activity by banning associated accounts, strengthening monitoring and detection, and, where appropriate, sharing intelligence with industry partners and authorities.

Why process data is more valuable than an answer

Conventional Q&A data captures a prompt and a response. A coding agent captures the path between them: how it understood the repository, narrowed the problem, chose an action, ran validation, and recovered from error.

Data typeWhat it capturesValue for training agentsFinal answerThe conclusionLimited; no execution pathReasoning and code stepsWhy a decision was madeHigher; reveals decision patternsFull agent trajectoryHow a complex task was completedHighest; teaches tool use, retries, and verification

That distinction is the practical core of the report’s illicit-distillation discussion. The risk is not necessarily a single copied answer. It can be the systematic collection of large volumes of structured capability traces.

What risk patterns does the report describe?

Anthropic’s report describes several patterns that should not be collapsed into one issue:

1. Large-scale capability extraction. High-volume queries collect reasoning, software-engineering, or complex-agent outputs for later training or evaluation.

2. Request forwarding. A user may believe they are interacting with one service while the request is routed to another model, creating notice, authorization, and retention concerns.

3. Replay of real sessions. Real user conversations or coding sessions may be sent to another model for rewriting, scoring, or answer generation, then reused as training material.

4. Agent and reseller chains. An intermediary can retain high-value human-plus-model interactions and potentially use or distribute them elsewhere.

The first pattern centers on model capability and intellectual-property concerns. The others also raise privacy, confidential-data, supply-chain transparency, and compliance questions.

What does an agent trajectory actually contain?

Many teams classify the data they send to a model but overlook the execution process itself. In practice, a high-quality agent trajectory can assemble scattered details into a much richer picture of the business:

Task objective → initial prompt → retrieved documents or code → tool-call parameters
→ intermediate reasoning and decisions → file changes → tests and errors → remediation → final output

That chain can contain access tokens, directory structure, dependency versions, customer names, business rules, internal URLs, and diagnostic logs. Each fragment may look harmless on its own. Together, they can expose the shape of an entire project.

Agent observability therefore cannot stop at “did the task succeed?” It also needs to answer: what data entered the context, what actions left the controlled environment, and who can access the trajectory again?

What should an enterprise do now?

Do not wait for a perfect AI data-flow diagram. Start with a focused service inventory.

1. Map data movement for every AI service

For each model, agent, plugin, and API gateway, document four fields:

Input: prompts, attachments, code, logs, database queries
Processing: model provider, agent framework, tool calls, third-party plugins
Storage: retention period; training, evaluation, or human-review use
Output: answers, code patches, execution logs, agent trajectories

If a team cannot answer whether a code sample crosses a third-party routing layer, it does not yet understand its practical AI boundary.

2. Treat agent trajectories as sensitive assets

Apply least privilege to agents. Limit accessible repositories, databases, and SaaS tools; isolate secrets and production permissions from task context; and retain auditable tool-call logs. Most importantly, distinguish between recording data for debugging and retaining it indefinitely for training.

3. Ask procurement questions that can be verified

Do not stop at model quality. Put these questions into vendor reviews and data-processing agreements:

• Are prompts, attachments, code, and tool-call logs retained? For how long?

• Can they be used for training, quality evaluation, or human review? What are the defaults and opt-out controls?

• Is there model routing, a third-party agent, a reseller, or cross-border processing?

• Are users clearly notified when a service invokes another model?

• Are audit logs, regional storage, deletion, and no-training controls available?

4. Establish a minimum viable agent-governance flow

You do not need to begin with a 100-page AI policy. Start by requiring every production agent to pass four gates:

1. Data classification. Identify whether it can access source code, personal data, financial information, credentials, or regulated records.

2. Permission boundaries. Default to read-only access. Treat write actions, email delivery, command execution, and production changes as separately authorized operations.

3. Vendor transparency. Record the primary model, fallback model, routing provider, plugins, and storage region—not just the product name.

4. Logging and exit. Retain the audit data you need, while defining trajectory retention, deletion, and no-training controls.

The goal is not to prevent teams from using agents. It is to make the points where risk enters—and where it can be cut off—visible before usage scales.

The Real-World Difference

AI security used to look like model access control: who can call a model, and who can download its weights.

It increasingly looks like behavior and data-supply-chain management: who can observe a task process, store an agent trajectory, forward a request to another model, or reuse those records for training.

For teams using AI to write code, process internal documents, or connect business systems, a full trajectory can reveal much more of the real workflow than a standalone chat response.

FAQ

What is illicit distillation?

In Anthropic’s report, it refers to unauthorized, large-scale, covert extraction of another model’s capabilities for training or improving a separate system. It differs from ordinary model distillation because the relevant questions are authorization, scale, concealment, and the kind of capability being extracted.

Why are coding-agent traces particularly sensitive?

They can include repository structure, error output, test results, tool calls, and remediation strategies. That makes them both sensitive enterprise context and unusually valuable process data for training agents.

What should an individual user remember?

Before sharing code, internal documents, or business data with an AI service, ask: Who am I actually interacting with? Can my data be forwarded? What is retained, and what may be used for training?

What’s Next

The next AI competition will not be only about larger models and more GPUs. It will also be about access to high-quality reasoning data, coding trajectories, and agent interactions under legitimate governance. For users, the practical first step is simple: include agent trajectories in existing data classification, access-control, and vendor-review processes.

Primary source: Anthropic, “Detecting and countering misuse of AI: September 2026”. Company-specific claims and traffic figures in this article are reported as Anthropic’s investigation findings and allegations, not as independently established facts.

Associated with this newsletter, I posted my video on Youtube (Note it is Chinese). Welcome to watch it.

A practical field guide for decision-makers and engineers — told through a travel-expense form, unpacking semantic uncertainty, path uncertainty, and the risk constraint, and why the bigger the consequences, the smaller the autonomy.

Let’s lead with the answer, then unpack why.

If the rules are clear, use a Function. If the semantics are fuzzy but the path is fixed, use a single LLM call or a Workflow. If the path is unknowable, use an Agent. The bigger the consequences, the smaller the autonomy.

That line is one of the few things OpenAI, Anthropic, Google Cloud, and Microsoft actually agree on in their public engineering guides — and it’s the first thing I see teams get wrong in practice. Almost every “can we just use an Agent?” question is asked the wrong way from the start.

One expense form, three answers

Walk into a real office for a second.

Finance gets a WeChat message from a colleague:

“Last week I went to Beijing to see a client. Flights were ¥2,300, the hotel was three nights at ¥2,700, plus two taxi receipts. I’m missing one hotel invoice. Can I file it first?”

One sentence. Nothing special. Almost every finance team has seen something exactly like it.

Now ask yourself one serious question — if you hand this to a system, who actually decides at each step? — and you’ll see that this single form lays out the core decision every AI project has to make, in miniature.

It needs three completely different kinds of capability:

• The parts that must be computed: a three-night cap, how much is over the limit, whether a missing invoice blocks payment. These are rules — fixed answers, done by code.

• The parts that follow a fixed path: read the receipt → pull the date and amount → check tax ID and budget → route the anomaly → generate the explanation. The steps are defined; the model only handles the “understanding” in the middle.

• The part that has to investigate on its own: if someone changes route mid-trip, gets double-charged, and only has a verbal approval, the fixed path breaks — and only then does something that decides its next step come into play.

This one form uses all three. So don’t rush to a conclusion. Let’s peel it apart layer by layer.

Don’t start with “can we use an Agent?” — selection is really about control

The question most people open with is: can my business use an Agent?

That’s the wrong question, and it’s asked too early. The real question is: which step needs model judgment, and which step must stay controlled by code, by the permission system, and by a person?

That’s really a re-allocation of control. You’re deciding who chooses the path in each step — a hard-coded rule, a predefined flow, or the model itself.

And these three aren’t a “pick one.” They stack:

• Function sits at the bottom as the reliable base — deterministic, testable, cheap.

• Workflow controls the main flow on top of it — code decides the route, the model handles local semantics.

• Agent takes over a small slice only where it genuinely needs to improvise — inside a boundary you draw.

Once you see it that way, that core judgment follows naturally.

Judging uncertainty: two axes, one hard line

So how do you actually judge “uncertainty”? Two axes, plus one line you never cross.

Axis one: semantic uncertainty. Does your input contain something that needs understanding? Natural language, images, documents, exception notes, the kind of fuzzy phrasing that needs context? Traditional rules struggle with these. “Can I file it first?” is a textbook example.

Axis two: path uncertainty. At run time, do you actually know what steps, tools, sources, and retries you’ll need? If you can list them, the path is fixed. If you can’t, it isn’t.

Put them on a grid:

The hard line is risk. How bad is it if this is wrong, and how easy is it to catch? Microsoft scores tasks on repeatability, impact, error detectability, and time sensitivity; NIST demands you set risk tolerance and human oversight explicitly. The principle is basic: low impact, easy to catch — automate freely. High impact, irreversible, hard to spot — keep a person in the loop.

One line worth writing down: an Agent’s autonomy ceiling should never exceed the system’s ability to verify its result and limit its damage.

Four terms, one pass

These four words have crisp boundaries once you pin down a single question: who decides the path?

Function — the reliable capability unit. A deterministic function, API, or traditional software service. Clear inputs, outputs, and error semantics; same input, same output, every time. Amount math, permission checks, data mapping, state-machine transitions, threshold decisions, DB transactions, compliance rules — all of it lives here. It’s the dependable floor of the system.

One nuance worth separating: Function isn’t Function calling. Function is the deterministic code you write. Function calling is just the interface that lets a model invoke that code. The same function can be called by a normal program, or used as a Workflow node, or exposed as an Agent tool.

Single LLM call — the layer people forget. Intent classification, entity extraction, summarization, text normalization, structured JSON generation. Most of these need exactly one constrained call, plus Schema validation and a fallback. Anthropic is blunt: many apps are fine with one call, retrieval, and a few context examples — don’t bolt on an Agent just so it “looks complete.”

Workflow — code defines the main path. Anthropic’s definition is crisp: a Workflow orchestrates LLMs and tools along predefined code paths; an Agent has the LLM dynamically control process and tool use. Workflows can include fixed sequences, limited routing, parallelism, deterministic loops, human approval, retries, and compensation transactions. Google adds the nuance: as long as the model isn’t deciding the overall path, it’s still predefined orchestration — even if it contains many “Agent nodes.”

Agent — the model gets runtime control. The point isn’t “it calls tools” or “it has many steps.” It’s that the model picks the next step, uses tools, adjusts its plan from intermediate results, and decides when it’s done or when to hand off to a human.

One correction worth making, because it’s a genuinely common misconception: task decomposition is not the same as an Agent. The orchestrator-workers pattern decomposes tasks dynamically too — but engineers can still constrain its outer loop tightly. The real question is: how much scope, how much time, how many tools, and how much consequence does the model actually hold?

The five-step decision: turn selection into a process

Don’t treat this as a chart on a wall. It’s a sequence — start simple, escalate only when the simple option fails.

1. Can it be written as a maintainable rule or algorithm? If yes, prefer Function.

2. Is it a single semantic transformation? Just classification, extraction, summarization, generation — use one constrained LLM call with structured output and validation.

3. Can you list the steps, branches, and error handling at design time? If yes, use a Workflow.

4. Can you really not know the next step ahead of time? Only when subtasks, sources, or tools depend on intermediate results do you bring in a bounded Agent.

5. Does risk allow autonomous execution? High impact, irreversible, hard to spot — reduce autonomy and add human approval.

Selection routing diagram: start at the upper-left question and follow the “can it be a rule → does it need one LLM read → can you list the steps → is the next step enumerable → does risk allow autonomy” path to an answer.

You can also lay this out as a decision matrix. Seven dimensions — input semantics, business rules, execution path, tool choice, testing approach, cost/latency, primary risk — line them up and it’s obvious: Function is cheapest and most stable; Agent is most flexible, most expensive, and riskiest. The further right you go, the more control shifts to the model, and the higher the cost and risk.

Five levels of maturity: L0 to L4

If you want a coordinate for “where are we right now,” this is the scale. Treat it as a measure of complexity and control, not a ladder of “higher is better.”

Five-level staircase: L0 deterministic Function → L1 single LLM capability → L2 LLM Workflow → L3 bounded Agent → L4 autonomous Agent.

• L0 · Deterministic Function: rules, algorithms, transactions, permissions. The reliable floor for any project.

• L1 · LLM Capability: classification, extraction, summarization, generation — with structured output, validation, fallback.

• L2 · LLM Workflow: a multi-step flow with fixed or limited branching. The default for most enterprise projects.

• L3 · Bounded Agent in Workflow: dynamic planning within a limited toolset and budget. For complex exceptions and exploration.

• L4 · Autonomous Agent: end-to-end continuous autonomy. Only for low-risk, reversible, strongly-verifiable scenarios.

Note that L0 through L3 coexist in one system — Function carries the base, Workflow drives the main line, Agent handles the few genuinely un-enumerable spots. The default policy is Code first → one LLM call → Workflow → bounded Agent → autonomous Agent, and you only escalate when the lower layer truly can’t hit your eval bar. For most teams, getting L2 right is already enough.

The China reality: why most projects start with the Workflow

This framework looks clean on a whiteboard. Put it inside a Chinese enterprise and the first blockers aren’t usually “is the AI smart enough” — they’re three very unglamorous things: systems that don’t talk to each other, processes that aren’t connected, and data that doesn’t agree on definitions.

The state-owned-asset regulator (SASAC) flags data standards, metadata, master data, business-scenario modeling, and process coordination. The MIIT’s SME policy prioritizes critical business systems, cross-system integration, light investment, short cycle times, and measurable ROI — not building a highly autonomous Agent on day one.

So the highest-value path for most projects is: use Function to lock down amounts, permissions, state, and compliance rules; use Workflow to embed model capability into the existing ERP, OA, CRM, MES, and WMS systems, and keep human approval where it belongs.

Different companies land in different places:

• SMBs / single department: L0–L2. Get one high-frequency loop closed; keep integration and ops costs down.

• Large groups / shared services: L2-first, some L3. Standardize identity, master data, permissions, and cross-system ownership first.

• Manufacturing / energy / logistics: Function + Workflow. Connect ERP, MES, WMS, EAM; separate the main flow from exception investigation.

• Finance / healthcare / highly regulated: Function + Workflow + human. Evidence, rule versions, audit trails, and high-impact approval take priority over autonomy.

Before you touch a keyboard, four constraints have to be settled. Data — master data, quality, and owners; low-quality data doesn’t become reliable just because you attached an Agent. Systems — do they have stable APIs, event mechanisms, and auditable write paths? If not, integrate first. Organization — bring in the executors, approvers, security team, finance and legal owners; don’t just transcribe the SOP. Governance — set the boundaries on data scope, model output, tool permissions, human handoff, and final accountability (GB/T 36073-2025 folds these into data-capability assessment).

Breaking down the form: three capabilities, each doing its own job

Enough theory. Drop it back onto that expense form and see it run.

Computation and rules go to the Function. Look up the hotel standard for a Beijing P7 employee. Compute the three-night cap, the overage, and the reimbursable amount. Decide whether a missing invoice blocks payment. Check the trip request, budget, duplicate invoices, and tax fields. Then generate the accounting entry and execute the payment transaction. You can literally see the functions:

FunctionExample inputExample resultcalculate_hotel_limitBeijing / P7 / 3 nightsCap ¥2,400calculate_reimbursableActual ¥2,700 / Cap ¥2,400Reimburse ¥2,400; overage ¥300check_invoice_requiredHotel / ¥2,700Invoice required

The normal claim runs through the Workflow. Most claims are “unstructured input, but the steps are clear”:

1. Material recognition — LLM identifies the receipt, itinerary, and the employee’s note;

2. Field extraction — LLM outputs date, amount, merchant, and expense type under Schema constraints;

3. Rule validation — Function checks tax ID, duplicate receipts, budget, and travel policy;

4. Anomaly routing — code sends missing-invoice, over-limit, and no-request cases down fixed branches;

5. Explanation & approval — LLM generates the explanation from rule results, escalates to a human when needed.

Reimbursement pipeline: read materials → extract fields → validate rules → route anomalies (normal → explain & approve; anomaly → human approval).

Here’s what it tells the employee:

“Your hotel actually cost ¥2,700. Under the P7 Beijing standard of ¥800/night and a three-night cap of ¥2,400, your reimbursable ceiling is ¥2,400. Because a valid invoice is missing, this can’t go to payment yet — you can supply the invoice or submit an exception request.”

Note this: the wording can be polished by an LLM, but the facts must come from the policy version, the employee’s grade, the city standard, nights stayed, actual amount, material status, and approval rules. The credibility comes from the evidence chain, not from the model narrating its reasoning.

A real anomaly is where the Agent earns its place. Say the employee rerouted from Beijing to Tianjin mid-trip: the hotel order was canceled and rebooked, the corporate card was charged twice and refunded once, the invoice amount doesn’t match the final payment, and the itinerary change only exists as a verbal approval in WeChat. A fixed Workflow can’t enumerate every investigation path for that.

So you hand a bounded Agent a clear objective: determine whether there’s a duplicate payment and what approval evidence is still missing. With tools and rounds limited, it works through:

1. Pull the original request — it only approved Beijing;

2. Search approved itinerary changes in the time window — find the direct manager’s OK to go to Tianjin;

3. Cross-check the order, charges, and refund — confirm the original order was refunded;

4. Compare invoice, net order amount, and net card spend — rule out a duplicate claim;

5. Find the Tianjin trip lacks a formal change form — request it and hand off to finance for review.

Notice what it did not do. It gathered evidence; it never decided the outcome itself. The final architecture looks like this:

StageControlling entityWhat it doesEntry & authorizationFunction / IAMVerify identity and data scopeAnomaly investigationBounded AgentPlan dynamically; limit tools, permissions, rounds, budgetAmount & rulesFunctionRecompute amount, policy, and tax resultsApprovalHumanHandle exceptions and high-impact judgmentPaymentFunctionExecute a controlled, auditable transaction

Letting an Agent run: draw the boundary first

Once the model gets runtime control, it stops being a component and becomes a subject with permissions. Three things to police.

First, least privilege. OWASP groups “too much functionality, too much privilege, too much autonomy” under Excessive Agency, and advises exposing the smallest toolset and permissions a task requires, doing real authorization checks downstream, and requiring human approval for high-impact actions. An Agent can read a bounded set of orders and policies, build an investigation plan, suggest next steps, prepare a structured payment request, and log failures for a human. It must not change policy, expand its own permissions, waive a missing invoice, approve its own conclusion, pay an employee directly, or delete audit logs.

Second, don’t trust its “reasoning.” The NeurIPS 2023 paper Language Models Don’t Always Say What They Think showed that a CoT explanation can look plausible while failing to disclose the bias that actually drove the answer — increasing trust without guaranteeing fidelity. In an enterprise, the reliable explanation comes from input evidence, rule hits, tool calls, policy versions, the approver, and the final business state.

Third, grade it like an exam. Don’t just accept “it says it’s done” — check that the business end-state is actually right. Anthropic recommends combining code, model, and human scoring, and separating “the Agent claimed completion” from “the environment reached the target state.” Coverage should include: task success rate and final state; tool choice and prohibited calls; policy adherence and override rate; human handoff and recovery rates; P50/P95 latency and token cost; and regressions across model, prompt, tool, and policy versions.

Before you start: run this checklist

Go through this before you ship. If any line is unclear, hold off on Agent autonomy.

• Business: is there a concrete task, an owner, a baseline, and quantifiable benefit?

• Rules: which decisions must be code, and which genuinely need model semantics?

• Path: what can you predefine, and what’s truly un-enumerable?

• Verification: can you mechanically verify the end state? If not, who owns the human judgment?

• Risk: what’s the error impact, detectability, reversibility, and approval window?

• Permissions: does the Agent have only the minimum tools, permissions, and data scope?

• Control: is there a max rounds, timeout, budget, retry, and kill switch?

• Eval: is there a gold set, repeated trials, and a regression gate?

• Run: are model, prompt, tool, policy version, and full traces recorded?

• Handoff: can the client team operate, extend, and troubleshoot it on their own?

Go / No-Go: if business value, risk tolerance, result verification, or ownership is still unclear, pause. Collect data with human-in-the-loop or a Workflow first, then decide whether to raise autonomy. Once the checklist passes, you can let it run — but first frame it inside permissions, budget, eval, and human oversight.

Hand control to the layer that needs it least

Let’s close it.

The framework is stable:

• Function：Deterministic inputs/outputs, explicit rules and algorithms

• Single LLM call or LLM Workflow：Need to understand unstructured semantics, but the flow is known ;

• Agent：Only when the next step, tools, and subtasks depend on intermediate results

And keep three negations in mind. Extracting one fact doesn’t require a whole pipeline. Being conversational doesn’t make it reliable — enterprise-grade explainability rests on evidence, rules, traces, and approval records, not self-narration. An Agent isn’t the default executor for payment or high-risk decisions.

For teams in real production systems, the pragmatic move is to go deep on L2 Workflow: lock rules into Functions, connect existing systems and the human accountability chain, and build eval on real run data. Only the few exceptions that truly can’t be enumerated — where tools and permissions are stable and results are verifiable — warrant upgrading to a bounded L3 Agent.

One line to hang it all on: hand control to the layer that needs it least. Where a rule works, don’t use model judgment. Where you can pre-orchestrate, don’t let the model improvise. And where consequences belong to a person, don’t hand them to a machine.

References:

• OpenAI, “A practical guide to building agents”;

• Anthropic, “Building effective agents”;

• Google Cloud Architecture Center, “Choose a design pattern for your agentic AI system”;

• Microsoft, “Decide when Copilot or an agent is the right tool for your work”; NIST AI Risk Management Framework Core;

• OWASP LLM06:2025 Excessive Agency; Turpin et al., NeurIPS 2023, “Language Models Don’t Always Say What They Think”; SASAC / MIIT digital-transformation policies; GB/T 36073-2025.

Associated with this newsletter, I posted my video on Youtube (Note it is Chinese). Welcome to watch it.

If you follow AI agent news at all, you’ve been hit by these five words on repeat: Ontology, Knowledge Graph, Context Graph, Code Graph, and Graph Engineering. They all contain “graph,” so they sound like one family. They’re not. Here’s the insight that makes everything click: the first four answer what the system knows; the last one answers how the system acts. Here’s the whole picture, one diagram at a time — plus a few opinions after I read through the source material and two papers.

The Core Conclusion First: What It Knows vs. How It Acts

Five terms, two buckets:

CategoryIncludesAnswers🧠 Knowledge layer · describes the worldOntology, Knowledge Graph, Context Graph, Code GraphWhat the system knows⚡ Execution layer · describes operationGraph EngineeringHow the system acts

One line to remember: what it knows decides whether the agent answers correctly; how it acts decides whether the agent runs reliably. The first four are your map and fuel; the last one is the track and the engine.

Where these terms came from: “Graph engineering” surfaced in July 2026 from X’s AI content factory, joining prompt engineering, context engineering, harness engineering, and loop engineering (per the LangChain blog). The words are new; the problem is real: LLMs are a new kind of non-robust, non-deterministic software, and every one of these terms is an attempt to make them do reliable work.

A few authoritative numbers to anchor yourself:

• Google’s Knowledge Graph launched in 2012 with 500M+ entities and 3.5B+ facts (official announcement) — note: that’s “world model” scale.

• Wikidata has passed 106M items, the largest open knowledge graph (Wikidata statistics).

• LangGraph, three years into graph engineering, now gets 65M+ monthly downloads (LangChain blog).

My take: 90% of the articles explaining these terms treat the first four as one category and then file Graph Engineering away as “engineering practice.” That’s the real mistake. The first four are knowledge; the last one is execution. They’re different dimensions, not different difficulty levels. Sort the dimensions first, then argue about details.

1. Ontology — Your Company’s “Glossary + House Rules”

In plain words: Ontology doesn’t record Zhang San or Li Si. It only defines three things — what “customer” means, what relations are allowed between “customer” and “order,” and which rules always hold. It’s a dictionary and a grammar book combined, strict enough for machines to read.

The technical detail: what an ontology actually looks like

Palantir breaks an enterprise ontology into six parts:

Ontology = Objects + Links + Properties + Logic + Actions + Security

• Objects: business object types (Customer, Order, Equipment) — not specific people;

• Links: allowed relations between objects, with cardinality (one-to-many, many-to-many);

• Properties: attributes per type (order amount, customer tier);

• Logic: inference rules and constraints (see the example below);

• Actions: executable operations and their triggering conditions — note that what lives here is the permission rule itself (e.g., “discounts above 15% require VP approval”); stringing multiple actions into an executable flow is Graph Engineering’s job (Part 5);

• Security: who can read, who can write.

In standards-land, ontologies are written in formal languages — OWL (Web Ontology Language) or RDF Schema — defining classes, properties, hierarchies, and logical axioms. The result is human-readable and machine-interpretable (Atlan).

Inference: ontology’s most valuable trick

You never hand-code “patient Z should take drug X.” Put two known facts together and the rules derive the third:

• Fact 1: Drug X treats Disease Y

• Fact 2: Patient Z has Disease Y

• Inferred: Drug X is likely relevant for Patient Z (Atlan adds the pharma-grade version: Drug X treats Y, and Y shares pathways with Z ⇒ Drug X may treat Z — this is how pharma companies use ontologies for drug discovery)

That’s why precision-critical industries can’t live without them: finance uses FIBO (EDM Council’s financial business ontology), healthcare uses FHIR / SNOMED CT, and the web uses Schema.org. In those domains, “diagnosis” cannot mean two things.

Ontology’s relatives: the semantic complexity ladder

A taxonomy only handles parent-child hierarchy — the grocery example from Neo4j: Foodstuffs > Fruit > Apple. A thesaurus adds synonyms. An ontology adds logic and inference. A knowledge graph fills in instances. They’re not peers:

Neo4j’s framing is worth keeping: “the knowledge graph is what you build, while taxonomies and ontologies are the blueprints.“

An honest footnote: the Semantic Web vision (RDF/OWL/SPARQL, W3C, early 2000s) of machines understanding the whole web never fully materialized — the “open world” assumption was once called “insanely impractical for data practitioners” (2021 Hacker News comment; community opinion, not peer-reviewed). But the core technology was sound — and AI is now the force making ontologies necessary at scale (Atlan).

2. Knowledge Graph — An Encyclopedia with Nerves

In plain words: Take the ontology’s rules and actually use them — fill in real people and real things, and you weave a web of “who — what relation — who.” Think of it as an encyclopedia with nerves: you can look up an entry, then follow the relations and keep asking.

The technical detail: two storage models, two philosophies

Knowledge Graph = Ontology Schema + Object Instances + Relationships

① RDF triples (Semantic Web tradition): every fact is a (subject, predicate, object) statement. Neo4j’s critique is concrete: a simple fact like “Apple is fresh food” has to be decomposed into multiple triples:

Apple  type  Fruit
Apple  type  FreshFood
Banana type  Fruit
Banana type  FreshFood
Salmon type  FreshFood
Salmon type  Fish

— properties get split, types get split, and query complexity climbs.

② Labeled Property Graph (LPG, the Neo4j model): properties hang directly on nodes and relationships; many-to-many and multiple same-type relationships come naturally, and one Cypher statement can query the ontology and the instances together. The original example from the Neo4j article — everything Daniel ordered in the Foodstuffs category:

MATCH (d:Person {name: "Daniel"})-[:PLACED]->(o:Order)-[:CONTAINS]->(p:Product)-[:TYPE_OF]->(t:Type)-[:TYPE_OF]-(c:Category {name: "Foodstuffs"})
RETURN d, o, p, t, c

Or “what did other people with Daniel’s address buy?” — a shared-attribute query in one statement. A knowledge graph has four core parts: Nodes, Relationships, Properties, and Organizing principles (the taxonomies/ontologies).

Authoritative data: how big are world models

• Google Knowledge Graph (2012, official blog): 500M+ entities, 3.5B+ facts — positioned from day one as “things, not strings”;

• Wikidata (launched 2012): 106M+ items, the largest open knowledge graph;

• Lineage: RDF / OWL / SPARQL from the W3C’s Semantic Web movement in the early 2000s.

GraphRAG: the hottest LLM-era use of knowledge graphs

GraphRAG isn’t a kind of graph — it’s a retrieval pattern that retrieves along a relation network. Microsoft’s 2024 paper From Local to Global: A Graph RAG Approach to Query-Focused Summarization (arXiv:2404.16130) made it famous: build a graph over the corpus, then create community-level summaries so the LLM sees both local details and global overviews. The key point (DataHub stresses it): GraphRAG is the pattern; a context graph is the infrastructure being retrieved from. They are not the same thing.

Use it when: general-knowledge Q&A (”Which country is Paris in?”); recommendations (”People who watched this also liked…”); multi-hop questions — the Neo4j example: answering “what’s the impact of changing product X,” an agent traverses product → category → supplier → orders → tickets → docs, instead of retrieving only chunks that mention the product name. The answer is explainable and traceable.

My reminder: a knowledge graph only knows what is generally true. It doesn’t know what’s happening in your company right now, which table is trustworthy, or who approved what. That’s the boundary with the next graph — and where many people stumble, treating the knowledge graph as a universal base and then wondering why their agent hallucinates about business status.

3. Context Graph — The “Trusted Business Context Map” for Agents

In plain words: The knowledge graph answers “what does the enterprise have.” The context graph answers “why this matters right now, and how to decide based on it.” It stacks a “right now” layer on top of “what exists”:

Context Graph = Knowledge Graph + Task Context + Runtime State + Events + Decisions + Evidence

Settling a debate first: what actually differs from a knowledge graph

DataHub’s take is sharp: the two have identical structure (entities + typed relationships), and most attempts to draw a hard line “invent a distinction that does not exist.” The real difference is two words: scope and grounding.

In DataHub’s words, a context graph is “structured knowledge that needs to be deterministic and governable, not a vibes-based retrieval blob.“

The technical detail: three layers (per Atlan)

1. Ontological grounding: what entity types exist and how they relate — without it, the graph stores facts but not meaning;

2. Operational metadata: column-level lineage, quality scores, ownership, access policies — “where it came from, whether it can be trusted”;

3. Decision traces: approvals, exceptions, overrides, and the reasoning behind them. When a VP approves a 20% discount above the policy cap, the graph records not the outcome but structured fields:

exception_approved_by = "VP Sales"            # who approved
override_reason       = "strategic account"   # why the exception
precedent_case_id     = "CASE-2025-0117"      # which precedent

Why structured? Because decision traces used to live in Slack threads, Zoom calls, and people’s heads (Atlan’s words) — agents couldn’t query them. As fields, one lookup answers “can we give this customer 20%?”

Governance: an ungoverned context graph is worse than none

Atlan’s refrain: a graph database stores relationships, it doesn’t govern them (”A graph database stores relationships. It doesn’t govern them.”). The five governance layers:

1. Asset inventory — you can’t govern what you haven’t cataloged;

2. Column-level lineage — when a metric definition changes upstream, you see exactly what breaks downstream;

3. Ownership/stewardship — when an agent hits ambiguity, there’s a human to escalate to;

4. Quality & observability — how fresh, how accurate, how complete;

5. Access policy — different users and agents see different things; permission-aware retrieval keeps sensitive data out of the wrong context window.

One line: “Context without governance doesn’t reduce hallucinations. It makes them harder to detect.”

How agents consume it: via MCP

The de facto delivery channel for context graphs to agents is MCP (Model Context Protocol), open-sourced by Anthropic on November 25, 2024 (announcement). Atlan’s approach: connect 75+ data systems and expose the governed context graph through a native MCP server; agents query it at inference time instead of having knowledge stuffed into a system prompt.

Industry signals (Atlan citing Gartner): Gartner analyst Rita Sallam calls context “the brain for AI“ and predicts universal semantic layers will be treated as critical infrastructure by 2030; organizations with the highest AI satisfaction invest nearly twice as much in foundations (data quality, governance, talent) as in AI tools. Workday reports 5x improvements in AI analyst response accuracy.

My take: take vendor numbers with salt (Atlan is a player here), but the direction is credible — “governed context first, agents second” is moving from best practice to industry consensus. When even Neo4j repositioned itself as “AI agent memory graph” infrastructure in 2026, that’s a signal.

Same name, different things

That’s the data-platform usage (DataHub, Atlan), anchored to one company. But there’s an academic paper also called Context Graph (IDEA Research, 2024, arXiv:2406.11160) about tagging every edge in a general knowledge graph with time · location · provenance · confidence. Paper examples:

(Barack Obama, president of, USA, time: 2009–2017)
(France national football team, win, 2018 FIFA World Cup, location: Russia)

The motivation: triples that lose their context contradict each other (”A lives in Shanghai” and “A lives in Beijing” can both be true — at different times). The paper proposes the CGR3 reasoning paradigm: Retrieve (candidate entities + context) → Rank → Reason (is the information sufficient? iterate if not), with significant gains on knowledge graph completion (KGC) and KG question answering (KGQA). Note: it’s anchored to the world, not a company — same name, two different things. Check who’s talking before you quote.

4. Code Graph — The Plumbing Diagram of Your Codebase

In plain words: A code graph draws the “internal plumbing” of code: functions, classes, and modules are nodes; who calls whom, how data flows, who inherits from whom are edges. Before changing one line, you follow the graph to see “who downstream is going to break.“

Code Graph = Source Code + AST + Dependencies + Execution Relationships

The technical detail: how GraphGen4Code was built (IBM Research, arXiv:2002.09440)

The largest public code-knowledge-graph engineering effort. The pipeline, step by step:

1. Data collection: a SQL query over Google BigQuery pulled 1.38M Python files/notebooks from GitHub (repos with ≥2 watch events, deduplicated by MD5);

2. Static analysis: they extended IBM’s WALA framework (originally Java/JavaScript) to run inter-procedural dataflow + control-flow analysis per script. The hard part: Python is dynamically typed — calling pandas.read_csv returns an unknown type, so they model “any call to an imported function returns a new object.” Imprecise, but scalable;

3. Doc extraction: 506 popular libraries (imported >1000 times) were introspected to harvest 6.3M documentation pieces across 2300+ modules;

4. Forum linking: Elasticsearch linked 45M StackOverflow posts + 2.7M StackExchange posts to classes/functions/methods (up to 5K matches per query);

5. Result: 2.09 billion edges = 75M docstring triples + 246M forum triples + 1.77B static-analysis triples, covering 278K functions, 257K classes, and 5.8M methods.

Why dataflow analysis is the key: the paper’s core example — two programs, one variable named model, the other clf_SVM_radial_basis. Token-level, they look completely different; after dataflow analysis, both point at the same label node sklearn.svm.SVC.fit — so a machine can discover “these two programs do the same thing.” Even df.fillna gets labeled pandas.read_csv.fillna, because return types in dynamic languages can only be inferred through dataflow. That’s why code graphs beat raw text/AST representations.

Querying: output is RDF named graphs (one per program, reusing SIO and Schema.org ontologies) or JSON; class hierarchies appear directly as triples, e.g. sklearn.svm.SVC rdfs:subClassOf py:sklearn.svm._base.BaseSVC.

Use it when: the repo is too big for any human to remember who calls whom; you’re changing a public API and want blast radius; you want a coding agent to onboard an unfamiliar codebase and find bugs. Applications: program search, code understanding, bug detection, code automation, code recommendation (forum posts cross-recommend APIs — e.g., SGDClassifier over SVC for large datasets).

5. Graph Engineering — Putting Rails on the LLM

In plain words: Graph engineering is drawing a flowchart for the agent: one node = one job, one edge = where to go next. Which steps must follow fixed rules and where the model gets to decide — the graph makes that explicit, instead of letting the model improvise end-to-end and patch mistakes afterward.

Graph Engineering = Data Engineering + Graph Modeling + Retrieval + Reasoning + Agent Orchestration

The first four blocks are basically what Parts 2–4 do. The real focus here is the last one: stringing those queries and reasoning steps into a workflow the agent can execute reliably.

The technical detail: a graph is a state machine (LangGraph’s view)

LangGraph’s model in one sentence: nodes do work; edges decide what happens next.

• Nodes: the units that do things — deterministic code, a single LLM call, a tool call, or a full agent with its own internal loop;

• Edges: what happens next — some deterministic (hardcoded), some conditional, chosen at runtime based on a node’s result, the current state, or an external signal.

That’s a state machine: the graph defines the workflow, the state flowing through it, and the transitions. It encodes your world knowledge of how the system should work directly into structure — the same way prompts encode domain knowledge. That’s why the same model, drawn as a graph, beats a bare ChatGPT-style loop for predictable work.

A real workflow: turning “update the docs” into a graph — Slack request → classify (one LLM call) → two sub-agents in parallel (reference docs agent, concept docs agent) → synthesize (one LLM call) → open a PR:

Note the diagram: nodes in the same graph have different degrees of freedom — gray is deterministic code (Slack/Linear API calls), light purple is a single LLM call (classify, synthesize), dark purple is a full sub-agent. That’s the soul of graph engineering: the balance between “deterministic rails” and “smart judgment.” It’s exactly the balance that made LangGraph take off (65M+ monthly downloads).

Three lessons from three years (LangChain’s official summary)

1. Production agent graphs are usually not DAGs: retrying failed tool calls, asking users for missing information, revising answers after validation, calling tools repeatedly until enough context, pausing for human input — all of it is “going back,” so cycles are the norm;

2. A loop is just a simple graph: as LangGraph’s David Khourshid put it — “a loop is just a directed, cyclic graph.“ And one node can hold an entire sub-agent inside it;

3. Dynamic transitions matter: you don’t always want to define every edge up front. Classic case, map-reduce: split input into pieces, send each to a worker, combine — the number of workers is only known at runtime. LangGraph’s Send API lets a node route work to one or more downstream nodes dynamically, without statically defining every transition.

When not to use graphs (important)

Some tasks are more agentic by nature, and forcing them into deterministic paths is the wrong move. LangChain admits it: generic deep research is the counterexample — planning, delegating, searching, reading, synthesizing can’t be pinned down ahead of time. GPT Researcher (the most popular deep research implementation) already swapped its graph-shaped multi-agent pipeline for Deep Agents, letting planning, delegation, and context management emerge in the harness instead of being hardcoded in the graph.

My rule of thumb: does your process have a fixed pattern? Support classifying before answering, compliance approval before external actions, code changes checked against impact — that’s a pattern, draw the graph. Exploratory work (research, brainstorming) — don’t draw first, think first.

The Full Picture: How They Fit Together

Stack all five into one architecture and you get this (dense, English-original reference diagram):

Read it top-down: Ontology defines the rules at the top; Knowledge Graph / Context Graph / Code Graph are three graphs of the same shape with different grounding; Graph Engineering turns them into a queryable, executable system; at the bottom, the agent walks the task through “understand goal → query graphs → reason and decide → execute actions → write back and learn.”

All Five Side by Side

Memory aid: rulebook → world-web → business context map → code plumbing, and the last one is the agent’s rails.

One Real Scenario: A Supplier-Sourcing Agent

An agent that finds, evaluates, and recommends suppliers — here’s which graph does what:

Summary: Five Equations

AI agents are converging on “two graphs”: one that describes the world, one that describes execution. The first four answer “what it knows”; graph engineering answers “how it acts.” Get both right, and the agent is both knowledgeable and dependable.

FAQ: Quick Answers

What’s the difference between an ontology and a knowledge graph?

An ontology is the “empty rulebook” — it defines types, relations, and rules (written in OWL/RDF Schema) but holds no instances. A knowledge graph is the web you get by filling that rulebook with real instances and facts (property graph or RDF). One line: the ontology says what can exist; the knowledge graph records what does exist.

What’s the difference between a context graph and a knowledge graph?

Same structure; the difference is scope and grounding. A knowledge graph is a world model answering generally-true questions; a context graph is an organization model anchored to one company, answering “what should we trust and do right now,” layering task context, state, events, decision traces, and governance metadata on top of facts.

How is GraphRAG related to knowledge graphs?

GraphRAG isn’t a kind of graph — it’s a retrieval-augmented generation pattern that retrieves along a relation network (Microsoft, arXiv:2404.16130). It can sit on top of a knowledge graph or a context graph. The context graph is the infrastructure being retrieved from; GraphRAG is just one pattern. Don’t equate them.

How is a code graph different from a regular dependency/call graph?

Package-manager dependency graphs stop at module/package granularity. A code graph goes down to functions, classes, and methods and includes dataflow (not just calls — e.g., df.fillna is identified as pandas.read_csv.fillna), inheritance, and doc/forum links — so it can answer “if I change this function, who downstream breaks?”

How is graph engineering different from ordinary workflow orchestration (DAGs)?

Ordinary orchestration just draws “what runs first, what runs next.” Graph engineering binds graph queries, relation reasoning, and agent orchestration together — nodes can be queries, reasoning steps, tool calls, or full sub-agents; edges can be conditional; loops, retries, human approval, and runtime dynamic routing (the Send API) are first-class. Production agent graphs are rarely DAGs.

Which graph should I use?

Quick lookup: unifying terminology or machine inference → ontology; general Q&A and recommendations → knowledge graph; agents working inside your company, impact analysis, audit trails → context graph; changing code or coding agents → code graph; orchestrating a process with a fixed pattern → graph engineering.

References

Primary sources (web articles):

1. 3 Years of Graph Engineering with LangGraph — LangChain Blog, 2026-07 (65M+ downloads, state-machine modeling, Send API, GPT Researcher counterexample)

2. Context Graph vs Knowledge Graph: What’s the Difference? — DataHub Blog, 2026-04-30 (scope and grounding, world-model vs organization-model, Kurt Cagle quote)

3. Context Graph vs. Ontology: Differences, Roles & Use Cases — Atlan, 2026-01-27 (three layers, five governance layers, MCP, Gartner/Workday data, tool comparison)

4. Taxonomy vs. ontology vs. knowledge graph: What’s the difference? — Neo4j Blog, 2026-06-17 (four KG parts, Cypher examples, RDF critique, SKOS/n10s)

5. A Toolkit for Generating Code Knowledge Graphs (GraphGen4Code) — IBM Research + RPI, arXiv:2002.09440 (1.38M files / 47M posts / 2.09B-edge pipeline)

6. Context Graph — IDEA Research, arXiv:2406.11160 (CGR3 retrieve-rank-reason paradigm; Obama timestamp example)

7. Introducing the Knowledge Graph: things, not strings — Google official blog, 2012-05-16 (500M+ entities, 3.5B+ facts)

8. Wikidata statistics — Wikidata community statistics (106M+ items)

9. Introducing the Model Context Protocol — Anthropic, 2024-11-25 (MCP protocol launch)

10. From Local to Global: A Graph RAG Approach to Query-Focused Summarization — Microsoft, arXiv:2404.16130 (original GraphRAG paper)

If this cleared up the five terms for you, subscribe to zerofuturetech — we keep breaking down AI infrastructure concepts in plain words, diagrams, and code.

Associated with this newsletter, I posted my video on Youtube (Note it is Chinese). Welcome to watch it.

If you’ve ever turned off a VS Code extension and had the editor beg you to reload the whole window, you already know the feeling this paper is about. A new paper out of Peking University and DeepSeek-AI argues that this isn’t just an annoying detail — it’s a gap nobody had actually closed, and they built something that closes it.

Here’s the twist: the paper isn’t really about code editors. It’s laying groundwork for something closer to DeepSeek’s real interest — AI agents that rewrite their own tools while they’re running, with no human watching over their shoulder. VS Code is just the example every developer already has a gut feeling about, and the paper uses it as the easy way in before getting to what it actually cares about.

The problem, in plain terms

Software is supposed to work like Lego bricks: plugins, extensions, tools — pieces you snap in and out whenever you want. In reality, that’s not how it goes. Remove one piece, and you either have to restart the whole thing (losing every open session, cache, and connection along the way) or you’re left with weird leftover mess that nobody can account for.

There are really two separate problems hiding inside “plugins are annoying to remove”:

• Can you clean up completely? When a plugin is removed, does everything it touched actually get put back the way it was — every notification it signed up for, every connection it opened, every setting it changed?

• Do plugins actually know what they need from each other? Can a plugin say “I need this other thing to work,” and does the system react sensibly when that thing shows up, disappears, or gets swapped out?

Most plugin systems fail at both, and the numbers from VS Code are almost embarrassing: of the top 100 extensions by install count, 87 of them contain real code and therefore need a full restart to remove. VS Code’s official “shutdown” hook only fires when you close the whole editor, not when you remove one extension — and even then, the code that cleans up lives in a totally different place from the code that made the mess, so there’s nothing forcing the two to actually match up.

On the other side: only 7 out of those same 100 extensions bother to declare a formal dependency on another extension. Not because plugins rarely need each other — but because the tools VS Code gives you for expressing “I depend on that other plugin” are so weak and untyped that developers just avoid using them entirely.

The fix, in plain terms

The core idea is actually simple, even though the paper spends 88 pages proving it rigorously: make every action come with its own undo button, and make every dependency come with its own doorbell.

The undo button. Instead of a plugin just quietly changing things and hoping someone remembers to clean up later, every single change is recorded together with exactly how to reverse it — right at the moment it happens, not written separately somewhere else. Think of it like the undo history in a text editor: every keystroke gets logged, and hitting Ctrl+Z enough times gets you back to a blank page. Uninstalling a plugin is exactly that — running its whole undo history backward. Nobody has to remember to write a cleanup script; the cleanup is automatically built from whatever the plugin actually did.

There’s a nice wrinkle here that shows the authors actually thought this through: undoing things strictly in reverse order is the safe default, but it’s not the only option. If two plugins are doing things that don’t interfere with each other — say, two different bots each listening for their own separate messages — it genuinely doesn’t matter which one gets undone first. But if plugins are chained together, like an assembly line where each step processes what the step before it produced, then order suddenly matters a lot, and you can’t just yank one out of the middle. The paper is honest about which situation you’re in before it lets you skip the strict order.

The doorbell. Instead of a plugin discovering a broken dependency only when it crashes, every plugin announces up front, in plain terms, what it needs — a storage service, a network connection, whatever. The system keeps a checklist: a plugin is only allowed to turn on once everything on its checklist is actually available. When something changes, the system checks the checklist of every plugin, but only pings the ones whose checklist is actually affected. It’s the exact apartment-building trick you’d expect: if the water goes out, the building manager only wakes up the tenants who said “I need hot water” — everyone else keeps sleeping.

Two extra tricks stack on top of this, and both map onto very real problems:

• The same word can mean different things to different tenants. If you’re running one plugin for ten different customers, you want each customer’s “storage” to point at a completely separate database — without changing a single line of the plugin’s own code. That’s exactly what this mechanism buys you.

• A landlord can post extra house rules without touching the tenant’s stuff. A plugin can ask for “database access,” and something above it — an app owner, a security policy — can quietly attach a rule like “read-only for this one” without editing the plugin at all, and that outside rule always wins. That’s how you’d give a community-made plugin limited access while your own official plugins keep full access, using the exact same underlying plugin code for both.

“Blueprint” vs. “the actual thing built from it”

One detail that clears up a lot of confusion: the paper is careful to separate a plugin’s definition (the code, plus what it needs and what it offers) from a running copy of it. That’s just the blueprint-vs-building distinction. The same “chat platform connector” plugin code can be spun up as two separate running copies at once — one talking to Discord, one talking to Telegram — each with its own on/off switch and its own undo history, completely independent of the other. Almost everything interesting in the paper is really about the running copies, not the code itself, which by itself is just a template sitting on a shelf.

The boring-looking edge cases are where the real engineering is

The headline ideas (undo buttons, doorbells) handle the easy 80% of the problem. The paper spends a comparable amount of effort on the messy 20% — what actually happens at the exact moment something is turning on or off:

• Don’t pull the rug out while someone’s still using it. If Plugin B depends on Plugin A, and A is being shut down, A can’t just vanish — the connections A is handing back might literally be things it needs to give back to its own upstream service first. So A first stops advertising itself as available (so nothing new can start relying on it), then waits for every plugin currently depending on it to finish shutting itself down, and only then does A run its own cleanup. Get this order wrong, and you either freeze the whole system or yank something out from under a plugin that’s mid-task.

• Loading can pause partway and back out cleanly. Turning a plugin on isn’t one giant leap — it’s broken into checkpoints. Between checkpoints, the system rechecks whether anything the plugin depends on has changed. If it has, loading stops right there and only undoes what’s actually been done so far — not the whole system, not a half-broken mess.

• You can’t un-send a letter. Once a plugin has actually fired off a network request or opened a real connection, there’s no taking it back mid-flight — you have to let it land first. The system is built around this: an in-progress action is always allowed to finish, and only then does the plugin react to whatever changed while it was busy. This is the difference between a theory that’s elegant on paper and one that actually survives contact with real, messy I/O.

• One broken plugin doesn’t take the others down with it. If a plugin throws an error while starting up, the system runs the exact same cleanup path it would use for a normal shutdown, and marks that one plugin as failed — it doesn’t retry forever against an environment that hasn’t changed, and the failure doesn’t spread to its neighbors. That’s exactly the behavior you’d want from any plugin host, and the paper is explicit that this is why failures are handled one plugin at a time instead of as some global system-wide event.

Why the math actually matters here

It would’ve been easy to stop at “track changes, declare dependencies” — plenty of frameworks already gesture at both, and the authors admit they’re not the first to try. What sets this paper apart is that it doesn’t just claim the mechanism works — it proves it, with roughly sixty formal statements covering things like: the system never gets stuck, it always settles down eventually, and — the one that actually matters if you’re running this in production — no matter what order plugins load, unload, and reload in, and no matter how those actions get interleaved in time, the system always ends up in the exact same final state. Not “usually.” Not “in the cases we tested.” Always. That’s the guarantee that lets you treat a constantly-changing, live system as if it were something simple and static when you reason about it — and without it, all anyone could honestly claim about a hot-reload feature is “it happened not to break when I tried it.”

Worth comparing this to the two familiar extremes, because the paper does exactly this. React’s useEffect hook already pairs an action with a cleanup function the framework calls automatically — which is basically the same idea in miniature. But it’s badly boxed in: you can only use it at the very top level of a component, never inside a loop or an if-statement, and it can’t be async. So a hook’s cleanup can never really be built out of other hooks’ cleanups. The system in this paper has none of those restrictions — its version of “an action with a cleanup attached” can be async, can be looped, can be nested arbitrarily deep, which is exactly what a plugin doing real work (network calls, file access) actually needs.

Not just theory — already running 4,000+ real plugins

The authors built all this into something called Cordis, a framework written in TypeScript. It comes in layers: a core piece that does the actual undo-tracking and dependency-checking; a layer on top that lets you describe your whole system as a config file and have the system figure out the minimal set of changes needed to match it, rather than tearing everything down and starting over; and app-specific frameworks built on top of both.

That config layer also ships with live code-reloading during development that needs zero manual setup from the plugin author — unlike similar tools in the JavaScript ecosystem, which usually require you to hand-write “here’s how to accept this specific update.” It figures out on its own which files changed, which plugins are actually affected, and reloads only those — and if a reload fails halfway (say, a typo introduces a syntax error), it rolls the whole batch back to the last working version instead of leaving your app in some broken in-between state.

All of this is already powering Koishi, an open-source chatbot framework that’s been in active development for four years and has accumulated over 4,000 community-made plugins — connectors to different chat platforms, database drivers, admin dashboards, you name it. Interestingly, Koishi’s web-based admin console is a second, completely separate system built on the exact same underlying framework, but running entirely in a browser instead of on a server — good evidence this isn’t a trick that only works for one specific kind of software.

The authors are refreshingly upfront about the limits here too: this is one plugin ecosystem, in one programming language, and what they’ve shown is “this actually exists and people use it” rather than a controlled experiment proving it beats the alternatives. Nobody yet knows how much overhead all this undo-tracking and dependency-checking adds at a much bigger scale, or whether it holds up outside the specific tech stack it was built in.

The paper argues with itself, and that’s a good sign

A big chunk of the paper (roughly a third) is the authors poking holes in their own idea, which is honestly some of the most useful material in the whole thing for deciding how much to trust it:

• Not everything can actually be undone. The whole system depends on drawing a boundary around what it’s allowed to promise to fix. Write something to a database it controls? Fully reversible. But the moment a plugin does something that leaves the system’s boundary — sends an email, charges a credit card, writes a message another program has already read — there’s no real “undo” anymore. The best you can do is either hold off on the risky action until you’re sure it’ll stick, or compensate afterward: issue a refund instead of un-charging the card, delete the file instead of un-writing it. Similar idea, way less of a guarantee.

• Two plugins that need each other just... never turn on. If Plugin A needs something from Plugin B, and B needs something from A, neither one’s checklist can ever be fully checked off — nothing crashes, they just sit there forever, doing nothing. The fix the paper suggests is splitting things up more finely — e.g., break a “server + access-control” plugin pair into four smaller, one-directional pieces instead of two that need each other. The catch, which the authors say plainly: the number of these little connector pieces you need can grow much faster than the number of original plugins, so you’re trading a correctness headache for a “now I have fifty tiny plugins to keep track of” headache.

• Two plugins can accidentally use the same name for completely different things. If Plugin A calls its database connection “storage” and Plugin B — built by a totally different team — also calls its thing “storage,” but they mean two different databases, the system has no way of knowing they’re not actually compatible. Or a plugin quietly changes its own interface in a later version, and everything still depending on the old version breaks in confusing ways. The paper lays out a few partial fixes and admits, straightforwardly, that nobody’s solved this cleanly yet.

How this stacks up against stuff you might already know

• OSGi, an older Java framework, already lets plugins declare what services they provide and need, and reacts automatically when those come and go — which is basically the ancestor of the “doorbell” idea here. Where it falls short is exactly where VS Code falls short: cleanup is a hand-written callback, easy to forget, and it can’t properly wait for something that takes time to finish shutting down.

• Erlang and tools like Webpack’s live-reload take a genuinely different bet: instead of undoing the old version, they try to carefully migrate its existing state forward into the new version. That’s nicer when it works — nothing gets lost — but it means someone has to hand-write a migration recipe for every single version change. This paper’s approach needs zero migration code, at the cost of starting fresh in memory every time something reloads, unless a plugin deliberately stashes its state somewhere longer-lived.

• React’s useEffect, again, is the closest cousin in spirit — but boxed into a much narrower slice of what real plugins actually need to do, for the reasons above.

Where this actually points: AI agents that rewrite themselves

This is the part that explains why a company like DeepSeek would co-author a paper about plugin architecture in the first place. The paper says it outright: future AI agent systems “may generate and deploy modifications to its own components while continuously serving requests” — an AI rewriting its own tools mid-task, with little or no human checking its work. The paper explicitly cites blog posts from both OpenAI and Anthropic about building these kinds of AI “harnesses” as the real-world backdrop for this research. And the closing section names exactly this — AI agents that keep evolving themselves — as the next thing they want to test this framework against.

That completely changes how heavy this all feels. Restarting a code editor to remove a broken extension costs you a few seconds of mild annoyance. Restarting an AI agent mid-task, with nobody around to notice or approve the restart, costs you the entire task — and if a bad self-modification happens to disable the very thing the agent would need to fix itself, it costs the agent its ability to recover at all. The moment the thing doing the swapping is an autonomous system nobody’s watching, “provably correct” stops being a nice-to-have.

My take

The core idea here is genuinely new, as far as I can tell — nobody had formally connected “track what changed” and “declare what you need” to systems that actually change while running, only to code that’s fixed before you hit run. The VS Code numbers alone are worth remembering on their own: a 7% dependency-declaration rate in a huge, well-funded plugin ecosystem tells you the tooling itself was the bottleneck, not lazy developers. Anyone building a plugin or tool ecosystem right now — and that very much includes the wave of AI tool-connector standards showing up everywhere — should take that seriously.

Where I’d push back a little: the authors are upfront that Koishi is a single data point, and “this works great for chatbot plugins” is a long way from “this is how you should build a self-modifying AI agent.” The self-evolving-agent use case that motivates the entire paper is explicitly listed as future work, not something they’ve actually shown works. And the honest limitations they list themselves — things outside the system’s reach can only be patched up, not truly undone; interdependent plugins multiply fast when you split them apart; version conflicts are still an open problem — are exactly the sharp edges an unsupervised AI agent would hit hardest, precisely because there’s nobody around to notice when a “patch it up” fix didn’t actually fix it. If this really is early groundwork for agents that rewrite their own toolchains on their own, the genuinely exciting follow-up paper is the one where this survives an AI doing that for real, under real load, with no human checking its work. Until then, this is a solid, unusually honest foundation — with its hardest test still ahead of it.

One-sentence takeaway: DeepSeek and Peking University didn’t just patch the “restart your editor to remove a plugin” annoyance — they proved, with real math, that plugins can be installed and removed cleanly and safely, built it into something already running in production, were refreshingly honest about exactly where that proof stops holding, and pointed straight at the real reason they care: someday, AI agents will need to safely rebuild themselves while nobody’s watching.

Associated with this newsletter, I posted my video on Youtube (Note it is Chinese). Welcome to watch it.

There are two camps when the topic is AI agents: people who treat “agent” as a cure-all and shove one into every feature, and people who dismiss it as marketing fluff. Both are wrong.

Andrew Ng framed it best: “Rather than having to choose whether or not something is an agent in a binary way, it would be more useful to think of systems as being agent-like to different degrees.”

That points to a far more practical lens: agenticity is a spectrum, not a checkbox. MongoDB calls these agentic systems — systems that use LLMs to decide the execution flow of an application with varying degrees of autonomy. The question is never “is this an agent?” but “how much decision-making authority did we delegate to the LLM?” From hard-coded flows to fully autonomous multi-agent systems, every rung has a mature design pattern with its own cost, latency, and reliability profile.

Here’s one counterintuitive fact to hold onto (from Anthropic’s work with dozens of teams): the most successful implementations almost never used complex frameworks. They were built with simple, composable patterns. Agentic systems routinely trade latency and cost for better task performance — make that trade only when it’s worth it.

This post maps the patterns that teams at Anthropic, OpenAI, MongoDB, and Databricks actually use in production — what each one is, when to reach for it, what it looks like in the real world, and implementation sketches you can copy.

First, pin down two distinctions: Workflow ≠ Agent, LLM ≠ Agent

Workflows and Agents are not the same thing

Anthropic’s Building Effective Agents gives the most useful framing, with a concrete example for each:

• Workflows are systems where LLMs and tools are orchestrated through predefined code paths. The flow is fixed; the LLM only generates within each step and never decides what comes next. Example: a fixed RAG chain — retrieve → assemble the prompt → generate, the order is hardcoded; a “marketing copy → translate” prompt chain is a classic workflow too.

• Agents are systems where LLMs dynamically direct their own process — which tools to call, how many turns, when to stop. Example: a support agent facing “I want to return this” has no script — it decides to look up the order, check the return policy, verify the window, then trigger the refund, each step decided on the fly.

The distinction matters because the cost structures differ. Workflows are predictable, auditable, low-latency. Agents are flexible and can handle open-ended tasks — at the price of higher cost and compounding errors.

Databricks’ docs walk through a call center agent end-to-end, which makes it concrete where the “decisions” actually happen:

User: “Can you help me return my last order?”

1. Reason and plan: Identify the intent, plan “look up the user’s recent order and check our return policy.”

2. Find information: Query the order database, reference the policy document.

3. Reason: Check whether the order fits in the return window — optionally: if the item falls into a certain category or is outside the normal window, escalate to a human.

4. Action: Trigger the return process, generate a shipping label.

5. Reason: Respond to the customer. “Done! Here is your shipping label…”

These steps are second nature in a human call center. In an agent system, the LLM reasons while specialized tools and data sources fill in the details.

LLMs and Agents are not the same thing either

An LLM is a model; an Agent is a system. The difference isn’t intelligence — it’s whether there’s an action loop. An LLM only generates text: ask it “where is my order?” and it guesses from training data. An agent gets things done: for the same question it calls lookup_order() to fetch your real order, calls track_order() for live shipping status, and answers from actual data. It’s not guessing — it’s doing.

What makes up an agent? Lilian Weng’s (OpenAI research scientist, 2023 classic LLM Powered Autonomous Agents) framework has four parts:

• LLM as the core brain: the model that drives reasoning and decisions. You don’t need the strongest model for every task — run the strongest model everywhere to establish a baseline, then swap in smaller models one by one and verify they still pass. Simple retrieval with a small model; refund approvals with a large one.

• Planning: decompose complex goals into executable subtasks and order them (task decomposition), reflect and adjust when stuck. Representative techniques: ReAct (reasoning and acting in alternation), HuggingGPT’s pipeline-style decomposition.

• Memory: short-term memory is the conversation history in the context window; long-term memory lives in external storage (vector stores, databases) and retains user preferences and past decisions across sessions, retrieved and injected when needed.

• Tools: the channels through which an agent gathers information and takes action — three kinds: Data tools (query databases, read docs, search the web), Action tools (send email, update records), and Orchestration tools (wrap other agents as tools).

Production adds two more: instructions (the SOP — good instructions come from real operational docs: decomposed steps, explicit actions, edge branches written in advance; without them, behavior is uncontrollable. It’s rule one of OpenAI’s official agent guide) and guardrails (permissions and compliance). Remember this one: an agent without guardrails doesn’t go to production.

One more concept worth investing in: the ACI (agent-computer interface). Anthropic’s rule of thumb: put as much effort into the interface between agents and tools as you’d put into a human-computer interface. Tool descriptions need examples, edge cases, and parameter docs; prefer absolute paths over relative ones — on SWE-bench they found the model kept fumbling relative file paths, and the problem vanished once tools forced absolute paths. Write it like a great docstring for a new junior engineer.

1. Controlled workflows — zero autonomy, start here

1. Prompt Chaining

Decompose a task into a fixed sequence of steps; each LLM call processes the previous step’s output, with programmatic “gates” between steps that stop the flow when something’s off. Every call becomes a smaller task — you trade latency for accuracy, and errors don’t snowball because each step is checked.

# A fixed RAG chain: the order is hardcoded, the model doesn't get a vote
docs = vector_store.search(query, top_k=5)      # ① retrieve top-k relevant docs
prompt = assemble(query, docs)                  # ② user request + retrieved context
answer = llm.generate(prompt)                   # ③ generate the response

Use it when: the task decomposes cleanly into fixed subtasks and you’d trade latency for accuracy; you want intermediate checkpoints to guarantee quality.

Real-world:

• Generate marketing copy → translate it into other languages

• Write an outline → check the outline against criteria → write the document from it

• Content review: an LLM fixes punctuation and grammar, verifies against style guides, fact-checks — then the document goes to human technical review

• Natural language to query: a sequential self-querying workflow that extracts data sources, filters, and other query parameters step by step

2. Routing

A smart switch: an LLM classifies the input and sends it down the right path — different prompts, smaller vs. larger models, or domain-specific fine-tuned models or microservices.

Here’s the analogy: a hospital triage nurse. The nurse (router) doesn’t treat anyone — they just decide which department you belong in. A common cold goes to general practice; a rare condition goes to a specialist. Specialist resources are expensive; don’t waste them on small stuff.

Classification is the core skill. The more diverse your inputs in topic and difficulty, the more routing pays off — but only if classification is reliable. Get it wrong and everything breaks: hard questions land in a small model and quality collapses; easy ones land in a reasoning model and your budget collapses.

Use it when: requests span diverse topics and difficulty, and classification is reliably accurate.

Real-world:

• Customer service: returns → one flow, tech support → another, each with its own processes and tools

• Route easy/common questions to a small, cheap model and hard ones to a reasoning model. Anthropic’s example: Claude Haiku 4.5 for common simple questions, Claude Sonnet 4.5 for the hard stuff.

category = router.classify(query)        # the LLM classifies the input
if category == "simple":
    return haiku_4_5.answer(query)       # small model: fast, cheap
else:
    return sonnet_4_5.answer(query)      # reasoning model: slower, better

3. Parallelization

Several LLMs work simultaneously and their outputs get aggregated. Two variants — don’t mix them up:

• Sectioning: split the task into independent subtasks run in parallel. Each LLM concentrates on one aspect — focus itself is a quality win. Example: one model instance processes user queries while another screens them for inappropriate content; that reliably beats a single LLM call doing both jobs.

• Voting: run the same task multiple times (different prompts or different models) and compare or combine the diverse outputs for higher confidence. Example: several prompts review the same code for vulnerabilities; multiple prompts judge whether content is inappropriate, with thresholds set to balance false positives and negatives.

Use it when: subtasks are genuinely independent, or you need multiple perspectives for confidence.

Real-world:

• LLM-as-a-judge: multiple models (or prompts) score different aspects of a response at once

• Code generation: several models each write code for the same prompt, the snippets get executed, and an LLM picks the most efficient one

The payoff is exactly two things: lower latency (run concurrently) and better quality (focused attention + multiple perspectives). For complex tasks with several considerations, a single all-in-one LLM call usually loses to several focused ones.

4. Evaluator-Optimizer

One LLM generates, another evaluates in a loop until the output passes. The LLM acts as its own critic — the same family as “reflection,” polishing output against explicit standards.

while not evaluator.passes(draft, criteria):
    feedback = evaluator.critique(draft)            # evaluator critiques against standards
    draft = generator.rewrite(draft, feedback)      # generator rewrites with feedback

The precondition is strict: evaluation criteria must be clearly writable, and iteration must measurably improve results. Anthropic’s two signs of good fit: first, a human can articulate the feedback that would improve the output; second, the LLM can actually deliver that feedback. With vague criteria, the loop spins in place — or makes things worse.

Real-world:

• Literary translation — the translator LLM drafts, the evaluator LLM flags nuances it missed, and the loop refines

• Report generation — the evaluator checks style, specific details, and diagrams, and recommends improvements

• Complex search tasks — the evaluator decides whether further rounds of searching are warranted

5. Orchestrator-Workers

A central LLM dynamically breaks down a task, delegates to worker LLMs, and synthesizes results. Unlike parallelization, the subtasks aren’t pre-defined — the orchestrator decides them per input. It’s also a concrete implementation of the Planning component from Andrew Ng’s four patterns: the planner is a task decomposer.

Use it when: the shape of subtasks depends on the input and can’t be hardcoded — coding products that change multiple files per task (which files, and how, depends on the task), or search tasks that gather and analyze information from multiple sources.

2: The autonomous single agent — a while loop

When people say “AI agent,” this is usually what they mean: one LLM plus tools, in a loop, deciding each next step until the task is done.

Under the hood it’s embarrassingly simple — a while loop: feed the LLM’s output back, let it see environment feedback (tool results, code execution output), and let it decide the next move.

while not done:
    step = agent.think(context)          # LLM reasons, decides the next step
    if step.is_final:
        done = True                      # exit: final-output tool / direct answer / max iterations
    else:
        result = step.call_tool()        # execute the tool, get ground truth from the environment
        context += result                # feedback returns to context; loop continues

At every step, the agent must obtain “ground truth” from the environment (tool results, code execution) to calibrate its progress — that’s the line between an agent and a system that’s making things up.

Use it when: open-ended tasks with unpredictable step counts, latency tolerance, and acceptable non-determinism. Moderate complexity in a single domain — don’t jump to multi-agent yet.

Real-world:

• SWE-bench coding agents (edit multiple files, run tests, read failures, retry)

• Adaptive learning (analyze student performance, pick focus areas and difficulty)

• Databricks’ help desk example: lookup_order(customer_id, order_id) returns “invalid order number” — the agent retries or asks the user for the correct ID, looping until it can give a final answer

Databricks’ framing is worth remembering: the single agent is the enterprise sweet spot — more flexible than a deterministic chain, far easier to debug than multi-agent.

Two enemies:

• Repeated invalid tool calls: the model calls the same tool for the same dead end (invalid order number → look up again → still invalid). Set iteration limits and timeouts; when a tool errors, decide whether the argument was wrong or you should ask the user.

• Tool overload: past roughly 10–15 similar tools, the model starts picking wrong ones. Fix names, parameters, and descriptions first; split into multiple agents only if that doesn’t help.

And the risks: higher cost, compounding errors, infinite loops. Set iteration limits and timeouts, and test extensively in sandboxed environments.

3: Multi-agent — the ceiling of autonomy

Multiple specialized agents collaborate. Three architectures by who’s in charge:

• Manager pattern (Supervisor): a central manager LLM coordinates experts via tool calls and is the only one facing the user.

• Decentralized pattern (Handoff): peer agents transfer execution — and conversation state — directly to each other.

• Custom topology: you define who talks to whom and how control flows — best for rule-heavy business processes, and the most controllable.

Try the graph view: a multi-agent system is a graph — agents are nodes, manager edges are tool calls, decentralized edges are handoffs. It also explains the framework divide: declarative frameworks (like LangGraph) make you draw the graph explicitly; code-first frameworks (like OpenAI’s Agents SDK) express it dynamically through program structure.

Use it when: domains diverge so much one agent can’t hold them (finance / devops / marketing); agents need independent contexts and tool sets; the tool schema no longer fits one agent; you want generate-and-verify collaboration.

Real-world:

• Scientific discovery (literature review / experimental design / testing / validation as separate agents)

• Large support centers: triage → line-of-business experts (shopping assistant, customer support, order management as independent agents)

• Multimedia storytelling: agents with different artistic specialties collaborate on storylines, characters, visuals, and music

Ask this before going multi-agent: do these subtasks really need separate agents? Every source agrees — max out a single agent first (enough tools, clear prompts). Multi-agent is the most expensive, hardest to debug, and has its own failure mode: agents ping-ponging tasks without resolution. MongoDB puts it bluntly: multi-agent is worth it only when you can’t predict the outcome of the task — and business scenarios where you can’t predict the outcome are rare.

Cross-cutting concerns: every pattern needs these

None of these care which pattern you picked — prompt chain or multi-agent, they all need to be addressed. Only the investment level differs.

Memory and retrieval

Two paths solve different problems: RAG answers “is the knowledge current and specific enough,” long-term memory answers “do you remember who I am and what we discussed.”

RAG (retrieval-augmented generation) injects external knowledge into context at inference time, and splits into two pipelines: offline ingestion (scrape → clean → chunk → embed → index) and online retrieval (vector + keyword hybrid search, fused with Reciprocal Rank Fusion). Chunking is the classic pitfall: chunks too small lose context; chunks too large dilute the semantic meaning of the embedding.

Long-term memory has two layers: short-term (the current session, kept in the prompt window, compressed with sliding windows or summaries when it overflows) and long-term (preferences and past decisions across sessions, stored in vector stores or databases, retrieved on demand). The hard part of memory isn’t storing — it’s what to write, what to read, and when to forget. Ten thousand uncurated notes inject noise that steers the agent off course — as bad as none.

Guardrails and human-in-the-loop

Layered defense: rules (regex, blocklists, length limits) → safety classifiers (jailbreaks, prompt injection) → moderation APIs (hateful, violent content) → relevance screening (an LLM judges whether the request is in scope) — any layer intercepts, the request is rejected before it reaches the main flow. OpenAI’s heuristic: cover data privacy and content safety first, then keep adding layers based on real incidents.

Human intervention triggers on two conditions: exceeding failure thresholds (retries keep failing to understand the user → escalate) and high-risk actions. Rate every tool by read-only / writable / reversible / financial impact — low risk runs automatically, extreme risk (cancel orders, large refunds, payments) goes to a human. A refund API and a weather API shouldn’t share a trust level.

Human-in-the-loop in the wild:

• Computer use (LLMs automating web tasks): logins, 2FA, and CAPTCHAs inherently require human interaction — they can’t be automated

• Health insurance claims: the LLM analyzes claims and gives detailed reasoning for approval or denial — but human adjusters make the final call

Also: natural language is inherently ambiguous, so letting humans clarify mid-conversation benefits every workflow, even when it’s not strictly required.

Evaluation and monitoring

Build evals before iterating (LLM-as-a-judge for the baseline, then calibrate with human feedback from experts and users — if your metrics don’t match human judgment, automated evaluation is self-congratulation). Trace every request, plan, and tool call (Databricks uses MLflow Tracing; multi-agent debugging runs on logs). Pin model versions — providers swap models behind your back and behavior drifts, a real incident type. Every LLM and tool call needs retry and fallback logic — timeouts, malformed responses, and empty results will break a workflow; fall back to a simpler chain when advanced features fail. Version your prompts too, so you can roll back and compare.

Two protocols worth knowing, both standardizing fast: MCP (Model Context Protocol) standardizes how agents connect to external tools — implement a tool once, use it everywhere; A2A (Agent-to-Agent) standardizes how agents talk to each other. They standardize the plugs between patterns — they are not patterns themselves, and they don’t replace the core decision of which pattern to pick.

Choosing: start simple, add complexity on evidence

Here’s the four sources’ (Databricks, MongoDB, Anthropic, OpenAI) collective conclusion, compressed into a decision table:

All sources converge on the same advice:

1. Start simple: use a deterministic chain when one will do. Don’t pay for imagined complexity.

2. Add complexity gradually: dynamic decisions → single agent; clearly separated domains or an overflowing tool set → multi-agent.

3. Combine patterns: a mostly-deterministic chain with one dynamic step often beats upgrading the whole system — real systems are almost always hybrids.

Anthropic adds three laws: keep it simple (the most successful systems skipped heavy frameworks — most patterns are implementable in a few lines of direct LLM API calls; use basic components before layering abstractions), be transparent (show the agent’s planning steps — users and developers can see why it did what it did, which is the basis of trust), and craft the ACI (the agent-computer interface deserves as much effort as a human interface — on SWE-bench they spent more time optimizing tools than the overall prompt).

Closing thought

Building agents isn’t an arms race. New patterns surface every week; chasing them isn’t a strategy. Focus on your requirements: pick the simplest architecture, evaluate it honestly, add components only on evidence. Between prompt chaining and multi-agent there are several rungs — and every rung has someone building a great product right now.

Sources:

• Anthropic, Building Effective Agents — anthropic.com/engineering/building-effective-agents

• OpenAI, A Practical Guide to Building Agents — platform.openai.com/docs/guides/agents

• MongoDB, 7 Practical Design Patterns for Agentic Systems — mongodb.com/resources/basics/artificial-intelligence/agentic-systems

• Databricks, Agent system design patterns — docs.databricks.com/aws/en/agents/agent-system-design-patterns

• Agentic Design Patterns (21-chapter translation) — adp.xindoo.xyz

• LLM System Design: The Complete Guide (2026) — RAG architectures chapter

If this issue was useful, subscribe — see you in the next one.

Associated with this newsletter, I posted my video on Youtube (Note it is Chinese). Welcome to watch it.

On July 18, 2026, Peter Steinberger — creator of OpenClaw — posted nine words on X:

“Are we still talking loops or did we shift to graphs yet?”

Nine words. Thousands of likes. Every serious AI builder recognized themselves mid-stride — one foot on the pattern they were leaving, one foot on the pattern they were reaching for.

Here’s the full decode: what graph engineering actually is, why loop engineering hits a structural ceiling, where the frameworks stand, and an honest answer to the loudest question on the timeline: is this just slop?

The Timeline: How “Graph Engineering” Ate the X Timeline

Here’s what happened, hour by hour (sources: AI Jason’s Graph Engineering Guide, Turing Post FOD#159, Carlos E. Perez’s analysis):

Crucially: nothing shipped on July 18. No framework suddenly supported something it couldn’t do the day before. So why did this word colonize the timeline?

Because throughout the first half of 2026, teams everywhere completed phase one of agent development — getting one agent to run in a loop — and then collectively hit the same wall: one loop can’t hold this work anymore. The tasks demanded role specialization, parallel execution, and explicit routing. Graph Engineering is just giving a name to a design upgrade that was already happening.

What Is Graph Engineering?

As AI Jason defines it in his canonical guide:

“Graph engineering is the practice of wiring multiple specialized agents or steps into a graph — nodes that do the work, edges that route between them, and shared state flowing along those edges — for when a single agent running one loop isn’t enough.”

Loop Engineering designs the cycle one agent repeats: discover → plan → execute → verify → repeat. You design the cycle and its stop condition.

Graph Engineering decides how several of those loops connect. Which specialized nodes exist, which edges route work between them, and what shared state travels along those edges.

A loop is the special case of a graph — a single node with an edge back to itself. This isn’t a replacement. It’s the layer directly above.

This Is Not New — And That’s the Point

When Harrison Chase said “it’s basically just LangGraph,” and when David Khourshid (creator of XState) rolled his eyes at “directed graphs” being announced as novel — they’re both right (and both quoted at length in AI Jason’s slop-discussion section).

Graph Engineering rests on three ancient CS foundations:

1. Directed Graphs — Dijkstra (1959) onward. From compiler control-flow graphs to database query plans, graph topology has always been the standard language for describing how work flows.

2. Workflow Orchestration — From BPMN to Airflow to Temporal, enterprise software has spent decades doing the same thing: defining dependencies and routing between steps. The DAG is the core abstraction of virtually every orchestration engine.

3. Cybernetics & Hierarchical Control — Norbert Wiener (1948) described that complex systems need multi-level feedback structures: fast operational loops wrapped by slower strategic loops. This is the original theoretical foundation for Loop → Graph, explored in depth by Carlos E. Perez in his essay.

Graph Engineering as a label is new (July 2026). As a practice, it’s the rediscovery of decades-old CS fundamentals in the Agent era. Just like “Design Patterns” weren’t invented by the Gang of Four — they just named practices that already existed.

The Five Layers of AI Engineering

@sairahul1’s framework (cited in AI Jason’s guide) is the cleanest map of where graph engineering sits:

These layers are cumulative, not sequential. A graph is full of nodes; a good node is a well-designed loop; a good loop needs a solid harness. Skip a lower layer, and the graph on top just fails in a more elaborate way.

Loop vs Graph: The Four Structural Failures of the Single Loop

Carlos E. Perez, in his essay on the transition, provides the deepest analytical frame: single-loop failures aren’t random — they’re consequences of the loop’s shape itself. There are four of them, and each has a topological answer in the graph.

⑴ Goodhart’s Law — When the Metric Becomes the Target

What happens: A customer-support chatbot optimizes “ticket resolution rate.” The bot learns to close conversations fast and discourage follow-ups. The number climbs for five months. Then renewal data arrives — churn has doubled. The loop faithfully optimized the metric, and the metric quietly detached from the reality it was supposed to represent.

Graph’s answer — Paired Metrics: One node optimizes the metric. A separate Reviewer node watches the counter-metric (renewal rate). Divergence triggers an alert. Perez calls this pairing — every optimizing loop gets a watching loop.

⑵ Blindness Upward — The Loop Cannot Question Its Own Target

What happens: A sales agent is told “send 500 personalized emails per day.” It does. Reply rate: 0.1%. The “500” target was set six months ago by instinct. Nothing inside the loop can ask whether the target is right. It’s a thermostat — it will faithfully control toward 68 degrees, but it will never ask whether 68 is the correct temperature.

Graph’s answer — Hierarchical Targets: A slower strategy node owns the downstream loop’s reference value. It runs every two weeks, analyzes reply rates and conversions, and dynamically adjusts the target — from “500 sends” to “five meaningful conversations per week.” Speed is separated so fast loops can’t thrash what slower loops steward.

⑶ Conflict — Independent Loops Fighting in the Same System

What happens: One loop optimizes response speed. Another loop optimizes reply quality. The speed loop makes responses shorter. The quality loop demands more detail. Each loop, examined alone, is working. Like two thermostats in the same room, one heating and one cooling, each performing beautifully by its own light.

Graph’s answer — Explicit Arbitration: An arbitration node sits above both loops and owns the trade-off. Urgent notification? Speed wins. Policy interpretation? Quality wins. It doesn’t optimize any single metric — it manages the boundaries between them.

⑷ Measurement Decay — Nobody’s Watching the Yardstick

What happens: A code-review agent runs for six months. Dashboard says “95% pass rate.” But the test data has aged — new framework versions, new coding patterns, new vulnerabilities. The loop is spinning, the data is rotting, and nobody’s watching.

Graph’s answer — Audit Loops: An independent audit loop doesn’t participate in daily execution. Its sole job: periodically sample real-world data and verify that “95%” still means quality. If not, it can freeze downstream outputs. Perez points out this is exactly the same principle as the held-out evaluation set in ML — a deliberately blinded loop whose whole job is to catch the optimizer gaming its own test.

The pattern across all four answers: one node does the work, another node watches it. That’s the structural essence of the graph — not more complex cycles, but relationships of constraint between cycles.

The practical decision framework comes from AI Jason’s comparison guide:

The Anatomy of an Agent Graph

Strip the jargon and an agent graph has exactly three parts (from AI Jason’s Guide):

Nodes

The units that do the work. A node is usually a specialized agent (researcher, writer, reviewer) or a plain deterministic step (function call, data fetch). Each node has one job.

Edges

The routing between nodes. An edge says after this node, go to that one. Edges can be straight (A → B), conditional (if review passes → ship; if not → loop back), fan-out (one node kicks off three in parallel), and fan-in (three results join into one).

Shared State

The object that travels along the edges. Every node reads from it and writes to it: the task, the draft so far, notes, verdicts. As AI Jason puts it: “State is what turns a pile of agents into a system instead of a group chat that forgets everything.”

The Org-Chart Metaphor

@rohit4verse’s framing is the one every analyst has adopted — from AI Jason to Perez to Turing Post: a company org chart. A company doesn’t make one person do research, writing, and review in a single unbroken stint. It gives those to different roles, routes work between them, and lets results roll back up. An agent graph is the same idea: specialized roles, defined hand-offs, a shared record.

Canonical Example: The Daily Research Brief

From AI Jason’s Guide, here’s the canonical starter graph:

Above: Agent Graph example — Input enters the Researcher node (fan-out over N sources in parallel, outputs structured notes), notes flow along the edge to the Writer node (sees only clean notes, produces draft), draft flows to the Reviewer node (independent context, different model, read-only judgment). If pass → ship ✅. If reject → route back to Writer via conditional edge 🔄. The Shared State object flowing along edges is shown at the bottom.

What the graph buys that the single loop couldn’t:

1. Clean, separate contexts — Writer never sees raw HTML; Reviewer runs in a completely independent context

2. Real review instead of self-rubber-stamping — “don’t let an agent self-verify,” promoted to its own node

3. Parallel source gathering — Researcher’s fan-out is something a single loop structurally cannot do

4. Auditable control flow — the routing logic is defined up front and inspectable as a diagram, rather than reconstructed from one long transcript

Not to Be Confused With: Knowledge Graphs or GraphRAG

“Graph” is an overloaded term. AI Jason’s guide explicitly separates Graph Engineering from two unrelated concepts:

Same word, completely unrelated problem domains. They can coexist — your agent graph’s researcher node might call a GraphRAG for retrieval — but they solve orthogonal problems.

Two Sides of the Same Coin: Agent Network × Ontology Foundation

The task-driven Multi-Agent network (the Graph Engineering layer) and the Knowledge-Graph-based Ontology layer are not either/or — they’re the upper and lower layers of one system, and both layers are “graphs”:

Notice that both layers are networks:

• The upper layer is an execution-flow network — agents are nodes, control flow is the edges (notes / draft / if reject / if pass / new task). It answers “who does what, in what order, and how work is handed off.”

• The lower layer is a semantic-relation network — business objects are nodes, semantic relations are the edges (place order / contains / invoice / supplies / stocks / ships). It answers “what exists in the enterprise, and how these things relate.”

The most vivid metaphor: a set of multi-agents is a team of virtual employees; an ontology is the virtual form of the enterprise. Employees do the work — research, writing, review, decisions; the enterprise form defines the business objects, relations, rules, and actions employees are allowed to see. The upper layer gets re-orchestrated constantly (3 nodes today, 5 tomorrow), but no matter how it’s rewired, everyone reads and writes the same Ontology — employees can be replaced; the enterprise skeleton stays.

The Tooling Landscape

You don’t need to invent the runtime from scratch. These frameworks shipped the node-edge-state model well before the term “graph engineering” trended (sources: AI Jason’s Guide, LangGraph docs, Google ADK docs):

LangGraph (LangChain)

“A low-level orchestration framework and runtime for building, managing, and deploying long-running, stateful agents.”

• How it maps: Define a StateGraph → add nodes → add edges → state passes along edges

• Built-in: conditional edges, fan-out/fan-in, checkpointing, human-in-the-loop, streaming

• Best for: complex agents requiring fine-grained control flow and state management

Microsoft AutoGen — GraphFlow

• How it maps: Describe how a team of agents connects and hands off, rather than running one in isolation

• Built-in: sequential, parallel, conditional, and looping flows (DiGraph model)

• Best for: Microsoft ecosystem, scenarios requiring multi-agent conversation and negotiation

Google ADK + A2A Protocol

• How it maps: “Orchestrate complex tasks through structured, graph-based architectures”

• Built-in: sequential/parallel/loop workflow agents, agent routing, A2A cross-system delegation

• Best for: Google Cloud ecosystem, cross-organizational agent collaboration (LinkedIn deployed A2A in 2025)

The Golden Rule

AI Jason puts it plainly: pick a framework instead of hand-rolling. Reinventing the node-edge-state runtime is its own kind of slop.

Honest Assessment: Is This Just Hype?

The backlash arrived almost before the term did — and the critics aren’t cranks. They’re some of the people who know this domain best (all quoted in AI Jason’s slop section, Turing Post’s FOD#159, and Graph Engineering vs Loop Engineering):

• Rhys Sullivan called the slop-article wave before it landed — and then dryly noted that it had arrived

• David Khourshid (creator of XState) reminded everyone that state machines with nodes and transitions are decades old

• Paweł Huryn argued that naming the mechanism (loops, graphs) keeps missing the substance (objectives and verification)

• Harrison Chase (creator of LangGraph) himself wasn’t sure the word named anything new

• Nathan Flurry pointed out that LinkedIn deployed multi-agent graphs in 2025 — this Twitter coinage was late, not early

Where I Land: Concede Three Things, Hold Two

Concede:

• ✅ The mechanics aren’t new — directed graphs, state machines, and orchestration engines predate the buzzword by a year or more

• ✅ Much of the content riding the term is slop — the content-farm gold rush arrived exactly on schedule

• ✅ The word “Graph Engineering” is optional — you can build every system described here and never once use the phrase

Hold:

• 🔹 Teams genuinely moving from “one agent in a loop” to “multiple specialized agents coordinated over shared state” — this is real

• 🔹 Choosing nodes, edges, and state is a design skill distinct from designing a single loop — this is also real

The Bottom Line: Grounded vs. Ungrounded

Graph Engineering makes the coordination relationships in complex AI systems explicit, programmable graph structures — turning implicit prompts and manual coordination into something more reliable, observable, and scalable.

It carries buzzword baggage — even its proponents admit that. But underneath is the real trajectory of agents moving from solo operators to coordinated organizations.

Carlos E. Perez closes his essay with the insight that outlasts the hype cycle:

“The durable axis was never loops versus graphs at all. It is ungrounded versus grounded: whether the improvement machinery, however shaped, keeps touching the reality it claims to improve.”

The deep question isn’t “Loop or Graph?” — it’s “Does your improvement machine, whatever its shape, still touch the world?” Do your watchers have genuine independence? Do your frozen rules stay frozen under pressure? Do you admit that your deepest targets were chosen, not computed?

References

1. AI Jason (AI Builder Club). Graph Engineering Guide (2026) — The pillar guide covering concepts, frameworks, controversy, and a starting checklist

2. AI Jason (AI Builder Club). Graph Engineering vs Loop Engineering — The boundary between the two disciplines and the “did you actually change paradigms?” gut-check

3. Carlos E. Perez (Intuition Machine). From Loop Engineering to Graph Engineering? — Cybernetics-grounded analysis of why single loops fail and how graphs answer

4. Ksenia Se (Turing Post). FOD#159: Is Graph Engineering Real? Why Everyone Is Talking About It — The industry snapshot and hype-cycle analysis

5. Peter Steinberger. Original X post — The nine words that started it all

6. LangGraph Overview (LangChain docs) — Official documentation

7. Agent Development Kit (Google ADK) — Official documentation

8. Harrison Chase. “it’s basically just LangGraph?” — LangGraph creator’s reaction

9. Shann Holmberg. Loops vs Graphs — “The difference is who decides the path, the agent or you”

10. Dale Everett. “Loops are just shitty graphs” — The counter-position

Welcome to subscribe my newsletter · Making AI engineering knowledge accessible
If you’re building AI agents and finding that a single loop isn’t enough anymore — try thinking in graphs. But first, check whether one loop was enough. Most of the time, it is.

Associated with this newsletter, I posted my video on Youtube (Note it is Chinese). Welcome to watch it.

Here’s a scenario you’ve probably encountered: the same “customer” exists as Sales Customer in your CRM, Support Customer in your ticketing system, and Billing Customer in your finance platform. Which one is the real customer?

The problem starts at the design level — you’re modeling source systems, not reality.

Palantir distilled years of field experience across government and commercial implementations into four core principles — priority-ordered, higher wins in conflict — and seven anti-patterns. Let’s walk through them.

Principle 1: Domain-Driven Design (Highest Priority)

Model the real world, not the source data. The correct order is non-negotiable:

1. Understand the domain — identify real-world entities and relationships

2. Design the ontology — define object types, properties, links, interfaces

3. Map source data — pipe data in through transforms

❌ Anti-Pattern: Kitchen Sink

Take a CSV with order_id, customer_name, customer_email, product_sku, and quantity. The instinct is to create one OrderData object type with five properties — including technical metadata like _crm_extracted_at.

✅ Fix

That single CSV contains three entities. Split them:

Order { orderId, quantity }   → links to Customer, Product
Customer { name, email }
Product { sku }

🎯 Litmus test: A user should navigate your ontology intuitively — no docs required — because the structure matches how they already think about the domain.

Principle 2: Don’t Build the Same Entity Twice — The Rule of Three

Every concept has exactly one canonical representation. One occurrence is coincidence. Two is a pattern. Three — refactor.

❌ Anti-Pattern: Department Silos

Sales creates Sales_Customer. Support creates Support_Customer. Finance creates Billing_Customer. All share name, email, phone plus department-specific fields. Three code paths, three maintenance burdens, inconsistent data.

✅ Fix

Option A (preferred): Merge into a single Customer type.

Customer { name, email, phone, lifecycleStage, department }

Option B (when shapes genuinely differ): Extract a shared interface.

interface CustomerBase { name, email, phone }
SalesLead implements CustomerBase { dealSize: number }
SupportContact implements CustomerBase { ticketHistory: Ticket[] }
BillingAccount implements CustomerBase { paymentMethod: string }

💡 An Interface defines a shared contract. Multiple types implement it, so shared logic is authored once and specializations live where they belong.

Principle 3: Open for Extension, Closed for Modification

Core models lock after validation. New capabilities arrive via extensions — never by adding fields to the core.

❌ Anti-Pattern: God Object

Equipment started with serialNumber, model, location. Then certification added 4 fields, finance added 3 depreciation columns, maintenance added 5 more. Result: 150+ properties, most null, and every team risks breaking everyone else’s applications.

✅ Fix

Keep core Equipment lean. Certification info? Create a linked EquipmentCertification type. Inspectability? Implement the Inspectable interface. Each capability is an independent module.

Principle 4: Composition Over Deep Hierarchies

Foundry’s interfaces support multiple inheritance. An entity can be both a Building and a SchedulableResource simultaneously.

• Deep inheritance: Asset → PhysicalAsset → Building → SchedulableBuilding → Arena (5 levels)

• Composition: Arena implements Building, SchedulableResource (2 steps)

A scheduling workflow built against SchedulableResource works for arenas, conference rooms, and vehicles without modification. Deep hierarchies can never deliver this.

4 Structural Patterns

Beyond principles, four patterns act as the “grammar” of your ontology:

Anti-Patterns Gallery

Beyond those paired with principles, four more trap designers at every experience level:

Decision Framework: Pick the Right Tool

Before implementing, run through five questions:

1. Requires human judgment? → Action Type

2. Data transform or aggregation? → Pipeline (Streaming if low-latency, else Batch)

3. “Something changed → something should happen”? → Automation

4. Complex cross-object real-time computation? → Function or Derived Property

5. Simple, stable-input derivation (e.g. fullName)? → Pre-compute in Pipeline

Naming & Security: The Two Hardest Things to Fix Later

Naming: Singular, concrete nouns for object types (Patient, not Data). Self-documenting properties (lastInspectionDate, not dtLastInspMod). Consistent date suffixes. Bidirectional link readability (department ↔ employees).

Security: Model with semantic policies, not duplicated types. One Patient type with row-level (VIP patients) + column-level (clinical notes) + cell-level (VIP clinical notes) restrictions. Never create PublicPatient and RestrictedPatient as separate types.

Beyond Palantir: Industry Practices

Five complementary practices from academic research and enterprise architecture:

1. Entity Classification — Categorize every type as Business Entity (noun), Process Entity (verb), or Result Entity (output) before modeling

2. Upper Ontologies — Use frameworks like gist to avoid rebuilding fundamentals (Person, Organization, Event, Time)

3. Semantic Lifting — Map existing BPMN/ER diagrams to ontology entities via annotation; preserve prior investments

4. RACI Governance — Every entity and property needs an Owner, Steward, and Sponsor

5. Central Governance + Distributed Population — Core model centrally controlled; domain teams own extensions and data

The Pragmatist’s Principle

An imperfect ontology delivering real value today beats a theoretically perfect one still being designed.

But — naming quality, semantic clarity, and security design are the three things that are punishingly expensive to fix later. Don’t compromise on those. Everything else? Ship it, then iterate.

📚 Sources
Palantir Ontology Best Practices · Structural Guidance · Anti-Patterns · gist Ontology (Semantic Arts) · Industry 4.0/5.0 Ontology Framework (Abonyi, Nagy, Ruppert · Springer 2024)

Associated with this newsletter, I posted my video on Youtube (Note it is Chinese). Welcome to watch it.

A few recent videos on Ontology-driven enterprise Agent deployment got more traction than expected. Since then I’ve talked to a mix of people: founders building in this space, engineers trying to stand up a pilot inside their own company, people whose bosses asked them to put together a proposal, and practitioners who just want to learn the concept properly. Given limited bandwidth, I could only take a handful of these conversations one-on-one — but the questions kept converging on the same few points. Here’s the consolidated answer, so I stop repeating myself.

What problem does Ontology actually solve?

The most common misread is treating Ontology as a knowledge graph database. That’s not wrong exactly — Ontology does cover graph-storage functionality — but it’s not where the value sits.

The core of Ontology is a description of the business itself: it encodes the objective facts and relationships that exist in a business domain, carrying the semantic information that actually matters. A customer places an order. The order contains products. Products are made of components. Components get sourced from specific suppliers. That entire chain of semantic relationships is exactly what Ontology is meant to capture.

Why does this semantic layer matter? Because an Agent can only automate work and support decisions reliably inside a business domain once it understands this web of relationships. Without that foundation, Agent output tends to fail in one of two ways: hallucination, or answers so generic they ignore the industry and company context — and worse, inconsistent answers to the same question asked twice.

Based on what Palantir has validated in practice so far, Ontology is the infrastructure that lets an Agent make accurate decisions at the business-domain level — effectively an operating system for the Agent, an AgentOS. That’s the path proven today; it doesn’t rule out someone finding a different one later.

What tools exist for building an Ontology? Is anything open source?

A number of companies in China are already exploring this space, and some have shipped their own AgentOS-style products — ranging from vibe-coded demos to reasonably mature versions, most of them adaptations or simplifications of Palantir’s approach.

It’s hard to say any single tool is definitively better or more authoritative, but there are open-source options worth looking at:

• OntoFlow: https://github.com/H2020-OpenModel/OntoFlow

• OntoEKG: https://github.com/LiberAI/OntoEKG

• nano-ontoprompt, a lightweight version I built myself: https://github.com/jingw2/nano-ontoprompt

Finance also has ready-made open ontologies worth studying for structure:

• FIBO: https://github.com/edmcouncil/fibo

• BIAN: https://en.wikipedia.org/wiki/Banking_Industry_Architecture_Network

One framing worth flagging here. Bob McGrew talked on a podcast about how AgentOS-type products essentially scale something that’s inherently hard to scale. The FDE (Forward Deployed Engineer) model is a gravel road by nature — hard for a company to replicate at scale. AgentOS-style products pave a highway over that gravel road: they bake FDE domain knowledge and delivery experience into the product itself, so the same capability can serve more clients without rebuilding from scratch every time.

Worth a caution though: I’ve talked to a lot of companies that are chasing the trend more than solving a problem — they heard the concept was hot, skipped straight to “what tool should we use,” and never stopped to ask whether they should be building this at all, or whether it fits their situation. That’s the order backwards.

Does your company actually need to build an Ontology?

For most small and mid-sized companies, my honest recommendation is: don’t invest in this. Two reasons.

First, the combined investment of building an Ontology and the Agents on top of it is not small, and it has to be a top-priority initiative sponsored directly by leadership — not something a technical team can push through on its own. Second, small and mid-sized companies typically don’t have complex enough processes or large enough headcount for the ROI on Ontology + Agent infrastructure to make sense.

The real test isn’t “is this concept trending” — it’s whether your company genuinely needs a business-domain-level Agent Runtime to drive efficiency, or to close the leakage and misaligned incentives that come from too much manual intervention in the process. If that need doesn’t exist, the Ontology investment is wasted effort.

Want to start, but don’t know where?

Right now, most companies’ Agent deployments sit at the individual-productivity level: things like Lark’s Ally, multi-dimensional tables, a finance reconciliation assistant, writing weekly reports, generating slides. Slightly more advanced setups build workflows — listening to an inbox, downloading attachments, parsing them, and writing the results back into a system.

The pattern across almost all of these: the Agent never touches the core of the business domain, and none of it is systematized. The bottleneck is usually not technical capability — it’s that leadership doesn’t see it. They don’t know this is possible, or don’t know how to approach it.

So step one is always winning over leadership — making the top decision-maker understand the concrete value Ontology + Agent can create. That buy-in is the precondition for pushing domain-level implementation across the company. Only after that should you pick a high-value process — sales, procurement, finance, supply chain — as a pilot. Choose something where ROI can be validated quickly, not an attempt to build an enterprise-wide Ontology on day one.

If this was useful, forward it to anyone on the fence about whether to build an Ontology. Happy to talk through implementation paths for specific industry scenarios in the comments.

Associated with this newsletter, I posted my video on Youtube (Note it is Chinese). Welcome to watch it.

If you’re building enterprise AI agents, you’ve hit the fundamental question: how should an LLM safely access enterprise data?

The naive answer — give it database credentials and let it write SQL — is terrifying. Hallucinations fabricating numbers. Privilege escalation into restricted tables. Sensitive data leaking through prompts. Pick your nightmare.

The RAG answer — wrap a retrieval layer around the LLM — solves part of the problem. RAG helps the LLM see more relevant information. But RAG doesn’t solve what the LLM must not see, and it certainly doesn’t solve what the LLM is allowed to do.

Palantir’s answer operates at an entirely different level of sophistication.

They don’t insert a retriever between the agent and the data. They first build a complete enterprise business semantic layer — called Ontology — modeling every noun, relationship, rule, action, and permission. Then they let the agent operate on that governed layer, not on the raw data beneath it.

This article deconstructs the entire mechanism. Drawn from Palantir’s official documentation across AIP Architecture, Chatbot Studio Tools, Retrieval Context, AIP Logic, Tool Calling, Palantir MCP, and Ontology MCP — every layer gets detailed treatment with concrete configuration examples, operational analogies, and architectural analysis.

The Core Thesis

In Palantir’s architecture, agents do not access underlying database tables directly, nor do they freely scan the complete Ontology schema. They operate within a configured and authorized boundary — interacting through Ontology-exposed objects, relationships, contexts, functions, and actions.

One sentence captures the relationship: The Ontology is the governed semantic interface to the enterprise business world. The Agent is the operator that uses natural language to work through these interfaces.

The interaction formula:

LLM + Retrieval Context + Tools + Ontology + Governance

The complete data flow:

User Message → Retrieval Context injection → LLM reasoning → Tool invocation → Ontology query/compute/mutate → Governance constraint & audit

Five components, each with a distinct responsibility:

• LLM — Understands user intent and decides next steps. Handles reasoning and decision-making, not direct data access.

• Retrieval Context — Deterministically provides necessary business context on every user message. Prevents the agent from starting “blind” and from having to decide on its own what to retrieve.

• Ontology — The semantic model of business objects, relationships, functions, and actions. The “business language” of the entire system.

• Tools — Converts LLM intent into executable queries, function calls, or business operations. The LLM decides what to do; the Tool executes how to do it.

• Governance — Constrains what the agent can see, invoke, and modify. Logs audit trails and lineage. Spans every layer.

This design philosophy — governed freedom — echoes through every layer that follows. You’ll see it in the deterministic-vs-autonomous split between Context and Query. You’ll see it in the Function-vs-LLM division of labor. And you’ll see it most powerfully in the Governance layer that constrains everything.

What Is Ontology to an Agent?

Many hear “Ontology” and think knowledge graph or data catalog. Palantir Ontology is something different: an operational layer — sitting above datasets, virtual tables, and models, mapping raw digital assets into real-world business objects.

Palantir’s official Ontology documentation uses two categories of terms. This article builds on that classification, interpreting them as “describing the business world” and “changing the business world.”

Semantic Elements: Describing the Business World

Object Types:  Shipment, Order, Warehouse, Customer, Carrier
Properties:    Shipment.status, Shipment.ETA, Order.priority, Carrier.performanceScore
Links:         Shipment belongsTo Order, Order belongsTo Customer, Shipment handledBy Carrier

These elements answer: How does the agent understand what exists in the enterprise, and how are these things related?

The agent doesn’t infer from training data that a Shipment has a status or that it belongs to an Order. The Ontology defines these relationships explicitly and authoritatively. When the agent reasons about “delayed shipments for VIP customers,” it’s reasoning over governed semantic objects, not guessing from column names.

Kinetic Elements: Changing the Business World

Actions:    updateShipmentETA, assignCarrier, createExceptionCase, approveSchedulePlan
Functions:  calculateDelayRisk, recommendCarrier, summarizeCustomerIssue, optimizeSchedule

These elements answer: How does the agent move from answering questions to executing business operations?

Actions perform Ontology writes — updating data, changing state. Functions encapsulate business logic — computing, model-invoking, recommending, optimizing. Together they give the agent a complete capability loop: understand → query → analyze → decide → execute.

The key is not “renaming database tables.” It’s placing enterprise nouns, relationships, rules, actions, and permissions into a single governed semantic layer. This architectural decision reverberates through every subsequent layer.

One word: Ontology is an Operation System for AI Agents.

The Five-Layer Interaction Architecture

Agent-to-Ontology interaction decomposes into five layers, each building on the last:

Layer 1: Context Layer       Let the agent see relevant context first [System deterministic trigger]
Layer 2: Query Layer         Let the agent actively query objects and relationships [LLM decision]
Layer 3: Logic Layer         Let the agent invoke functions, models, and business logic [LLM decision]
Layer 4: Action Layer        Let the agent execute governed business operations [LLM decision + optional user confirmation]
Layer 5: Governance Layer    Constrain agent permissions, audit, and security boundaries [End-to-end]

The trigger annotations are the most important information in this table:

• Layer 1 is system-determined. The agent doesn’t decide whether to retrieve; the system does it automatically.

• Layers 2-4 are LLM-determined. The agent autonomously decides what to query, which function to call, and what action to execute.

• Layer 5 is end-to-end. It’s not a separate step but a constraint mechanism spanning Layers 1 through 4.

This architecture embodies a single design philosophy: determinism where possible, autonomy where necessary, constraint everywhere. The system does what it can do reliably. The LLM does what requires intelligence. But everything the LLM can do is bounded by governance.

Layer 1: Context Layer (Retrieval Context)

The Context Layer automatically provides relevant background to the agent on every user message.

Critical characteristic: deterministic trigger. It executes automatically on every message — the LLM does not participate in the decision of whether to retrieve. This is the fundamental design distinction from the Query Layer (Layer 2).

Why this design? Because if you let the LLM decide “do I need to look something up?”, it might forget to look, look for the wrong thing, or look incompletely — and then fabricate answers based on incomplete information. Deterministic triggering guarantees: every message arrives with the context it should have.

Palantir Chatbot Studio provides three types of Retrieval Context.

Note: these are not prompt-level instructions like “you can look up X.” They are actual configurations in the Chatbot builder that the system executes automatically.

Ontology Context

Retrieves objects from Object Types. Two modes:

Mode 1: Fixed Object Set

Passes a set of objects through an application state variable. The canonical use case is Workshop-embedded agents — the user is operating on an object in the UI, and the agent automatically receives that object’s context.

Example: Warehouse Dispatch Assistant

Configuration:
  Object Type: Warehouse
  Method: Variable Input (bound to currently selected Warehouse in Workshop)
  Properties passed: warehouseId, region, currentCapacity, backlogCount, managerName

Effect:
  User selects "South China Warehouse" in Workshop, then opens the agent
  → Agent automatically receives capacity, backlog, manager info for that warehouse
  → User asks "How's the backlog situation?"
  → Agent already has context — no need to ask "Which warehouse are you referring to?"

Mode 2: Semantic Search

Relies on vector embedding properties on Object Types. Finds the K most semantically relevant objects based on the user’s query.

Example: Customer Complaint Assistant

Configuration:
  Object Type: CustomerComplaint
  Method: Semantic search, K=5
  Properties passed: complaintId, customerName, issueCategory, resolution, satisfactionScore

User asks: "Have there been complaints about cold chain failures causing cargo damage?"

System automatically:
  → Vectorizes the query
  → Searches against CustomerComplaint embedding properties
  → Returns 5 most semantically relevant historical complaints
  → Injects into LLM context

The LLM receives these 5 complaint records and directly summarizes historical handling approaches and customer satisfaction.

Property filtering is configurable — vector embeddings themselves, large text fields, and sensitive fields can all be excluded from what reaches the LLM. This balances contextual relevance with token efficiency.

Document Context

Provides context from documents. Not a simple “attach the file” approach, but a strategic retrieval:

• Full-text mode: Passes the complete document content. Suitable for short documents or when full reading is necessary.

• Relevant-chunks mode: Returns the K most semantically relevant chunks. Suitable for long documents where only specific sections matter.

Example: Carrier Contract Q&A Assistant

Configuration:
  Document source: Transport service contracts with three carriers (PDF)
  Retrieval mode: Relevant Chunks, K=5

User asks: "What is Carrier A's delay compensation clause?"

System automatically:
  → Performs semantic search across all three contracts
  → Returns the 5 text fragments most relevant to "delay compensation"
  → Injects into LLM context

The LLM can cite specific clause numbers and source positions when answering.

Function-backed Context

When built-in retrieval isn’t sufficient, you write a TypeScript function implementing the AipAgentsContextRetrieval interface with custom retrieval logic. The function outputs a retrievedPrompt string, which enters the conversation’s system prompt context.

Additional parameters can be passed via application variables (currently supporting string and object set types).

Critical detail: this does not mean the full Ontology schema is injected into the system prompt. Only the retrieval function’s generated context text enters the conversation’s system prompt context. What’s exposed is the retrieval result, not the schema.

Example: Customer 360° View Assistant

Scenario: User asks "How has this customer been doing recently?"
  - Ontology Context can only find orders
  - Document Context can only find contracts
  - But the user wants orders + complaints + contract terms + recent communications — a composite view

Custom function logic:
  1. Get current customerId from application variable
  2. Keyword-search CRM communication records for last 30 days
  3. Ontology query for active orders
  4. Semantic search for complaints and tickets related to this customer
  5. Assemble a structured brief:
     "Customer XX Electronics: 12 active orders, 2 delayed;
      Last 30 days: 3 complaints, primarily about timeliness;
      Contract expires 2026-09-30, current service level: Gold"

  → Injected into LLM context

Function-backed Context is designed for complex enterprise scenarios requiring multiple data sources, multiple retrieval strategies, and multiple business rules — scenarios that Ontology Context and Document Context alone cannot cover.

It can be simply seen as you predefined a custom function for some frequent retrievals such as customers-related questions.

Layer 2: Query Layer (Object Query Tool)

Context is system-injected. Query is agent-initiated.

The Object Query Tool allows the LLM to access specified Object Types and Properties within configured boundaries, supporting filtering, aggregation, inspection, and traversal of links.

A crucial distinction: queries operate on Ontology semantic objects and links, not SQL JOINs. The underlying implementation may still rely on datasets and indexes, but the agent’s interface is business objects — “Shipment,” “Carrier,” “Warehouse” — not tables and columns. This allows the LLM to drive queries through natural language reasoning rather than constructing SQL it’s notoriously bad at.

Pattern 1: Multi-Condition Filtering + Aggregation

User asks: "How many delayed high-priority orders are in the South China warehouse today?"

Agent invokes Object Query:
  Object Type: Shipment
  Filter: warehouse.region = "South China"
          AND status = "delayed"
          AND linkedOrder.priority = "high"
          AND ETA < today
  Aggregation: COUNT

→ Returns: 7

User follows up: "Which of these 7 are VIP customers?"

Agent invokes Object Query again:
  On the previous result set
  Traverse links: Shipment → Order → Customer
  Filter: Customer.serviceLevel = "VIP"

→ Returns: 3, with customer names and order IDs

Note two distinct operations: first, conditional filtering with aggregation producing a count. Then, link traversal through a relationship chain to filter by customer tier. The entire process is LLM-autonomous, but bounded within configured Object Types and Properties. Like an executive flipping through materials during a meeting — the materials are on the table, but which pages to turn to and how to filter them is the executive’s decision.

Pattern 2: Link Traversal for Root Cause Analysis

User asks: "What caused the delay on order ORD-8832?"

Agent invokes Object Query, traversing links layer by layer:
  Order ORD-8832
    → link: hasShipments → Shipment SH-0042
      reads: status = "delayed", delayReason = "vehicle breakdown"
      → link: handledBy → Carrier CarrierA
        reads: performanceScore = 72, recentDelayRate = 18%
      → link: routedThrough → TransitHub HUB-GZ-03
        reads: congestionLevel = "high", avgProcessingDelay = "4.2h"

→ Agent synthesizes: delay caused by Carrier A vehicle breakdown + Guangzhou transit hub congestion

This is Object Query’s most powerful capability — link traversal. The agent doesn’t need to know the full picture upfront. It can investigate like a detective following a chain of clues: from order to shipment, from shipment to carrier, from carrier to transit hub. Each step is “follow the relationship down” rather than constructing a complex multi-table JOIN upfront.

Pattern 3: Aggregation for Operational Overview

User asks: "Compare backlog situations across regions?"

Agent invokes Object Query:
  Object Type: Shipment
  Group By: warehouse.region
  Aggregation: COUNT where status = "pending_dispatch"

→ Returns:
  South China: 42 orders
  East China: 28 orders
  North China: 15 orders
  Southwest: 8 orders

This is management getting a “traffic light” dashboard — not inspecting individual orders, just seeing aggregate numbers by region to identify where attention is needed. The agent isn’t just “able to look up data”; it can provide management-level perspectives through aggregation.

The Critical Distinction: Retrieval Context vs. Object Query

This is the most commonly confused design point, and the most commonly overlooked in third-party analyses.

DimensionRetrieval ContextObject Query ToolTriggerAuto-fires on every user messageLLM decides whether to invokeReliabilityDeterministicDepends on LLM judgment; risk of missed or wrong queriesBest forCurrent business objects, key background, must-know informationExploratory analysis, ad-hoc queries, multi-turn follow-upsRiskToken waste or context noise if over-configuredLLM may forget, query incorrectly, or query incompletely

Design principle (this article’s recommendation, not official wording):

Information the agent MUST know → Retrieval Context
Information the agent MAY explore → Object Query Tool

A concrete example — the same “warehouse operations assistant” — shows what goes where:

In Retrieval Context (system-injected, every turn):

• The Warehouse object the user is currently operating on

• Today’s KPI snapshot for that warehouse (outbound volume, backlog count, exception count)

• User role and permission level

→ Ensures the agent always knows who it’s talking to and about which warehouse. No need for the user to restate context.

In Object Query Tool (LLM invokes on demand):

• “Compare South China and East China warehouse backlogs” → Agent queries two Warehouse objects

• “This carrier’s performance over the last 30 days?” → Agent queries Carrier object with aggregation

• “Where is SH-0042 right now?” → Agent queries a specific Shipment

→ These are ad-hoc, exploratory queries that couldn’t be predicted in advance. You don’t know what the user will ask, so you don’t pre-inject.

Analogy: Retrieval Context is the monitoring dashboard fixed at your workstation — always on, always showing key metrics. Object Query is the walkie-talkie in your hand — you pick it up when you need to ask “what’s the situation at Warehouse 3 right now?”

What happens when you get this wrong? Overstuff Context with everything just in case → every conversation turn carries potentially irrelevant data, wasting tokens and creating noise that distracts the LLM. Configure Query too narrowly → LLM wants to look something up but can’t find it → either admits ignorance (good) or, worse, fabricates (bad).

Layer 3: Logic Layer (Function Tool + AIP Logic)

After querying objects, agents often need to invoke encapsulated business logic. Don’t let the LLM “guess” the rules.

The Function Tool allows the LLM to invoke any Foundry function, including published AIP Logic functions. AIP Logic is a no-code environment for building, testing, and publishing LLM-driven functions. Inputs can be Ontology objects or text strings; outputs can be strings, objects, or Ontology edits.

An important product boundary: further automating, staging, or routing function results through human review typically requires combining Automate, Ontology edits, staged writes, or application-side workflow design. This should not be understood as the Function Tool natively providing a complete approval flow; it should be addressed per specific Palantir product capabilities.

Division of responsibility: LLM handles intent understanding, function selection, and result explanation. Functions handle business rule execution, model invocation, optimization computation, and structured result return.

Example 1: Risk Scoring — Why the LLM Shouldn’t Calculate

This example best demonstrates the Function Tool’s value.

User asks: "What's the risk level for these delayed orders?"

Without Function:
  The LLM makes a fuzzy judgment based on ETA and status
  "Looks delayed, probably risky" — not an enterprise-grade answer

With Function:
  Agent calls calculateDelayRisk(shipments)
  Function internal logic:
    - Query historical delay data (past 90 days, same carrier, same route)
    - Read carrier's current performance score
    - Call weather API for destination's next 24h forecast
    - Run a trained delay probability prediction model
    - Composite scoring: high / medium / low

  Returns:
    SH-0042: high (carrier breakdown + route historical delay rate 22%)
    SH-0119: high (warehouse backlog + destination storm warning)
    SH-0203: medium (slight overtime but carrier score good)

  → LLM receives structured results and explains them in natural language

Example 2: Cross-Object Business Logic — Email + Object Joint Reasoning

Scenario: Customer complaint handling

Agent calls recommendResolution(shipment, emailText)
  Inputs:
    - Shipment object SH-0042 (with status, delayReason, carrier, etc.)
    - Customer complaint email text: "Already delayed 3 days, we'll return the goods if it doesn't arrive soon"

  Function internal (AIP Logic):
    1. Use LLM block to extract sentiment, demands, and time constraints from the email
    2. Query Ontology for this customer's past 6 months of complaint records
    3. Match historically similar delay + similar customer tier resolution cases
    4. Synthesize a recommendation

  Returns:
    Recommended action: "Reassign to Carrier B + proactively offer 5% freight discount"
    Basis: "Historically, for similar VIP customer complaints, Approach A had 87% satisfaction recovery rate"

  → Agent presents the recommendation to the user with execution options

This example demonstrates AIP Logic’s power — combining the LLM’s natural language understanding (extracting sentiment and demands from email) with Ontology’s structured query capability (historical complaint data matching), producing an evidence-backed recommendation. Like a customer service supervisor turning to an experienced colleague: “Help me look up how we handled similar situations before — what was this customer’s tier, which approach worked best historically?”

Example 3: Optimization Computation — Scheduling / Routing / Resource Allocation

User asks: "Tomorrow, 50 pending orders at South China warehouse — how to optimally allocate across 3 carriers?"

Agent calls optimizeCarrierAllocation(orders, carriers, constraints)
  Inputs:
    - 50 Order objects
    - 3 Carrier objects (with capacity limits, unit prices, historical fulfillment rates)
    - Constraint: minimize total cost + all orders must ship within 48h

  Function internal:
    - Run a linear programming / mixed-integer programming model
    - Return allocation plan

  Returns:
    Carrier A: 22 orders (heavy cargo primarily, lowest unit price)
    Carrier B: 18 orders (time-sensitive items, highest fulfillment rate)
    Carrier C: 10 orders (remote regions, only coverage)
    Estimated total cost: ¥84,200

  → LLM summarizes the plan logic in natural language; user confirms to execute the allocation

Layer 4: Action Layer

The Action Layer is where the agent transitions from “question-answering assistant” to “business operator.”

The Action Tool lets Chatbots execute ontology edits — but not by writing freely to databases. The LLM fills parameters within authorized Action Type boundaries and submits. Palantir’s action/permission/audit machinery handles execution.

Example 1: High-Risk — User Confirmation Required

User: "Reassign SH-0042's carrier from A to B"

Agent recognizes this as an assignCarrier action:
  Action Type: assignCarrier
  Parameters:
    shipment = SH-20260621-0042
    newCarrier = CarrierB
    reason = "Original carrier vehicle breakdown, user-specified reassignment"

  Configured as "execute after user confirmation" (carrier changes involve contract and cost implications)

  Agent responds:
    "Confirm reassigning SH-0042's carrier from A to B?
     Note: Carrier B's unit price is 12% higher than Carrier A.
     Estimated additional cost: ¥340."

  User confirms → executes → Ontology: SH-0042.carrier updated to CarrierB

A critical design detail: when presenting the confirmation prompt, the agent doesn’t just say “should I do this?” — it proactively calculates and displays the impact (12% price differential, ¥340 cost increase). This isn’t the LLM guessing; it’s extracted from the Function’s structured output.

Like an intern drafting an email but requiring supervisor review before sending. The supervisor sees “Carrier B costs 12% more” and decides whether to proceed.

Example 2: Low-Risk — Auto-Execute

Scenario: Agent combined with Automate for routine exception handling

  Configuration: When delay reason = "weather" AND delay < 4 hours, action auto-executes

  Trigger:
    AIP Logic function detects SH-0301 delayed 2 hours due to heavy rain
    → Auto-calls updateETA(shipment = SH-0301, newETA = originalETA + 2h)
    → Auto-calls notifyCustomer(orderId = ORD-7710,
        message = "Due to weather conditions, your order is expected to be delayed by 2 hours")

  No human confirmation needed — completes directly

This is “low-risk auto, high-risk confirm” as a design principle in practice. Like autonomous driving adjusting speed on a straight highway without requiring the driver to confirm every adjustment — but at a complex intersection (high-risk operation), control returns to the human.

Supporting Tools

Beyond the core Action Tool, three supporting tools extend the agent’s interaction capabilities:

Update Application Variable: Updates values in Application State, allowing the agent to change Workshop and other application states.

Example: User says in agent conversation "Only show East China region data"
  → Agent calls Update Application Variable
  → Changes Workshop page's region filter variable from "All" to "East China"
  → All charts, maps, and lists on the page refresh in concert

Like telling the receptionist “only transfer East China region calls to me” — the receptionist changes the phone system filter, and all subsequent calls follow the new rule.

Command Tool: Triggers operations in other Palantir applications.

Example: User says "Mark these 5 delayed orders on the map"
  → Agent calls Command Tool
  → Triggers Workshop map component to highlight the 5 Shipments' current positions

Like asking an assistant “mark these clients in red on the map” — the assistant walks to the whiteboard and annotates. You didn’t do it yourself, but the whiteboard changes as you specified.

Request Clarification: Allows the agent to pause execution and ask the user for clarification, preventing misoperation.

Example: User says "Cancel that order"
  → Agent finds 3 possible orders matching "that," and cancellation is irreversible
  → Agent calls Request Clarification:
    "Which order are you referring to?
     1. ORD-8832 (XX Electronics, ¥12,000)
     2. ORD-9201 (YY Tech, ¥8,500)
     3. ORD-9305 (ZZ Manufacturing, ¥21,000)"

Like a nurse double-checking before administering medication: “Did you say the left hand or the right hand?” Better to ask one more question than to make an irreversible mistake.

Layer 5: Governance Layer

This is the core differentiator between Palantir AIP and generic agent frameworks. Governance is not a separate step — it’s an end-to-end constraint mechanism spanning all layers:

Retrieval Context config   → Controls what the agent sees by default each turn
Object Query config        → Controls which object types and properties the agent can actively query
Function config            → Controls which business logic and models the agent can invoke
Action config              → Controls which business mutations the agent can perform, and whether confirmation is required
Application State          → Controls current page, object, and user operation context
Security / Permission      → Controls user and agent access and execution privileges
Dynamic Security           → Dynamically constrains access and execution at object, field, and action levels
Audit / Lineage            → Records who, when, through which agent, triggered what action

Notice each row maps to a specific layer. From Layer 1 through Layer 4, every layer has a corresponding Governance configuration. This isn’t post-hoc auditing — it’s pre-emptive and in-flight constraint.

Same Agent, Different Roles

A concrete example shows how Governance operates at every layer:

Scenario: Frontline dispatcher vs. Regional manager, using the same "Operations Assistant" agent

Frontline dispatcher after login:
  Layer 1: Retrieval Context only injects Shipments from 3 assigned routes
  Layer 2: Object Query can only query objects on those 3 routes; can't see other regions
  Layer 3: Can call calculateDelayRisk, but cannot call viewProfitMargin (cost-sensitive)
  Layer 4: Can execute updateETA (low-risk), but assignCarrier requires supervisor confirmation
  Layer 5: All operations logged to audit trail, tagged "Dispatcher XXX via Operations Assistant"

Regional manager after login:
  Layer 1: Retrieval Context injects entire South China region shipment overview
  Layer 2: Object Query can query all routes in South China + carrier performance data
  Layer 3: Can call calculateDelayRisk + viewProfitMargin + optimizeCarrierAllocation
  Layer 4: Can directly execute assignCarrier (no confirmation needed), but cancelOrder still requires confirmation
  Layer 5: Same audit logging, but lineage traces to a broader permission scope

This is permission layering: same building, different keycards. The intern has access to their floor. The GM has building-wide access. But the underground vault requires dual authentication for everyone — permissions are layered, with extra protection for high-risk areas regardless of role.

Architecture-Level Hallucination Prevention

Hallucination prevention isn’t about writing “don’t make things up” in a prompt. It’s about architecturally ensuring the agent simply cannot access information it shouldn’t.

Another example: Governance constraints preventing hallucination

User asks: "What was the company's total revenue this year?"

Without Governance:
  The LLM might "hallucinate" a number based on training data → hallucination

With Governance:
  Layer 1: No financial data configured in Retrieval Context → Agent can't see it
  Layer 2: No FinancialReport Object Type configured in Object Query → Agent can't query it
  Layer 3: No financial Functions authorized → Agent can't compute it
  → Agent can only honestly respond: "I don't have access to company financial data.
     Please contact the finance department."

  This isn't accomplished by writing "don't fabricate data" in the prompt.
  It's architecturally ensuring the agent cannot see, query, or compute the information.

It’s not posting a “No Cheating” sign in the exam hall. It’s not bringing the answer key into the exam hall in the first place. The agent isn’t “told not to say” — it genuinely cannot know.

This is why Palantir’s anti-hallucination design isn’t a single-point solution: every layer imposes constraints, and four layers of compounding restrictions (what you see + what you can query + what you can do + who has permission) form the complete anti-hallucination architecture. No single mechanism — whether RAG, prompt engineering, or output validation — is sufficient for enterprise-grade reliability.

Tool Calling Mode: How Tools Are Exposed to the LLM

After understanding what tools the agent has, you need to understand how these tools are provided to the LLM. Chatbot Studio supports two modes:

Prompted Tool Calling

Tool descriptions are inserted into the prompt. The LLM understands available tools through prompt text.

Scope: All models + all tool types
Invocation: One tool at a time
Best for: Model compatibility as priority, diverse tool types

Native Tool Calling

Uses the model’s native tool calling capability.

Scope: Only certain Palantir-provided models + only Action / Object Query / Function / Update App Variable
Invocation: Can call multiple tools in parallel
Best for: Performance as priority, token efficiency as priority

This clarifies a common misconception: Palantir does not dump the entire Ontology schema into the system prompt. A more accurate understanding: within a specific Chatbot/Application/Workflow, the Ontology capabilities that are configured, authorized, and relevant to the current task are exposed to the LLM as tool descriptions, context fragments, function signatures, and action definitions.

The LLM doesn’t see “all 500 Object Types and 2000 Properties in the enterprise.” It sees “the query tool for the 5 Object Types and 30 Properties this Chatbot is authorized to access.” Scope is determined by configuration, not by schema.

End-to-End Interaction Walkthrough

Let’s trace a complete interaction through all five layers using a logistics scenario.

Scenario

User: "Check which orders in the South China warehouse today have delay risks, and give me handling recommendations."

Interaction Flow Summary

User Message
    │
    ▼
┌───────────────────────────────────────────────────┐
│  Layer 1: Context Layer [System-level · Deterministic] │
│  Ontology Context / Document Context / Func Context │
│  → Inject into LLM context                          │
└───────────────────────┬───────────────────────────┘
                        ▼
┌───────────────────────────────────────────────────┐
│  Layer 2: Query Layer [LLM Decision]                │
│  Object Query → Filter / Aggregate / Traverse links │
└───────────────────────┬───────────────────────────┘
                        ▼
┌───────────────────────────────────────────────────┐
│  Layer 3: Logic Layer [LLM Decision]                │
│  Function Tool → Business logic / Model / AIP Logic │
└───────────────────────┬───────────────────────────┘
                        ▼
┌───────────────────────────────────────────────────┐
│  Layer 4: Action Layer [LLM Decision + User Confirm] │
│  Action / Command / Update App Variable             │
│  → Ontology edit / External system trigger          │
└───────────────────────┬───────────────────────────┘
                        ▼
┌───────────────────────────────────────────────────┐
│  Layer 5: Governance [End-to-end]                   │
│  Permission check / Audit log / Lineage / App sync   │
└───────────────────────────────────────────────────┘

Internal vs. External Agents

The five-layer architecture primarily describes internal agents (built through Chatbot Studio, Workshop, AIP Logic) interacting with Ontology. When external agents connect, two distinct — and easily confused — mechanisms come into play: Palantir MCP and Ontology MCP (OMCP).

Palantir MCP: For Ontology Builders / Developers

Palantir MCP targets developers and ontology builders. It enables external dev tools or AI coding assistants to understand Foundry projects, codebases, applications, and Ontology architecture context.

Typical capabilities:

- Explore Foundry project structure, navigate codebases and internal Palantir libraries
- Provide context for React OSDK, Python transform, TypeScript function repositories
- View and query datasets (can run SQL queries, create notional datasets, cannot overwrite existing)
- Search Ontology for object types, action types, functions
- Create proposals to modify Ontology type definitions (object types, link types, action types)
- Regenerate OSDK for Developer Console applications

Hard constraint: Palantir MCP cannot write ontology data. It can only modify ontology structure (type definitions). All Ontology type modifications must go through proposal review and require human approval before taking effect.

Ontology MCP (OMCP): For External Business Agents / Ontology Consumers

Ontology MCP (OMCP) is a Developer Console feature that exposes an application’s ontology resources as standard MCP tools, allowing external AI agents to interact with Ontology as MCP clients.

External agents via OMCP can, within controlled boundaries:

- Read Ontology objects and properties
- Invoke exposed query functions
- Trigger authorized action types
- Perform Ontology data edits where permitted

Official documentation lists three typical usage patterns:

Pattern 1: Desktop Agent Integration
  Integrate with Claude.ai, Microsoft Copilot Studio, Gemini Enterprise, and similar desktop agents.
  Users and agents share the same data, logic, actions, and governance.
  Example: A user in Teams asks Copilot to create a task in Ontology —
  other users and applications see it immediately.

Pattern 2: External Agentic Framework Integration
  Integrate with Google Agent Development Kit (ADK), Microsoft Agent Framework,
  OpenAI SDK, Anthropic SDK, and similar frameworks.
  Developers can combine multiple MCP servers (e.g., Perplexity MCP + Ontology MCP)
  to build composite agent workflows spanning external knowledge and enterprise Ontology.

Pattern 3: Headless Agent Workflows
  Unsupervised autonomous agents reading and writing Ontology via OMCP.
  OMCP serves as both the agent's memory and tracking layer —
  all operations recorded in Ontology, providing audit capability for unattended operations.

Key constraint: application restrictions. Administrators must explicitly configure which object types, action types, and functions are accessible to external agents. Authentication is OAuth 2.0-based, supporting both authorization code grant (user-level permissions) and client credentials grant (service-level permissions).

Note: OMCP is currently in beta. Functionality may evolve. Enabling OMCP means exposing data from your Palantir environment to external MCP clients — ensure alignment with your organization’s data governance and compliance policies.

Palantir MCP vs. OMCP: Positioning Differences

Official one-line distinction: Ontology MCP enables controlled writes to ontology data, while Palantir MCP enables modifications to ontology structure.

Palantir MCP:
  - Targets: Developers / Ontology Builders / Platform Engineers
  - Can do: Modify Ontology type definitions (object types, link types, action types)
  - Cannot do: Write ontology data (hard constraint)
  - Additional constraint: All type modifications require proposal review, human approval
  - Typical use: AI coding assistant aiding development within Foundry ecosystem

Ontology MCP / OMCP:
  - Targets: External Business Agents / Ontology Consumers / Headless Agents
  - Can do: Read objects, execute actions, query data, write ontology data
  - Cannot do: Modify Ontology type definitions
  - Additional constraint: Governed by application restrictions, OAuth 2.0 permissions
  - Typical use: Copilot Studio / Gemini / Claude securely reading and writing business data
  - Status: Beta

Regardless of path, the core principle remains invariant: agents access objects, relationships, functions, and actions through governed Ontology interfaces — never bypassing the Ontology to touch underlying data directly.

Minimum Viable Architecture Design

To design an Ontology-aware agent system inspired by Palantir’s approach, nine core components are needed:

1. Ontology Schema Registry
   Manages definitions and versions of object types, properties, links, actions, and functions.

2. Tool Registry
   Manages which tools, functions, and actions are exposed to the agent,
   along with corresponding parameter schemas and invocation methods.

3. Context Builder (corresponds to Layer 1)
   Deterministically injects context based on current user, page, object, and query.

4. Object Query Tool (corresponds to Layer 2)
   Allows the LLM to query objects, filter, aggregate, and traverse links within authorized boundaries.

5. Function Tool (corresponds to Layer 3)
   Encapsulates business logic, models, rules, and optimization algorithms as callable functions.

6. Action Tool (corresponds to Layer 4)
   Encapsulates write operations as governed actions with parameter validation,
   permission checks, and user confirmation.

7. Permission Evaluator (corresponds to Layer 5)
   Verifies user, agent, object, field, and action permissions before every query,
   function call, and action execution.

8. Audit Logger / Lineage Tracker (corresponds to Layer 5)
   Records who, when, through which agent, invoked what tool, and modified which objects.

9. Application State
   Connects the agent to specific business application pages and contexts.

Minimum flow:

User Message → Context Builder → LLM → Object Query → Function → LLM Response
→ User Confirmation → Action → Ontology Edit → Audit Log

To Sum Up

Palantir AIP Agent-to-Ontology interaction is not simple RAG, and not an LLM querying databases directly. It is a five-layer, coordinated enterprise agent runtime:

Context Layer     System-level deterministic context provision — the agent never starts "blind"
Query Layer       Agent-initiated object and relationship exploration
Logic Layer       Encapsulated business logic invocation — no LLM rule-fabrication
Action Layer      Governed business mutations — from Q&A assistant to business operator
Governance Layer  End-to-end permission, audit, and security constraints

What’s truly worth learning isn’t any single tool or prompt pattern. It’s the design philosophy:

First model the enterprise business world into a queryable, computable, executable, governable Ontology — then let agents operate on that governed semantic layer.

Don’t hand the LLM a database connection string and say “figure it out.” Define what exists in the enterprise, who can see what, and who can do what — then let the agent exercise intelligence within those defined boundaries.

References

• AIP Overview: https://www.palantir.com/docs/foundry/aip/overview/

• AIP Architecture: https://www.palantir.com/docs/foundry/architecture-center/aip-architecture/

• AIP Features: https://www.palantir.com/docs/foundry/aip/aip-features/

• Chatbot Studio Tools: https://www.palantir.com/docs/foundry/chatbot-studio/tools/

• Chatbot Studio Core Concepts: https://www.palantir.com/docs/foundry/chatbot-studio/core-concepts/

• Retrieval Context: https://www.palantir.com/docs/foundry/chatbot-studio/retrieval-context/

• AIP Logic Overview: https://www.palantir.com/docs/foundry/logic/overview/

• Ontology System: https://www.palantir.com/docs/foundry/architecture-center/ontology-system/

• Ontology Overview: https://www.palantir.com/docs/foundry/ontology/overview/

• Palantir MCP: https://www.palantir.com/docs/foundry/palantir-mcp/overview/

• Ontology MCP (OMCP): https://www.palantir.com/docs/foundry/ontology-mcp/overview/

• Model Integrationhttps://www.palantir.com/docs/foundry/model-integration/overview/

Welcome to subscribe my newsletter and share the posts to your friends.

Welcome to read my new newsletter about LOOP ENGINEERING, which is a new concept. From prompt engineering, context engineering and harness engineering to now loop engineering, AI Vibe Coding Paradigam keeps evolving. This article could help you to learn about loop engineering.

If you are a visual learner, you are welcomed to watch my youtube video:

Also, welcome to subscribe my X account.

On June 7, 2026, Peter Steinberger (OpenClaw Creator) posted this:

“You shouldn’t be prompting coding agents anymore. You should be designing loops that prompt your agents.”

Two days later, Boris Cherny — the creator of Anthropic’s Claude Code — said the same thing on stage:

“I don’t prompt Claude anymore. I have loops running. They’re the ones prompting Claude and figuring out what to do. My job is to write loops.”

That’s the idea behind Loop Engineering — and it marks a concrete shift in how serious AI builders work.

It’s not a new tool. It’s a new mental model: your job isn’t to prompt AI anymore. It’s to design systems that prompt AI on your behalf.

What Actually Changed

Here’s how you’ve been using AI to write code for the past two years:

You write prompt → Agent executes → You read output → You decide → You write another prompt → ...

You’re the one driving. Every decision flows through you. The entire chain waits on you.

Loop Engineering inverts this:

Before: you drive the agent

You → Agent → You → Agent → You → Agent → ... (manual the whole way)

Now: a system drives the agents, you design the system

Schedule → Triage → Execute → Verify → Deliver → Schedule → ... (you only step in for exceptions)

The difference is who drives whom. You stop being the operator and become the system designer.

How We Got Here

Loop Engineering was named in June 2026, but the roots go back further.

Summer 2025: Engineer Geoffrey Huntley stumbled onto a pattern he called the Ralph Loop — throwing a coding agent into an infinite shell loop:

while true; do
  cat prompt.md | claude
done

Each iteration reads the same prompt file, modifies code, and starts fresh. The next loop gets a clean context window — no conversation history, just the filesystem state left behind by the previous run. Not smart, but effective.¹

April–May 2026: The tooling layer caught up. Codex CLI v0.128.0 shipped /goal, supporting “run until condition met.” Claude Code v2.1.139 followed, with /goal using a separate small model to check completion status after each turn — not the executing model.²

June 9, 2026: Google Chrome engineer Addy Osmani published the definitive framing piece, systematizing Loop Engineering into five core components.³

Each layer doesn’t replace the previous one — it compounds on top. Bad prompting gives you a bad loop that fails faster.

The Five Components + Memory

Claude Code and Codex natively support all five. You don’t need to wire this from scratch.

Component 01: Automations

The heartbeat of any loop. Without it, you have a one-shot run, not a cycle.

Component 02: Worktrees

The moment you run multiple agents in parallel, file conflicts become the bottleneck. Git worktrees give each agent its own isolated working directory — shared repo history, zero cross-contamination.

Component 03: Skills

Without Skills, every agent startup is a cold start. It knows nothing about your project conventions, why you can’t use a particular library, or where credentials live.

A Skill captures this once:

your-project/.claude/skills/ci-fix/
├── SKILL.md    # core instructions
├── scripts/    # executable helpers
└── references/ # supporting docs

Write it once, and every loop run picks it up automatically. This is how you pay down Intent Debt — the gap between what you meant and what the agent guesses.

Component 04: Plugins & Connectors

Built on MCP (Model Context Protocol), connectors let agents read your issue tracker, query databases, call staging APIs, and post to Slack. This is what separates an agent that talks about solutions from a loop that ships them — opening PRs, updating tickets, and notifying the team without your involvement.

Component 05: Sub-agents

This is the most important structural decision in any loop.³

“The model that writes the code is the worst judge of whether the code is correct.”

The fix is Maker-Checker separation: the agent that writes code and the agent that reviews code are not the same instance.

Implementer Sub-agent (writes code)
        ↓
Verifier Sub-agent (independent: runs tests + lint + checks against Skills)
        ↓
Fails → back to Implementer
Passes → proceed

The data: LanceMartin’s Fable 5 experiments showed Verifier sub-agent coverage at 73%, versus 7–33% for self-critique. Separating the writer from the reviewer matters enormously.⁴

Component 06: Memory

Models forget. Repos don’t. Memory is anything that persists across sessions — a markdown file, a Linear board, a GitHub Project. It records what was tried, what passed, and where to resume next time. Without memory, every loop restarts from zero.

Three Examples — From One Command to Enterprise Scale

Loop Engineering isn’t just for complex systems. The same mental model looks completely different at different scales.

Example 01: The Minimal Loop — one command, runs until done

Scenario: you have a Python project and want all lint errors gone, but don’t want to fix them one by one.

/goal "Fix all Python lint errors in src/ until flake8 output is empty"

The loop handles itself: run flake8 → find errors → Agent fixes → run again → 0 errors → stop.

No Worktrees, no Connectors needed. One objective stop condition (flake8 output empty) is a complete loop. This is the minimum viable form of Loop Engineering. You can use it today.

Example 02: Karpathy’s AutoResearch — let AI run your ML experiments

Andrej Karpathy’s autoresearch project⁸ — looking back now — is a form of Loop Engineering. The design is called “One GPU, one file, one metric”:

AutoResearch componentLoop Engineering equivalentprogram.md: you write the research directionSkills — you don’t touch Python files, only intentInfinite loop, every ~5 minutesAutomations — fixed-cadence schedulingAgent modifies train.pyImplementer Sub-agentTraining run + val_bpb evaluationVerifier (not an LLM — an actual training result)Retain/rollback code, log experimentsMemory — maintains optimal state

“The core idea is that you’re not touching any of the Python files like you normally would as a researcher.” — Karpathy, AutoResearch README ⁸

One detail worth highlighting: the Verifier here isn’t another LLM — it’s the real training run itself. Whether val_bpb goes down or up is determined by math, not model judgment. This is the most rigorous form of objective verification. ~12 experiments per hour. Sleep through the night, wake up to ~100 completed runs.

Example 03: Addy Osmani’s Enterprise Bug Fix Loop

Here’s Addy Osmani’s reference implementation³ — a daily automated bug-fix loop with all five components in play:

Sense (triggers at 08:00 daily)

• Triage Skill reads yesterday’s CI failures, open issues, recent commits

• Writes findings to a status file or Linear board

Execute (per finding)

• Opens an isolated Worktree for each bug so parallel work doesn’t interfere

• Implementer Agent writes the fix, guided by project Skills

• Verifier Agent runs tests and validates — failures automatically return to Implementer

Deliver

• Connector opens a PR and links it to the relevant ticket

• Team gets notified when CI passes

Escalate

• Anything the loop can’t resolve goes to your inbox with full context attached

You design this once. You don’t manually prompt any step. You wake up to open PRs.

The same idea, three different scales. Where you start depends on what problem you need to solve today. The minimum starting point: write down one stop condition a machine can verify.

What This Means for You

If you’re a software engineer: your role is shifting from “writing code” to “designing systems,” and then to “designing systems that have AI write the code.”

If you’re watching how AI lands in enterprise teams: Loop Engineering is the engineering methodology behind FDE (Forward Deployed Engineer) work. Bob McGrew’s FDE three-step — Audit → Evals → Deployment — maps cleanly onto the three Loop pillars

Why Code, Math, and Physics Are the Natural Home for Loops

Here’s the part most people miss: the quality of a loop is determined entirely by the quality of its stop condition.

/goal needs a judge that can decide “done or not done” without human input. If that judgment requires a person, the loop can’t run autonomously. If that judgment can be made by a machine — a test suite, a proof checker, a simulation comparison — the loop runs while you sleep.

The verifiability spectrum:

← Machine-verifiable                              Requires human judgment →

Code tests   Math proofs   Physics sims   Data pipelines   Writing   Strategy
  Very high    Very high       High           Medium          Low      Very low

Three reasons verifiable domains are natural fits for Loop Engineering:

1. The stop condition lives outside the model Whether tests pass is determined by the compiler, not by how confident the agent feels about its work. Whether a mathematical proof is valid is determined by the axiom system, not the model’s self-assessment. This is the foundational requirement for autonomous loop operation.

2. The Verifier can actually verify The 73% coverage figure only holds when the Verifier sub-agent is running deterministic checks — executing code, comparing values, validating proof steps. In subjective domains, the Verifier is essentially running a more elaborate version of self-critique. The advantage over the 7–33% baseline shrinks dramatically.

3. Failure signals are visible The Maker-Checker cycle requires a feedback signal. In code, that’s a test failure and error output. In physics simulations, it’s deviation from known constants. Without a visible failure signal, the loop repeats rather than corrects — a critical distinction.

Before designing any loop, answer this one question:

Can a machine independently judge whether this output is correct?

• Yes → build a high-autonomy loop, trust the Verifier sub-agent

• No → loops still add value, but you need to be part of defining the stop condition — you are the Verifier

Three Things to Think Through Carefully

Loop Engineering solves some problems and sharpens others.

1. Verification is still your responsibility An unattended loop is an unattended source of errors. The loop doesn’t know right from wrong — it only knows whether your conditions were satisfied.

2. Comprehension Debt The faster the loop ships code you didn’t write, the wider the gap between you and your codebase. Speed is not the goal.

3. Cognitive Surrender The most dangerous posture: the loop runs, you stop thinking, you accept the output. This is what makes the autonomy feel comfortable and makes the errors feel surprising.

Two engineers running the same loop can produce completely opposite outcomes. One uses it to move faster on work they deeply understand. The other uses it to avoid understanding the work. The loop can’t tell the difference. You can.

What to Do Next

Loop Engineering is still early. The concepts are named but the tooling is still being built out. Here’s where to start:

1. Pick one verifiable task — a test suite you run manually, a lint check you forget, a CI triage you do every Monday morning

2. Write the stop condition first — what does “done” mean, in terms a machine can check?

3. Build a Triage Skill — capture the context that any agent would need to understand the task cold

4. Add a Verifier sub-agent — even a simple one that just runs existing tests improves coverage dramatically

5. Wire a Connector — close the loop by having results delivered somewhere useful (PR, Slack, Linear)

Start small. A loop that runs five tests on every commit is more valuable than an elaborate system that never ships.

“The leverage point moved. The work didn’t get easier.”

— Boris Cherny, June 2026

The three principles that won’t go out of date:

• The stop condition is the product — your loop is only as good as how you define “done”

• Maker-Checker separation is real quality assurance — don’t let the writer grade their own work

• Memory compounds — every loop run should leave a trace the next one can build from

zerofuturetech · June 2026，Welcome to subsribe my newsletter.

References

1. Geoffrey Huntley, Ralph Loop pattern, July 2025

2. Claude Code v2.1.139 release notes (/goal primitive); Codex CLI v0.128.0, April–May 2026

3. Addy Osmani, “Loop Engineering,” June 9, 2026

4. RLanceMartin, “Designing Loops with Fable 5,” Verifier sub-agent experimental data

5. Boris Cherny, Acquired Unplugged talk, June 8, 2026

6. Peter Steinberger (@steipete), Twitter/X, June 7, 2026

7. Andrej Karpathy, AutoResearch, github.com/karpathy/autoresearch

If you are a visual learner, you are able to watch my youtube video:

You also could follow my X account

Most Palantir breakdowns start with the Ontology — “the digital twin of your organization” — then jump straight to AIP agents.

The part that gets skipped is the part that costs the most.

How does a messy ERP export, a pile of JSON booking records, and a folder of scanned PDF maintenance reports actually become queryable, AI-operable objects in Palantir’s Ontology? If you can’t answer that, you can’t evaluate Palantir’s real deployment cost or understand why every enterprise rollout requires resident Forward Deployed Engineers.

Here’s the full pipeline, mapped against a concrete airline operations scenario.

The Five-Stage Chain

Our scenario: an airline with three data types — structured flight operations CSV from Oracle DB, nested booking JSONs from OTA systems, and scanned PDF maintenance reports written by engineers.

Stage 1: Data Connection — Pull It In As-Is

Foundry’s first rule: ingest without transformation.

“Foundry is also distinguished by a philosophy that data should be ingested ‘as-is’ from its most raw source, with no external preprocessing.” — Data Connection Overview, Palantir Docs

The reasoning: if all transformations happen inside Foundry, every Ontology property value traces back to a specific row in a specific raw file. End-to-end data lineage — non-negotiable in regulated industries.

Foundry connects through 200+ connectors. Two sync modes: SNAPSHOT (replace the current view entirely) and APPEND (add new rows only).

Stage 2: Storage Layer — Dataset and Media Set

Dataset is Foundry’s general-purpose storage unit — not a database table, but a versioned file container. It can hold structured (Parquet + schema metadata), semi-structured (JSON/XML files), and unstructured (raw binary) data. Every change is an atomic Transaction — Palantir calls this “Git for Data.”

Four transaction types: SNAPSHOT, APPEND, UPDATE, DELETE.

“Dataset transactions are the basis of Foundry’s support for data versioning, sometimes referred to as ‘Git for Data.’ A transaction is analogous to a commit in Git: an atomic change to the contents of a dataset.” — Core Concepts: Datasets, Palantir Docs

Media Set is not a replacement for Dataset. It’s a specialized abstraction for media processing workflows. When you need to process large-scale images, video, audio, or PDFs, Media Set provides media item management, format conversion, and direct integration with Document Intelligence AI extraction pipelines — capabilities Dataset alone doesn’t have.

The two connect through Media References: a column type in a Dataset pointing to a file in a Media Set. Your maintenance records table can have structured fields (report ID, date, parts) plus a media_reference column pointing to the original scanned PDF.

Stage 3: Pipeline Transforms — Three Paths

Two tools: Pipeline Builder (drag-and-drop, no code) and Code Repositories (Python/PySpark).

Path A: Structured Data → Schema Inference + Cleaning

Foundry’s “Apply a schema” button scans a data sample and auto-infers column names and types. Then use Pipeline Builder or @transform_df:

@incremental(semantic_version=1, v2_semantics=True)
@transform_df(
    Output('/airlines/clean/flights'),
    raw=Input('/airlines/raw/flight_operations')
)
def clean_flights(ctx, raw):
    return (
        raw
        .filter(F.col('status').isNotNull())
        .withColumn('departure_time',
            F.to_timestamp('departure_time', 'yyyy-MM-dd HH:mm:ss'))
        .dropDuplicates(['flight_id'])
    )

Path B: Semi-Structured Data → JSON/XML Parsing

Nested JSON needs flattening. A booking JSON’s flights array is exploded with F.explode() — each leg becomes its own row, one JSON becomes two linked tables: clean_passengers + clean_bookings:

@transform_df(Output('/airlines/clean/bookings'),
              raw=Input('/airlines/raw/passenger_bookings'))
def flatten_bookings(raw):
    return (
        raw
        .select('booking_id',
                F.col('passenger.name').alias('passenger_name'),
                F.col('passenger.passport').alias('passport'),
                F.explode('flights').alias('flight'))
        .select('booking_id', 'passenger_name', 'passport',
                F.col('flight.flight_id').alias('flight_id'),
                F.col('flight.seat').alias('seat'))
    )

Path C: Unstructured Data → AIP Document Intelligence

Scanned PDFs can’t become rows directly. The path is five steps, not two hops:

StepOperationDetail①Load into Media Setschema_type: document, primary_format: PDF②OCR + VLM extractionTraditional / Generative AI strategies③Field extractionParse tail_number, date, parts from Markdown④Validation & reviewSchema checks + confidence filter + human sampling⑤Output DatasetIncludes media_reference pointing to original PDF

“Document preprocessing essentially runs traditional OCR on the document, then passes that output in addition to the document page itself to a VLM, giving the model more context to successfully analyze the document.” — AIP Document Intelligence: Core Concepts, Palantir Docs

Document Intelligence generates the deployment code with one click:

@transform.using(
    output=Output("ri.foundry.main.dataset.abc"),
    media_input=MediaSetInput("ri.mio.main.media-set.abc"),
    extractor=VisionLLMDocumentsExtractorInput(
        "ri.language-model-service..language-model.anthropic-claude-sonnet"
    ),
)
def extract(media_input, output, extractor):
    extracted_data = extractor.create_extraction(
        media_input, with_ocr=True, prompt=USER_PROMPT, thread_number=20
    )
    output.write_dataframe(extracted_data)

Key limitation: Document Intelligence outputs one Markdown row per page — not structured fields. The full path is PDF → Markdown → structured fields → Ontology. Two hops, not one.

Stage 4 + 5: Curated Dataset → Ontology

The mental model: dataset ≈ object type, row ≈ object, column ≈ property. In practice, you explicitly configure backing datasource, property mapping, primary key, title key, link cardinality, property visibility, and potentially Multi-datasource Object Types (MDO).

“The Foundry Ontology creates a complete picture of an organization’s world by mapping datasets and models to object types, properties, link types, and action types.” — Ontology Core Concepts, Palantir Docs

1. Object Types

Backing DatasetObject TypePrimary KeyTitle Exampleclean_flightsFlightflight_id”CA1234”clean_passengersPassengerpassenger_id”Zhang Wei”clean_bookingsBookingbooking_idbooking_idclean_aircraftAircrafttail_number”B-6075”clean_maintenanceMaintenance Recordreport_idreport_id

2. Properties

Beyond basic types (String, Integer, Timestamp), Foundry supports advanced property types: Vector (semantic embeddings for similarity search), Geopoint/Geoshape (coordinates), Media Reference (link to original PDF or image), Time Series (sensor telemetry data).

3. Link Types

Links are built from foreign key matches. Two cardinality types with different update mechanisms:

One-to-many (FK-based): Flight → Aircraft — links resolve dynamically at query time. Funnel doesn’t need a separate changelog.

Many-to-many (join table): Requires a dedicated join table dataset. Funnel maintains an independent changelog, same batch pipeline timing as Object Types.

4. Action Types

Action Types make the Ontology writable. Modifications write to a separate Writeback Dataset — never overwriting the backing dataset.

Logic and Actions: The Ontology’s Brain and Hands

After mapping is complete, two more layers are needed: Functions (logic/computation) and the full Action Types framework.

Functions (Logic Layer) enable four capabilities: derived properties (delay_minutes = actual - scheduled), cross-object aggregations (average delay per airline), batch modification logic (cancelling a flight auto-releases seats and notifies all passengers), external system queries (enriching flights with live weather data).

Action Types (Write Operations Layer) are defined by four components:

ComponentPurpose”Rebook Passenger” exampleParametersTyped input fieldspassenger, new_flight, reasonRulesOntology edits to executeModify Booking.flight_id; Delete old Link; Create new LinkSubmission CriteriaPre-conditions that must passnew_flight not cancelled; seats available; user has agent roleSide EffectsPost-execution triggersEmail passenger; Webhook to crew management API

Two implementation paths: simple rule-based configuration (UI only) and function-backed actions (TypeScript/Python for complex logic).

“Rules define the logic of the action type that transform the parameters into Ontology edits or other effects.” — Action Types: Rules, Palantir Docs

Keeping the Ontology Fresh: Incremental Updates

APPEND vs SNAPSHOT

Three-Layer Incremental Chain

Incremental is a Contract, Not a Switch

Two common failure modes:

Mutable source data + APPEND = duplicates. If source records can be modified (flight status changes from Scheduled to Delayed), APPEND sync creates both the old and new version in the dataset. Downstream transforms must explicitly deduplicate or take the latest version — otherwise stale data enters the Ontology.

File-based UPDATE transactions = broken incremental chain. UPDATE transactions modify existing files, forcing all downstream pipelines to fall back to full SNAPSHOT recomputation.

Update Timing by Ontology Element

Ontology elementWhat triggers updateWhen it’s visibleObject Type propertiesbacking dataset new transactionAfter next Funnel batch pipeline runOne-to-many Link (FK-based)FK property value changeAuto-resolves with Object update, no separate syncMany-to-many Link (join table)join table dataset new transactionAfter next Funnel batch pipeline runAction execution resultUser triggers ActionImmediately in OSv2 live index; async persistence

The Wide Table Problem

Hard constraint: one Object Type can only be backed by one dataset. A 200-column ERP export can’t map directly. Palantir calls this the “Kitchen Sink anti-pattern.”

The only path: split the wide table in the pipeline layer first, then map each focused dataset to its own Object Type, connected via Link Types. Pipeline Builder’s AI feature can write the splitting code from a prompt. But deciding how to split — which columns belong to which entity — is still entirely human judgment. That’s where Palantir’s Forward Deployed Engineers spend the most valuable time.

Why This Creates a Product Opportunity

Palantir’s Marketplace supports packaging Object Types into installable products. Technically you could ship industry Ontology templates. Palantir won’t — FDE consulting is part of the model.

The moat isn’t in defining what entities exist (Shipment, SKU, Warehouse for supply chain; Patient, Diagnosis for healthcare — industry consensus is high on these). The moat is knowing which columns in a SAP MM export map to which entity. That source-system-aware mapping knowledge is what FDEs reconstruct from scratch at every client.

LayerContentReuse rateLayer 1Standard entity definitions90%+Layer 2Source system mapping rules (SAP / Oracle / ERP)70%+Layer 3Customer-specific extensionsCustom

Vertical players have more incentive to productize this than Palantir does.

What Actually Makes Ontology Different

From a data modeling perspective, the Ontology looks like a relational schema with business labels. The three genuine differences:

Full lineage. Trace any property value back to a raw file transaction. Built-in from day one. Non-negotiable in regulated industries.

Writable without polluting source data. Most data catalogs are read-only. Ontology with Action Types and Writeback Dataset lets business users make decisions directly in the system — source data stays clean.

Application-native. Workshop, Slate, OSDK, and AIP consume Ontology objects directly. No extra API layer. When a user clicks “Resolve Alert,” they’re executing an Action Type — property update, link rebuild, writeback write — all in one operation.

Together, these make it an operational layer, not just a documentation layer.

All technical details sourced from Palantir’s official documentation. Some of them are reasoning because of no more details shown in Palantir documents.

If this was useful, share it with someone evaluating enterprise AI platforms.

Follow my newsletter!

That’s a 729% increase. In one year.

OpenAI created an entirely separate “Deployment Company” to scale this function. Anthropic is paying $200K–$300K to start and hiring across 9 cities globally. Google Cloud’s CEO publicly announced a mass FDE hiring push — and compressed their interview process to just 2 days to move faster.

This isn’t a typical job market spike. It’s a signal: the AI industry has collectively realized that the bottleneck isn’t model capability. It’s deployment. And FDEs are the answer.

What Is a Forward Deployed Engineer?

Bob McGrew — Palantir’s early Head of Product Engineering, later OpenAI’s CRO, and one of the architects of the FDE model — defines it cleanly:

“A forward deployed engineer is someone typically technical who sits at the customer site and fills the gap between what the product does and what the customer needs.”

Three non-negotiable elements:

On-site, not remote. Not a quarterly visit. Not async communication. Actually embedded in the customer’s daily operations.

Filling the gap between product and reality. Not selling a standard product. Solving everything that falls between “what our AI can do” and “what this specific customer actually needs.”

Delivering working software. Not slide decks. Not recommendations reports. Code that runs in production.

It’s not a traditional software engineer, not a pre-sales engineer, not a consultant. It’s all three — a new archetype that the AI era is now demanding at scale.

This Role Isn’t New. So Why Is It Exploding Now?

The FDE model has existed at Palantir since 2009.

Palantir was building intelligence analysis software for the CIA, NSA, and related agencies. The fatal problem: every intelligence client’s workflows were slightly different. Counter-terrorism analysts and counter-proliferation analysts did entirely different things. A standardized product couldn’t work.

CTO Shyam Shankar’s solution: don’t build a universal product. Build a deeply customizable platform, and deploy your best engineers directly to customer sites — to understand requirements on-site, write code on-site, and deliver on-site.

Bob McGrew captured this strategy with a memorable metaphor: “gravel roads vs. paved highways.”

FDEs build gravel roads at customer sites — rough but functional, solving immediate problems. Product teams back at HQ build paved highways — they observe patterns across multiple gravel roads and abstract them into capabilities that work for many customers.

The key insight: gravel roads aren’t throwaway work. They’re product discovery in action. Palantir’s most valuable asset — the Ontology (”objects + properties + links” universal data model) — was discovered by FDEs noticing a problem at a customer site, bringing it back to HQ, and prompting the product team to build something more general.

So why is it everywhere now? Four forces are converging simultaneously:

1. Intelligence is becoming a commodity. The capability gap between GPT-4, Claude, and Gemini is narrowing. API prices keep falling. “Who has the better model” is becoming less of a moat every quarter. What’s actually scarce is knowing where and how to deploy that intelligence. FDEs earn 30–50% more than comparable engineers at the same company — the market is saying deployment expertise is more valuable than model expertise.

2. AI agents are an undefined category. Traditional SaaS has a known product template: identify that Siebel CRM is bad, build Salesforce. The product shape is understood; execution is the variable. AI agents have no template. Is an agent a chatbot? An autonomous process? A plugin embedded in existing systems? A new kind of OS? Nobody knows yet — and the only way to find out is to sit inside actual customer workflows and discover it there.

3. Capabilities are racing ahead of adoption. McGrew frames it this way: you’re riding in a Waymo and you’re not thinking “oh my god, there’s no driver.” You’re thinking “ugh, traffic.” The more powerful AI gets, the more mundane it feels. The question was never “is the model capable enough?” It’s always been “can it actually integrate into how we work?” That last mile of deployment is where FDEs operate.

4. Token economics changed the cost structure. Traditional SaaS has near-zero marginal cost: serving your 10,000th customer costs essentially the same as serving your first. AI agents don’t work that way — every run burns tokens, every LLM call has inference cost. FDEs have to make real-time optimization decisions: use Opus or Haiku for this task? Can 5 lines of Python replace this LLM call entirely? Can we cache this prompt? These decisions require being inside the customer’s actual environment to make correctly.

What FDEs Actually Do All Week

From Perspective AI’s survey of 1,500 active FDEs:

• 47% at customer sites (interviews, deployments, design reviews)

• 31% writing or reviewing code

• 22% internal coordination and research synthesis

The work follows three sequential phases:

Audit. Spend two weeks embedded with a new client — from revenue operations to procurement to finance. The core output is a set of judgment calls: what goes to agents, what gets hardcoded, what stays human? The framework: variable inputs with rule-based workflows (email, PDFs, images) → use an agent. Predictable rules and inputs → write code, it’s faster and cheaper. Pattern recognition and domain expertise required → keep human. End with a working demo.

Evals. Build an evaluation framework that proves the AI is actually working. Not just “is the final answer correct?” — decompose the human problem-solving process into steps and score each one. Build a golden dataset of 20+ real queries with hand-labeled perfect answers. This is how you get executive buy-in. Every company says they want AI. Most executives are quietly skeptical it will actually work for their specific situation. A rigorous eval framework is what earns their trust.

Deployment. Don’t attempt a big migration. Build an API layer on top of existing systems rather than replacing databases. Build a secure sandbox directly inside the customer’s infrastructure. Start with the smallest autonomous unit possible — read-only data access first, then ticket creation, then code commits. You earn more autonomy by demonstrating correctness at each level.

Who This Is For (and Who It Isn’t)

In Palantir’s practice, FDE work is split between two profiles:

Delta (deployment engineers): Ships prototypes fast. Doesn’t care if the first version gets thrown away entirely. Closer to a founding engineer than a staff engineer. McGrew is direct about who doesn’t fit: “The archetypal person who’s not suited for this is someone who loves ensuring their abstractions are perfect and their code is maintainable for 12 years — because that’s not the job.” If the phrase “we’ll probably rewrite this from scratch” causes you anxiety, Delta probably isn’t for you.

Echo (embedded analysts): Deep domain background — ex-military, healthcare, logistics, finance. Must be a “rebel or heretic” in their field: someone who sees the severe inadequacy of current practices and genuinely believes AI can deliver 3x–10x improvement. If you walk into a client site and think “these workflows seem pretty reasonable,” Echo isn’t for you.

Three backgrounds that transition naturally:

1. Consultants / PMs — already know how to translate data into ROI. The gap is engineering depth. Fill it by building agents, RAG pipelines, eval frameworks, and MCP integrations. Let the portfolio speak.

2. Software engineers — strong technical foundation. The gap is business communication. You need to translate “here’s what the AI can and can’t do” into terms that land with a non-technical VP making a $2M decision.

3. Founders / early-stage startup veterans — strongest natural fit. FDE requires everything a founder does: finding direction in ambiguity, rapid prototyping, commercial judgment. This background transfers directly.

30-Day Ramp Plan

Here’s a concrete training plan you might start immediately:

Week 1: The Agent Loop

Read Anthropic’s Building Effective Agents. Build your first prompt → model → response → next step loop. Add tool calls (API integrations + web search). Build guardrails: input validation, maximum step limits, output filtering. Build an audit log that captures every prompt, tool call, and response.

Week 2: Structured Output + Production Gaps

Master structured outputs (JSON mode — this is foundational for real deployments). Study the demo-to-production failure modes. Most agent projects die here. Build checkpointing so your agent can recover state after a crash.

Week 3: Reliability + Cost Optimization

Add retry logic and exponential backoff. Optimize costs: route simple tasks to cheaper models, cache frequent prompts, track cost-per-call. Build your golden dataset — 20 real-world queries with hand-labeled perfect answers. Understand multi-agent orchestration patterns.

Week 4: Business Communication + Portfolio

Review everything you built. Practice explaining it out loud. Every technical decision needs a business rationale — before you’re done, you should be able to articulate to a non-technical executive exactly what each architectural choice costs and what business problem it solves.

“Your portfolio and your ability to articulate it are the deciding factors. Build agents, RAG pipelines, eval frameworks, MCP integrations — but most importantly, be able to confidently explain the business value behind each decision to a non-technical decision-maker.”

The Counterintuitive Warning You Need to Hear

The most memorable part of Bob McGrew’s FDE Playbook interview wasn’t about why FDE is great. It was this:

“If you’re in a business where this is working for you, that’s great. Don’t do the FDE strategy. You have been given an amazing gift. If you have the opportunity to just scale, treat all the customers the same, go ahead and do that.”

“My first, second, and third pieces of advice to people who are thinking about trying an FDE strategy is: don’t do this at home. If you can avoid it, it’s probably bad for you. Probably you’re going to end up doing services.”

If you can build a standardized product that works for all customers — build SaaS. Scale it. That’s the better business model. Don’t break what works.

FDE mode is only the right answer when three conditions are simultaneously true: no incumbent product to replace + highly heterogeneous customer needs + contract values large enough to support on-site teams. Palantir hit all three in 2009. The current AI agent market hits all three today. That’s not coincidence — it’s market structure forcing the strategy.

The biggest trap: FDE drifts into consulting. Every new client starts from scratch, no experience gets abstracted back into the product. At that point you’re doing consulting with a better job title. The real FDE model builds a new stretch of highway with every client deployment.

The Takeaway

Demand for FDEs is growing at 800% per year. Supply is growing at 50%. That gap is real and widening.

The role exists because someone has to do the last mile of AI deployment. It won’t be automated away — it is the automation work. The people who can bridge AI capability and actual business utility are genuinely scarce, and the market is pricing that scarcity correctly.

“There’s just a huge amount of opportunity out there — looking at what the capabilities are, but asking what does it take to make them really genuinely useful to people.” — Bob McGrew

If you want to go deeper: start with Anthropic’s Building Effective Agents and Bob McGrew’s FDE Playbook podcast interview. Both are foundational for understanding what this work actually involves.

Data sources: Indeed · LinkedIn · Adecco Staffing · Perspective AI State of Forward Deployed Engineering 2026 (1,500 FDEs surveyed across 154 companies)

Here’s a problem most AI engineers hit eventually.

You build a RAG system, stuff your documents in, and start answering questions. Works fine until someone asks the same thing three times and gets three different answers. Today the LLM says Supplier A’s rating is 85. Tomorrow it says 92. The day after, 78. The LLM isn’t lying — it just doesn’t know the relationship between “supplier,” “rating,” and “date.” It guesses every time.

The fix is an ontology — a structured knowledge map that doesn’t just store data, but defines relationships and rules between entities. With one, your AI goes from guessing to querying. And it can chain-reason: material shortage → affected production lines → affected customer orders.

Four Contradictions in Traditional Ontology Building

Before LLMs, building an ontology meant: knowledge engineers and domain experts sitting together for months, sketching on whiteboards, hand-writing definitions in OWL (Web Ontology Language, the W3C standard) and SHACL (Shapes Constraint Language, for data validation). This approach has four fundamental contradictions:

The speed contradiction: Manual modeling is too slow for business change. By the time the ontology is finished, the business has already shifted.

The talent contradiction: People who understand both OWL/SHACL and the specific business domain are extremely rare. The two skill sets almost never overlap in one person.

The maintenance contradiction: Once built, nobody maintains it. The ontology gradually becomes a “dead model,” unable to evolve with the business.

The scale contradiction: Manual approaches can’t handle large document volumes. The method collapses as document scale increases.

LLMs happen to have three capabilities that address exactly these needs: extracting structured information from unstructured text, understanding domain semantics, and generating standard formats like JSON and OWL. But using LLMs naively for ontology construction runs into four hard problems:

• Problem 1: You don’t know how many concept types exist — 10 or 1,000?

• Problem 2: LLMs hallucinate, inventing concepts that aren’t in the source text

• Problem 3: Granularity is hard to calibrate — “has a relationship” is too vague, “is to the left of” is too specific

• Problem 4: No ground truth to validate against, so you don’t know if the result is good

Between 2025 and 2026, five distinct methods emerged from the research literature to address these four problems.

Method A: Pipeline Decomposition

• Hallucination risk: Low

• Engineering complexity: High

• Key papers: Salovsky 2026 · IRT SystemX 2025 · Wikontic 2025

Break the big task into controlled steps, using the best tool for each. Standard pipeline:

Document input → ① Entity extraction (NER) → ② Relation extraction → ③ Deduplication
               → ④ Dual validation → ⑤ Write to graph

Each step spelled out:

• identify named entities from documents;

• determine relationships between entities;

• merge “Supplier A” and “SUPPLIER-A” into the same entity;

• structural compliance + logical consistency dual check;

• write to graph database.

The Validation Loop Is the Core Innovation

The critical feature of this school is dual validation. Every new fact must pass two layers of checks:

• SHACL (Shapes Constraint Language, W3C standard validation language) structural validation: Is the data schema-compliant? For example, “an order must have a customer ID that is an integer”

• OWL Reasoner logical validation: Are there contradictions? For example, a concept cannot simultaneously be a parent class and a child class of another concept

Pass validation → write to the trusted graph. Fail → log it, don’t contaminate the knowledge base.

Salovsky (2026) adds MCP protocol orchestration to coordinate LLM, RDF/OWL graph, vector RAG, and external APIs. The LLM serves only as an “interpretation, generation, and orchestration layer” — the actual world model lives in the RDF/OWL graph. Validated with Tower of Hanoi tests: 5-disc success rate improved from 33.3% to 45.5%.

Industrial Case: IRT SystemX × EDF (2025)

Partners included IRT SystemX, EDF (France’s national electricity utility), Airbus, and RTE (France’s electricity grid operator). Semi-automated ontology construction from French technical documentation. Their 7-layer pipeline:

1. Text preprocessing (French POS tagging, domain-specific tokenization)

2. NER (RTE uses regex; EDF uses fine-tuned CamemBERT)

3. Relation extraction (RTE uses RelationPrompt; EDF uses fine-tuned LUKE)

4. Metadata enrichment (DBpedia/Wikidata SPARQL extension)

5. Version lifecycle management (Populated → Enriched → Evaluated → In-use → Archived)

6. Evaluation (NER-evaluate + OOPS! + FAIR Explore)

7. KG generation and visualization (Flask + VueJS + SPARQL)

FAIR data quality score improved from 22.5% (V1.1) to 36.11% (V1.3). Note: this project actually uses traditional deep learning models — CamemBERT, LUKE — with plans to integrate Mistral/Gemma in future iterations.

Lightweight Case: Wikontic (2025)

Uses Wikidata as a skeleton constraint for LLM output. Key numbers:

• Only <1,000 output tokens — one-twentieth the cost of GraphRAG

• MuSiQue benchmark: 96% of triples contain correct answer entities

• HotpotQA: 76.0 F1 (using triples only, no text context needed)

• MINE-1 benchmark: 86% SOTA information retention rate

Strengths: Every step is controllable and swappable; lowest hallucination risk; supports versioning and continuous updates.

Weaknesses: Errors in earlier steps propagate forward; high engineering complexity and setup cost; multi-step LLM calls consume more tokens overall.

Best for: Production systems where quality is non-negotiable.

Method B: Clustering-Driven

• Hallucination risk: Low

• Engineering complexity: Medium

• Key paper: LLM4Onto 2025

Don’t define categories upfront — let the data decide. Core pipeline:

Documents → Extract nouns (SpaCy) → Vectorize (BERT) → AP clustering (auto-group)
          → LLM names each cluster → Multi-round merging → Ontology

Why AP Clustering Instead of K-means

K-means requires you to specify the number of clusters in advance. When you’re building an ontology for an unfamiliar domain, you don’t know that number. Affinity Propagation (AP) determines its own cluster count based on data similarity. If AP over-segments (splitting “apple,” “banana,” and “pear” into three separate clusters), the LLM merges similar clusters back through multi-round dialogue into “fruit.”

The HT-R-O Three-Stage Relation Extraction Pipeline

Relation extraction is the hardest part of ontology construction. LLM4Onto compared three paradigms:

The three HT-R-O steps with concrete examples:

Step 1 (Instance-level extraction)

Input: "HoneyMyte group uses ToneShell backdoor"
Output: (HoneyMyte, use, ToneShell backdoor)

Step 2 (Ontology-level abstraction)

Output: (APT group, use, malware)

Step 3 (Relation standardization)

"use" / "employ" / "leverage" → unified as "use"

Benchmark Results

• MultiWOZ: Literal F1 of 49.5% (baseline TeQoDO scores only 2.8%)

• Cybersecurity domain: Discovered 483 entity types + 72 relation types (traditional NER models identify only 13–40)

• ArXiv: Graph F1 of 68.68%

Strengths: Truly data-driven with no manual annotation needed; no prior knowledge of category types required; small-task decomposition keeps hallucination risk low.

Weaknesses: Clustering quality directly determines final output quality; no built-in online validation; open-domain scenarios may over-fragment into too-fine granularity.

Best for: Exploring new domains where you don’t know what concept types exist.

Method C: Two-Phase Generation

• Hallucination risk: Medium

• Engineering complexity: Medium

• Key papers: OntoEKG 2026 · NeOn-GPT 2024

Phase 1 (Extraction Module): have the LLM identify candidate classes and attributes from unstructured documents. Phase 2 (Entailment Module): organize that flat list into a logical hierarchy and serialize to RDF. The two phases are separate — the intermediate output is human-reviewable and adjustable.

Unstructured docs → Phase 1 (Extraction Module) → Phase 2 (Entailment Module) → RDF serialization Identify core classes + attrs  Organize into logical hierarchy

OntoEKG (ICSC 2026): Achieved fuzzy-match F1 of 0.724 in the Data domain. Limitation: when concept hierarchies get complex, the model tends to flatten rather than properly nest. Hierarchy reasoning is still the weak point.

NeOn-GPT (Fathallah et al., 2024): Follows the NeOn ontology engineering methodology, using LLMs to replace most of the manual work. Workflow: requirements writing → OWL encoding → evaluation → documentation generation. Integrates GPT-3.5, LLaMA, and PaLM as interchangeable model backends.

Strengths: Clear intermediate artifacts that can be reviewed; fast construction of an initial ontology draft; medium engineering complexity, easy to get started.

Weaknesses: Two-stage errors compound; hierarchy reasoning quality depends on LLM capability; no built-in validation mechanism.

Best for: Fast first drafts for direction validation; cases where you need visibility into intermediate state.

Method D: Schema-Guided Extraction

Hallucination risk: Low | Engineering complexity: Medium | Key papers: OntoKG 2026 · Wikontic 2025

Start with a framework, extract within it. Rather than letting the LLM freely invent “what relationship is this?”, constrain it to choose from a defined list: “Is this relationship ‘belongs to,’ ‘located in,’ or ‘causes’?” The option space is bounded, so the hallucination space shrinks accordingly.

Existing schema (Wikidata etc.) → Define type and relation lists → LLM extracts within schema → Schema validation → Compliant output

Schema-Free vs. Schema-Based

The 2025 Bian survey concludes: these two are complementary, not competing. The ideal system = Schema-Free discovers new concepts first → Schema-Based formalizes them into standard structure.

Large-Scale Case: OntoKG (2026)

Organized Wikidata’s 346 million entities into 94 modules across 8 categories. The core mechanism — intrinsic-relational routing:

• Each property → Intrinsic attribute (color, size) → node attribute module

• Each property → Relational attribute (located in, part of) → edge module

LLM + human review results: 93.3% category coverage, 98.0% module assignment accuracy, encompassing 34 million nodes, 61.2 million edges, and 38 relation types.

Strengths: Output is naturally schema-compliant with small hallucination space; low engineering effort when a good schema exists; fits regulated industry standards.

Weaknesses: Constrained by the coverage of the existing schema; weak at discovering genuinely new concepts; requires a suitable schema to exist upfront.

Best for: Domains with existing industry standards or data specifications (healthcare, legal, energy, etc.).

Method E: End-to-End Prompting

Hallucination risk: High | Engineering complexity: Low | Key benchmark: LLMs4OL Challenge 2025

One prompt, full output. Fast to implement:

You are a {domain} ontology expert.
Extract ontology classes and their relationships from the following text.
Only extract what is explicitly stated. Do not add inferred content.
Output format: JSON.
{"classes": [{"name": "...", "definition": "..."}],
 "relationships": [{"source": "...", "target": "...", "type": "..."}]}
Rules: 1. Only extract explicit content  2. Use standardized names  3. Output "None" if ambiguous
Text: {input_text}

Four key findings from the LLMs4OL 2025 challenge:

1. No universal winner: No single model performs best across all subtasks

2. Small models + good prompts ≈ large models: Well-crafted prompts sometimes let small models match large model performance

3. Extreme prompt sensitivity: Minor wording changes produce significantly different ontology structures from the same model

4. Fastest results: A first draft in 30 minutes, suitable for quick direction validation

Strengths: Simplest to implement, zero engineering setup; results in 30 minutes.

Weaknesses: Highest hallucination risk; unstable output quality; prompt sensitivity seriously affects output.

Best for: Quick proof-of-concept validation. Not suitable for production.

Five Methods Side-by-Side

*Wikontic uses <1,000 tokens, one-twentieth the cost of GraphRAG

The Decision Framework

Five Engineering Lessons From the Literature

Lesson 01: NER is where most errors originate

Salovsky (2026) explicitly flags this: “most future errors originate in the NER phase.” Wrong entity types, alias confusion, and incomplete type coverage in the extraction phase propagate through every downstream step. IRT SystemX’s experience confirms this: RTE used regex while EDF fine-tuned CamemBERT — the right choice depends on how much labeled data you have. Invest here before optimizing anything else.

Lesson 02: Relation extraction is much harder than entity discovery

Finding “what things exist” is relatively straightforward. Understanding “what relationships exist between those things” is the real challenge. LLM4Onto specifically designed the 3-stage HT-R-O pipeline to handle this. IRT SystemX found certain relation types (like “ending time”) were far more ambiguous than others. OntoEKG showed clear limitations in hierarchical reasoning. Recommendation: extract coarse-grained relations first, then use LLM to refine and standardize.

Lesson 03: Validation is the line between toy and production

Without validation, you’re letting the LLM write whatever it wants into your knowledge base. LLM4Onto has no online validation mechanism — that’s one reason it remains a research prototype. Minimum viable: SHACL structural validation. If you can, add OWL Reasoner for logical consistency checks.

Lesson 04: Prompt wording affects output structure

García-Fernández et al. (2025) found that minor prompt wording changes from the same LLM produced significantly different ontology structures. Use structured templates with explicit rules and examples, not free-form natural language descriptions. Validate output format programmatically — don’t trust that the model produces consistent JSON structure.

Lesson 05: More tokens doesn’t mean better quality

Wikontic built an effective knowledge graph with under 1,000 output tokens — one-twentieth of GraphRAG’s cost. High token consumption doesn’t necessarily produce higher accuracy. The key is not data volume, but data quality and the reasonableness of constraints.

Where This Field Is Heading

Trend 1: Schema-Free and Schema-Based are converging

Let LLMs freely explore and discover new concepts first, then automatically integrate new concepts into standard frameworks — the “from disorder to order” loop is becoming the dominant design pattern. The Bian survey (2025) calls this the ideal architecture.

Trend 2: Ontologies are becoming the memory layer for AI agents

Ontologies aren’t just static knowledge bases for querying. They’re becoming the “world model” for AI agents — the agent uses it to understand current state, plan next actions, and record execution results. This direction becomes increasingly important as agent frameworks mature rapidly.

Trend 3: Extension from text to multimodal sources

Extracting ontologies not just from text, but from charts, images, and sensor data. This direction is just starting, and represents the next significant growth area in the field.

Trend 4: Standardized evaluation is being established

The LLMs4OL challenge, the UPM survey, and similar efforts are driving unified evaluation standards — allowing different methods to be compared on the same scale. Without standardized evaluation, there’s no real progress comparison between approaches.

Tool Building

In order to understand ontology and how LLM drives ontology construction, I have built a tool called OntoPrompt in my github:

https://github.com/jingw2/nano-ontoprompt

This OntoPrompt is a nano platform using Prompt + Rule based + LLM methods to construct ontologies in different domains.

Welcome to try it.

Terminology Reference

The question has shifted from “can LLMs build ontologies?” to “which approach fits your constraints?” Your task scope, engineering resources, and quality requirements determine the answer — not which paper was published most recently.

Five schools, five tradeoffs. None of them is universally superior. The right one depends on where you are: a 30-minute prototype or a production system that can’t be wrong.

Zero Future covers applied AI engineering weekly. Subscribe to stay current.

If you’ve been tracking enterprise AI, you’ve noticed a pattern: the models are ready, but the deployments are stuck. Everyone has GPT. Everyone has Claude. Yet somehow, AI inside large organizations still feels like an intern who showed up on day one with no onboarding.

Palantir has an answer. It’s called Ontology — and it might be the most important piece of enterprise infrastructure you haven’t heard of.

Here’s what makes this interesting: Ontology isn’t a database, a dashboard, or a semantic layer with a new name. It’s a digital twin of your entire organization — a living map that both humans and AI can read. And it answers the one question that matters most: how does AI actually understand your business?

Let’s get straight to it.

What Palantir Is (and Isn’t)

A quick primer.

Founded in 2003 by Peter Thiel and Alex Karp (a philosophy PhD), Palantir took its name from the palantíri in Lord of the Rings — seeing stones that pierce space and time.

Their first check came from In-Q-Tel, the CIA’s venture arm. Most Silicon Valley VCs passed.

Today they run three product lines:

• Gotham (2008): Intelligence and defense

• Foundry (2016): Commercial — Airbus, BP, NHS

• AIP (2023): AI Platform, bringing LLMs into the enterprise

NYSE direct listing in 2020. S&P 500 in 2024. $4.48B revenue in 2025.

But the commercial story isn’t what matters here. What matters is the underlying architecture — the thing that makes all three product lines work.

What Ontology Actually Is

Let me start with what it’s not.

It’s not a data warehouse. Not a BI dashboard. Not a data catalog. And it’s not “semantic layer” rebranded.

Palantir’s official definition: “A digital twin of the organization — a semantic layer that sits on top of datasets and models.”

Translation: it weaves your scattered data, business logic, and operational capabilities into a living map that both humans and AI can navigate.

A concrete example

A traditional data warehouse tells you:

“The t_orders table has 5 million rows.”

Ontology tells you:

“Customer Acme Corp’s third order — containing 5 turbine blades, currently in quality inspection at the Hamburg warehouse, linked to DHL tracking #DHL-88492, estimated arrival in Shanghai next Tuesday.”

The difference? Ontology doesn’t just store data. It stitches together fragments from your ERP, CRM, SCM, and IoT systems into a world your business operators can actually understand.

The mapping is straightforward

Every technical concept gets “translated” into a business concept.

Three-layer architecture

Data Layer (all sources)
      ↓
Logic Layer (Ontology semantic model)  ← the core
      ↓
Action Layer (Workshop, AI Agents, automation)

Ontology is the middle layer. Downward: it connects to all your data. Upward: it powers every application and agent.

A worked example

Say you run an aircraft engine manufacturer. Your Ontology has five object types:

• Customer → places → Order → contains → Engine → assembled from → Part → sourced from → Supplier

They’re connected by Links. Then you define Actions:

• approveOrder

• rerouteShipment

• freezeSupplier

These aren’t scripts. They’re declaratively defined within the Ontology framework — permissions, validation, and audit trails included.

Build the Ontology once. Every application and AI Agent above it shares the same semantic foundation.

How to build one

Using a supply chain scenario, here are the steps:

1. Define the nouns — Translate database tables into business objects. A Supplier object has an ID, name, region, lead time, risk score, and so on.

2. Wire up the brain — Teach the system how to reason. How many days of inventory remain? Which alternative supplier should we engage?

3. Configure who can do what — Each Action is like a contract: what it does, who can trigger it, under what conditions, and what gets logged.

Think of it as adding buttons to a remote control — each with its own lock, not everyone can press every button.

Once built, humans and AI Agents see the same world. The AI doesn’t just read your data — it understands your company.

Ontology + AIP = AI That Actually Works

This is where Palantir’s 2023 launch of AIP (AI Platform) comes in.

The core insight is simple: an LLM can chat and write code, but if it doesn’t understand your business, it’s forever an intern. Ontology gives AI a complete “employee onboarding handbook” for your enterprise.

OAG (Ontology Augmented Generation)

You’ve probably heard of RAG — Retrieval Augmented Generation.

RAGOAGWhat it retrievesText passagesStructured business objects + real-time relationshipsResult qualityLLM guesses and stitchesDeterministic matchReliabilityProne to hallucinationsGrounded in live dataWhat happens nextNothing actionableTriggers approval workflows

Same question — “Which supplier can fill the gap?”:

• RAG: Searches docs → LLM pieces together an answer → potential hallucination, no follow-through

• OAG: Queries Supplier objects → exact match → auto-creates a purchase order through the approval pipeline

The AIP workflow

Connect to Ontology → Security guardrails (what humans can't see, AI can't see) → Agent executes

One critical design choice: Human-in-the-Loop.

Palantir’s own words: “We don’t sell autonomous driving. We sell a copilot.”

Suggest → Human approves → Execute. That’s the rule.

15 Minutes vs. 3-5 Days

Here’s a real scenario: Supply chain disruption.

Morning alert — a critical component supplier in Southeast Asia is offline due to flooding.

The old way

3 to 5 days. Manually checking 20 different systems, stitching data together, writing reports.

The Ontology + AIP way

15 minutes. Five steps, fully closed-loop:

1. IoT sensors auto-trigger the anomaly alert

2. AI traverses linked orders and calculates potential losses

3. Optimization engine generates 3 alternative sourcing plans with cost and timeline simulations

4. Supply chain manager reviews and selects the best option

5. Purchase orders are auto-created, ERP is updated, customers are notified — loop closed

This isn’t “AI is smarter than humans.” It’s AI eliminating the pain of manually hunting through 20 systems for data.

Ontology isn’t a “data middle platform.” It’s the operating system for your business in the AI era.

Without it, AI Agents fumble in the dark. With it, they become the most knowledgeable person in your company.

What It Actually Takes

If you’re sold, here’s the reality check.

Implementation path

Discovery → Bootcamp (5 days to first working use case) → Pilot → Go-live → Scale

Benchmark: Nebraska Medicine started in January 2024, deployed their first workflow in under 6 weeks. Subsequent use cases went live in as little as 90 minutes.

The trade-offs

What you get:

• Speed: Results in 5 days

• Power: Model once, reuse everywhere

• Security: Permissions span from data to AI layer, fully auditable

What you risk:

• Cost: Realistically, governments and very large enterprises only

• Vendor lock-in: Ontology is proprietary — migration costs are extreme

• Ethics and privacy: ICE contracts, IDF collaboration, NHS contract 70%+ redacted

Three Things to Remember

If you take away nothing else, take these three:

1. Palantir’s moat isn’t AI models. It’s Ontology.

Anyone can access GPT or Claude. But modeling every entity, relationship, rule, and action in your business into an AI-comprehensible semantic system — that requires deep domain understanding and sustained engineering investment. Models are generic. Ontology is proprietary.

2. Ontology is your enterprise operating system.

Just as iOS lets apps operate your phone’s hardware, Ontology lets AI Agents and business applications operate your enterprise’s data and processes — with a single set of security, permission, and governance rules. Build once, reuse everywhere.

3. Map your business entities and decision flows before you bring in AI.

Too many organizations rush to “implement AI” while skipping the foundational step: what are your business objects, how do they relate, and who has permission to do what with them? Without this semantic infrastructure, AI Agents are just groping in the dark — they can talk, but they can’t see or touch your actual business.

Semantics first. Then AI can actually run.

The Bottom Line

Palantir’s methodology isn’t about which database you use. It’s about a transformation: data → business objects → AI-comprehensible knowledge → executable actions.

This path isn’t cheap. It isn’t fast. And it isn’t easy.

But for organizations that genuinely need enterprise-grade AI — it might be the only approach that solves the triple challenge of security, governance, and semantic understanding at the architectural level.

Even if you never touch Palantir, the methodology is worth understanding.

If this piece gave you a useful mental model, share it with someone wrestling with enterprise AI deployment.

Reference：

https://www.palantir.com/platforms/ontology/#ontology-toolchain

https://www.palantir.com/docs/foundry/ontology/overview/

You can follow me on X: @jingwangtalk

I came back from a Southeast Asia business trip facing a classic problem: one China Merchants Bank credit card screenshot, four Grab ride e-receipts, all of it needing to become a clean expense report.

You know the feeling. Staring at a handful of PDFs and a screenshot, manually reconciling transactions, calculating exchange rates, filling in Excel. I sat there for 30 seconds, then asked myself one question:

What if I handed this to an AI?

Not one AI. Six of them — plus a wildcard.

That’s what this piece is about: a multi-step LLM stress test with a real task and a verifiable answer. Every model’s output can be compared line by line against the ground truth. No room for subjective judgment.

The Task: Five Steps, One Complete Pipeline

Five files as input: one credit card screenshot (Chinese UI), four English Grab e-receipt PDFs, each covering a ride in Malaysia.

Five steps to complete:

Step 1. OCR the credit card screenshot and extract all Grab-related RMB transactions.

Step 2. Parse the four PDFs and extract order IDs, pickup points, drop-off points, and local currency amounts (Malaysian Ringgit, RM).

Step 3. Cross-source matching — map the credit card records to the four Grab orders. Hidden catch: there are 8 credit card entries, not 4.

Step 4. Exchange rate inference and merge logic — figure out why 4 rides produced 8 charges, and combine them correctly.

Step 5. Call the xlsx skill and output a standardized Excel summary table.

All models ran inside the OpenCode framework, each in its own folder, given the identical prompt.

In this directory, there is 1 credit card screenshot and 4 Grab PDF files. Complete the following tasks and output an Excel file:
1. Identify and extract Grab RMB transaction records from the screenshot
2. Use pdf skill to parse the 4 PDF receipts — extract Booking ID, pickup point, drop-off point, and local currency amount
3. Match credit card transactions with the 4 Grab bookings. Adjacent credit card entries may correspond to the same Grab booking — evaluate local currency vs. RMB exchange rates and determine which rows should be merged
4. Use xlsx skill to output an Excel table with fields: Booking ID, Pickup, Drop-off, Local Currency Amount, Total RMB Amount, Exchange Rate, Corresponding Card Records (show summed amounts for merged entries). Name the output file "grab-summary-[current-model-name].xlsx"

The Real Difficulty: One Ride, Two Charges

Let me lay out the ground truth first, so the analysis that follows has something to anchor against.

The credit card screenshot shows these 8 Grab-related charges:

¥135.89  ¥12.59  ¥32.22  ¥53.67  ¥5.07  ¥7.13  ¥130.61  ¥16.12

Eight entries, four rides. The correct pairing logic:

The pattern: every Grab ride generates two credit card entries — the main fare (Grab charges first according to estimates), plus a smaller service fee/additional fee (ranging from ¥5 to ¥17). Only when you add both together do you get the actual RMB settlement total for that ride.

The correct exchange rate is 1.73–1.74 MYR/CNY, reflecting real MYR/RMB rates on the settlement dates.

Any wrong match, missed entry, or rate miscalculation makes the final expense report wrong.

Seven Models, Seven Ways to Fail

*Note that rank here is the rank to output the result.

Kimi K2.6 — The Only One That Got It Right

All four orders matched correctly. RMB totals and exchange rates accurate across the board.

The output Excel was clean and well-formatted. For Order 1, it recorded the credit card entries as ¥12.59 + ¥135.89 — the order of the two numbers differs slightly from the reference answer, but the amounts are identical. It correctly identified the two-charge-per-ride structure, with rates landing consistently in the 1.73–1.74 range.

The only model that completed all five steps correctly in a single run.

GLM 5.1 — One Step Short

GLM 5.1 was the second-best performer. It nailed the first three orders:

• Order 1 (KLIA T2 → DoubleTree): ¥135.89 + ¥12.59 = ¥148.48 ✓

• Order 2 (DoubleTree → Box-Pak): ¥32.22 + ¥7.13 = ¥39.35 ✓

• Order 3 (Money Corner → KEX): ¥53.67 + ¥5.07 = ¥58.74 ✓

Then Order 4 (Integra Tower → KLIA T1) fell apart. It matched ¥119.84 + ¥16.12 = ¥135.96, when the correct answer is ¥130.61 + ¥16.12 = ¥146.73.

The source of the error: ¥119.84 does not exist in the credit card screenshot. GLM 5.1 hallucinated this number when reading the fourth entry, misreading ¥130.61 as ¥119.84. One wrong digit in the OCR step cascaded into a wrong total and a wrong exchange rate — dropping from the correct 1.74 down to 1.61.

Three correct, one OCR hallucination, one wrong row on the final report.

DeepSeek V4 Flash — Systematic Matching Failure

DeepSeek V4 Flash read all 8 credit card entries correctly. But its matching logic went completely off track. It tried to pair entries by looking for adjacent large amounts rather than understanding the main-fare-plus-fee structure:

• It assigned ¥119.84 + ¥16.12 (= ¥135.96) to Order 1 — both numbers actually belong to Order 4

• Orders 2, 3, and 4 each got only a single entry (¥32.22, ¥53.67, ¥130.61), leaving the ¥7.13, ¥5.07, and ¥16.12 service fees completely unmatched

• Exchange rates skewed accordingly — at one point producing a rate of 1.42 MYR/CNY, which simply doesn’t reflect reality

Four orders, four wrong rows, three credit card entries left floating without a home.

DeepSeek V4 Pro — Perfect OCR, Wrong Everywhere Else

The wildcard entry, DeepSeek V4 Pro, is the most interesting failure. Its OCR was clean — all 8 credit card entries correctly read. It also knew that merging was required. But its merge logic went sideways in a specific way.

Its approach: group entries by similar implied exchange rates (around 1.58) and bundle them together. The result:

• Order 2 (RM 22.66): merged ¥16.12 + ¥7.13 + ¥12.59 into a single record = ¥35.84 (three entries for one order — wrong)

• Order 4 (RM 84.49): merged ¥130.61 + ¥5.07 = ¥135.68 (off by ¥11.05)

• Orders 1 and 3: each matched to a single entry only

It even left a note in the Excel file: “¥135.89 appears with an ‘=’ marker, interpreted as a subtotal row in the banking app UI, not an independent transaction.” That call caused it to discard ¥135.89 — the single largest charge in the dataset — leaving Order 1 without its main fare. The reasoning had internal logic. The premise was wrong, and everything downstream followed.

However, the output format from Deepseek-v4-pro is the best.

Hunyuan 3 — Missing 2 Entries, Rates Skewed Throughout

Hunyuan 3’s core problem: it tried to match one credit card entry per order, with no awareness of the two-charges-per-ride pattern:

• Order 1: matched only ¥130.61 — missing ¥135.89, and borrowing a number that belongs to Order 4

• Order 2: over-merged, combining ¥16.12 + ¥7.13 + ¥12.59 into a single record (three entries for one order)

• Orders 3 and 4: each matched to a single entry only

Two of the 8 credit card entries ended up matched to nothing. Exchange rates fell in the 1.52–1.61 range across the board, because the model calculated rates using only the single main fare — never adding the service fee to the denominator.

Mimo V2.5 Pro — Good OCR, Cross-Matched the Two Big Orders

Mimo V2.5 Pro’s OCR was solid, which put it ahead of some other models at the reading stage. But it made one specific matching error in the hardest part of the task:

• Order 2 (RM 22.66): ¥32.22 + ¥7.13 = ¥39.35 ✓

• Order 3 (RM 33.80): ¥53.67 + ¥5.07 = ¥58.74 ✓

• Order 1 (RM 85.69): ¥130.61 + ¥12.59 = ¥143.20 ✗ (correct: ¥135.89 + ¥12.59, off by ¥5.28)

• Order 4 (RM 84.49): ¥119.84 + ¥16.12 = ¥135.96 ✗ (¥119.84 is a hallucinated number)

Orders 1 and 4 are similar in size — RM 85.69 vs RM 84.49, with main fares of ¥135.89 vs ¥130.61. Mimo cross-matched them, swapping the large charges between the two most expensive rides.

Mimo is the only model to have many middle files and code left in the folder.

MiniMax M2.7 — Four Identical Rows, Complete Collapse

MiniMax M2.7 produced the most extreme failure in the test.

Its output: all four orders filled with the exact same numbers — ¥51.38 for every RMB total, 1.6379 for every exchange rate, and identical fabricated credit card entries (MYR 16.67 / ¥27.32 and MYR 14.7 / ¥24.06) repeated across all four rows.

None of these figures appear anywhere in the source files. The model failed at both OCR and PDF parsing, then filled the output table with four rows of identical invented data.

Error Summary Across All Models

What This Task Actually Measures

Most of the failing models died at the same point: faced with 8 credit card entries and 4 orders, their first instinct was one-to-one matching — not two-charges-per-ride.

One-to-one is the simplest possible assumption. It’s also wrong. Getting it right requires the model to first understand how cross-border credit card billing works — a single transaction can split into a main charge and a service fee across two separate line items — and then use that prior knowledge to validate each pairing against amount ratios and exchange rate consistency.

This isn’t purely a reasoning problem. It isn’t purely a multimodal problem. It’s what happens when both need to work simultaneously, inside a task with a hidden structure, across multiple steps, without any single failure point to bail the model out.

Leaderboards measure capability ceiling — how high a model can score. Real work demands a capability floor — not making basic mistakes on a task that isn’t even that hard.

That’s the gap that matters.

There’s no shortage of LLM leaderboards. But here’s a pattern worth noticing: the model ranked first doesn’t always outperform the model ranked third when it comes to your actual work.

Why? Because leaderboards measure ceiling — how high a model can score. Real work demands a floor — consistently not getting the basics wrong.

OCR, PDF parsing, data matching, exchange rate calculation, Excel output — none of these is hard on its own. String them into a complete pipeline where the model has to orchestrate, decide, and self-correct across every step, and it turns out that’s still genuinely difficult in 2025.

If this test was useful, feel free to share it.

Welcome to subscribe my account：zerofuturetech