Okay, so the VPN says Connected, but the user still cannot open an internal application.

You paste the details into an AI assistant and ask what is wrong. It gives you the familiar list: DNS, routing, firewall policy, certificates, authentication, or perhaps the application itself.

Nothing on that list is unreasonable. It is also not particularly useful because you still have half a dozen possible causes and no idea which one deserves your attention first.

This is where we tend to ask too much of the model. We want it to jump from a loosely described symptom to the root cause, when the more useful question is much smaller:

What single test should I run next?

Troubleshooting needs a sequence, not a longer list

A language model is very good at generating possibilities. Ask it why an internal application is unreachable over a VPN and it can easily give you 12 things to check.

An engineer does not need 12 things to check at once. You need the next test that will eliminate some of them.

This is not about validating a diagnosis after the model has already made one. The shift happens earlier: use the model to choose and sequence the observations that will make a defensible diagnosis possible.

That distinction runs through Robert McMillen’s The Practical Network Engineer’s Guide: map the expected path, choose a test that answers one question, and use the result to shrink the fault domain.

The VPN example makes this particularly clear because Connected confirms only that the tunnel was established. It does not confirm that the client received the correct DNS settings, selected the correct route, has a viable return path, or can reach the required service.

Those are separate questions, so they need separate tests.

Suppose this is what we know:

The user’s internet connection works. The intended VPN profile shows Connected. The internal application at payroll.corp still does not open.

At this point, asking the model for the root cause encourages it to fill in the gaps. Asking for the next test forces it to work with the evidence you actually have.

The first useful boundary may be name resolution:

Resolve-DnsName payroll.corp

If that fails, the investigation moves toward the DNS settings assigned to the VPN interface, resolver reachability, the DNS suffix, or the internal record itself.

If it returns the expected internal address, DNS is no longer the leading question. You can then inspect the route Windows selects for that address:

Find-NetRoute -RemoteIPAddress “10.20.30.40”

Replace 10.20.30.40 with the address returned by the DNS check. The result shows the local address, interface, selected route, and next hop Windows will use.

If the route points through the expected VPN interface, test the service the user is actually trying to reach:

Test-NetConnection payroll.corp -Port 443

A failed port test does not immediately prove that the firewall is responsible. The application may not be listening, the service may be bound to another interface, or the return path may be wrong. But the result has still helped because the investigation is now smaller than it was before.

That is the job we should give the model: not to guess the ending, but to help us choose the next branch.

Give the model this constraint

Instead of asking:

Why can’t this user access the internal application?

Try this:

You are assisting with a network investigation. Do not diagnose the root cause or recommend a configuration change yet.

Reported symptom:

[Describe what the user cannot do.]

Confirmed evidence:

[List each completed check, where it was run from, the source interface or address, the target, the port or parameters used, and the exact result.]

Remaining possibilities:

[List the fault domains still under consideration.]

Choose the single next test that will reduce the fault domain the most. Use read-only commands only.

Return:

1. The test and exact command

2. Why this test should come next

3. Where the test must run from and which parameters must remain fixed

4. What a successful result would rule out or make less likely

5. What a failed result would suggest

6. What this test still cannot prove

Do not recommend a fix. Your job is to choose the next observation required before diagnosis.

The final line matters. Without it, the model may still turn a test-selection task into a remediation plan because producing an answer feels more complete than admitting the investigation is still open.

You can use the same prompt for a BGP session that will not establish, an inter-VLAN path that works in only one direction, a firewall rule that appears correct, or a service that responds locally but not remotely. The commands change, but the discipline does not.

AI does not need to solve the incident in one response to be useful. Sometimes its best contribution is helping you avoid three irrelevant checks and one premature configuration change.

What network problem would you try this prompt on? Reply and tell me. I would be interested to see which fault domains come up most often.

Thanks,

Sayali

Editor-in-Chief

Upcoming tomorrow - Build AI Agents With the Google Engineer Who Co-Chairs KubeCon

Abdel Sghiouar is a Cloud Developer Advocate at Google, co-host of the Kubernetes Podcast from Google, KubeCon Co-Chair and a CNCF Ambassador. On Saturday he walks you through the part most agent tutorials skip: what it takes to run an agent you can actually secure, observe and afford, without being locked to one vendor or one hosted model.

Saturday, October 31st at 9:00 AM EDT. Four hours, hands-on, recording included.
Use code FINAL40 for 40% off

LAST CALL - REGISTER HERE

You start with a working agent on your own machine using an open model like Gemma, no cloud account needed. Then you move the same logic to the cloud, add authenticated MCP tools with proper service accounts and IAM, and build an Orchestrator, Worker and Reviewer network where agents delegate and share state. The last lab puts it on a serverless platform with tracing, cost tracking and guardrails for failed tool calls and runaway loops. You leave with the full codebase and the infrastructure scripts to run it again yourself.

P.S. - There’s also a VIP Pass that includes 2 e-books exclusively for you

Okay, so multi-agent systems are having their moment. Once an AI agent can call network tools, the next step seems obvious: give it a coordinator, a troubleshooting specialist, a configuration specialist, and a validator.

That sounds impressive, but it does not mean every network incident needs an entire team of agents.

This was one of the more useful takeaways from our recent Multi-Agent Network Operations workshop with Eduard Dulharu. The session demonstrated a full agent team, but Eduard also stressed that you should start simply and add architectural complexity only when the problem requires it.

You can also purchase the event recording if you’d like a deeper understanding of the topic.

The lab began with a familiar situation. Users at Branch 7 could not reach services in the data center, while the monitoring dashboard showed no obvious failure. The supplied logs emphasized repeated interface flaps, so a single agent with access to all the available tools followed the loudest symptom and bounced the interface.

The link came back. The service did not.

The actual cause was an OSPF area mismatch between the two ends of the link. The logs never showed it, so the agent would not find it by staring harder at the flap messages.

Eduard put it simply: the agent had all the right tools, but it did not have a method.

More agents do not create a troubleshooting method

The successful diagnosis followed an ordered runbook. It checked reachability, the route, the neighbor and adjacency state, OSPF settings on both ends, interface counters, logs, and a known-good branch for comparison.

That sequence is the troubleshooting knowledge of a senior engineer. The model did not decide which command looked useful next. The workflow collected evidence in a known order, and the model helped rank and explain the findings.

If you use Ansible, you already know why this matters. You would not give a model every available module and ask it to invent a playbook while production is failing. When the steps are known, you encode them in the order they need to run.

The same rule applies here. A multi-agent team is not the starting point. It is the final rung of a complexity ladder.

Where does Branch 7 land?

Branch 7 needed a fixed runbook to find the fault, which places the diagnosis on rung 2. It only needed a team because the fix then had to pass through separate configuration and validation responsibilities.

That distinction matters. Finding the fault and governing the fix are different problems, and they do not automatically require the same architecture.

A simple interface-status question may need only one model call. If the same eight checks must run every time, a fixed workflow is more predictable and easier to test. A single agent becomes useful when tool selection must adapt within one bounded domain.

The multi-agent team earns its place when the incident crosses domains, contains conflicting evidence, or requires one component to challenge another component’s work. In the workshop, the troubleshooter had read-only access, the configuration specialist selected a documented fix, the scripted validator checked the proposal, and a human approved the change before it touched the network.

The validator does not have to be another model. In this case, it was plain code. A scripted post-check can confirm outcomes that are known in advance, such as both ends sharing the expected OSPF area and the adjacency returning to Full. A validator agent earns its place only when the check cannot be written beforehand and requires bounded interpretation.

If several agents receive the same context, use the same tools, and judge the same output, you have not created independent specialists. You have created several chances to repeat the same mistake.

More agents also mean more handoffs, more failure modes, more latency, and more cost. None of that is automatically justified because the architecture diagram looks sophisticated.

Before choosing an orchestration framework, take one real incident and place it on the ladder. Stop at the first level that solves the problem safely. Move upward only when you can name the responsibility that the next component needs to own.

Where would your most common network incident land: model call, workflow, single agent, or multi-agent team? Reply and let me know.

Thanks,

Sayali
Editor-in-Chief

Upcoming this Tuesday - Agentic AI for Platform Engineering: Building Agent-Ready Platforms

Tuesday, October 27th at 9:00 AM EDT. Four hours, hands-on, recording included.
Use code FINAL40 for 40% off

AI agents are starting to use your platform the way engineers do, and most IDPs weren’t built for that. In this 3-hour hands-on workshop from Guillermo Vigil and Ramiro Alvarez of Resizes you give agents their own identity, scoped permissions and controlled tool access, then put them to work on an alert triage workflow with approval gates and observability around every action. You leave with a pattern for turning your existing IDP into one agents can use safely.

REGISTER HERE

Also Upcoming next week -
1. Next Saturday - Build AI Agents With the Google Engineer Who Co-Chairs KubeCon

Abdel Sghiouar is a Cloud Developer Advocate at Google, co-host of the Kubernetes Podcast from Google, KubeCon Co-Chair and a CNCF Ambassador. On Saturday he walks you through the part most agent tutorials skip: what it takes to run an agent you can actually secure, observe and afford, without being locked to one vendor or one hosted model.

Saturday, October 31st at 9:00 AM EDT. Four hours, hands-on, recording included.
Use code FINAL40 for 40% off

LAST CALL - REGISTER HERE

You start with a working agent on your own machine using an open model like Gemma, no cloud account needed. Then you move the same logic to the cloud, add authenticated MCP tools with proper service accounts and IAM, and build an Orchestrator, Worker and Reviewer network where agents delegate and share state. The last lab puts it on a serverless platform with tracing, cost tracking and guardrails for failed tool calls and runaway loops. You leave with the full codebase and the infrastructure scripts to run it again yourself.

P.S. - There’s also a VIP Pass that includes 2 e-books exclusively for you

2. Next Wednesday - Building Endpoint Security with Intune, Security Copilot & AI Agents

Wednesday, October 28 at 9:00 AM EDT. Four hours, hands-on, recording included.
Use code FINAL35 for 35% off

Device issues, policy conflicts and vulnerabilities eat a lot of admin time to track down. This 3-hour workshop run by Sourabh Sarkar shows where Security Copilot and Copilot in Intune cut that down, including a 70-minute hands-on Intune lab. You also build and test a custom Security Copilot agent for a real security scenario.

REGISTER HERE

Agent Vault live demo: Oct. 21, 1 PM ET. Save your spot.

Join this 30-minute live demonstration of Infisical Agent Vault to see how credential brokering allows agents to reach approved services without reading the underlying secrets. The session will also cover expiring access, service-specific permissions, agent identities, and session logs.

Save your spot

Your AI agent should never see the device password

If you use Ansible Vault, you already know not to leave device passwords sitting inside a playbook.

But an AI agent creates a slightly different problem. You can store the credential securely and still expose it when the agent needs to use it.

Imagine a troubleshooting agent responding to a failed BGP session. It queries NetBox for the intended peer, checks live metrics, collects the relevant logs, and then connects to the router to inspect the neighbor state.

The easiest way to make that work is to give the agent credentials for every system it needs. Put the API tokens and SSH password in environment variables, let the tools retrieve them, and move on.

Everything works, but the agent can now potentially read the same credentials it uses.

That matters because anything available to the agent can find its way into a prompt, tool trace, debug log, or model request. A read-only account limits what the credential can do. It does not prevent that credential from being exposed and reused elsewhere.

Ansible Vault solves a different part of the problem

Ansible Vault protects secrets while they are stored and shared. The encrypted value can live alongside the automation without leaving the password in plain text.

At runtime, however, the automation still needs the decrypted credential to authenticate.

With a conventional playbook, that execution path is relatively predictable. An AI agent can choose tools, process external input, preserve context across several steps, and generate extensive logs along the way. Handing the credential directly to that process creates a wider exposure path.

This is where credential brokering changes the model.

Instead of returning a password or API key to the agent, a broker checks the agent’s identity and the action it is requesting. If the request matches policy, the broker uses or injects the credential outside the agent’s readable context. The agent gets the access or result it needs, but not the reusable secret behind it.

In short:

Secrets management: 
Where is the credential stored? 


Credential brokering: 
Can the agent complete the task without seeing the credential? 

You still need secure storage. Brokering adds a boundary at the point of use.

Define access around the task

For the BGP troubleshooting agent, the policy could look like the following. This is an illustrative policy, not product-specific syntax.

Agent identity: 
bgp-triage-agent 
 

Approved targets: 

NetBox 
Network observability platform 
Edge routers in the affected site 


Approved actions: 

Read intended peer state 
Query BGP and interface metrics 
Collect relevant logs 
Run approved show commands 

 
Credential handling: 

Brokered outside the agent context 
Never returned in prompts, tool output, or logs 

Session lifetime: 
15 minutes  

Blocked actions: 

Configuration changes 
Credential retrieval 
Access to unrelated devices 

Escalate: 
Any request that requires write access  

Audit: 
Record the agent identity, target, action, and result 

Notice that the scope is based on the task, not simply the user account. The agent can inspect the failed peer without receiving standing access to every router or permission to change the configuration.

If the investigation concludes that a configuration change is required, that becomes a separate workflow with its own approval and access policy. The troubleshooting credential does not quietly become a deployment credential.

That distinction is easy to miss when an agent works perfectly in a lab. A shared admin account gets the demo running quickly, but it also makes it difficult to answer basic questions later: Which agent accessed the device? What did it request? Could it read the secret? When did that access expire?

Before connecting an agent to another production system, ask one question:

Can the agent complete this task without ever receiving the credential it uses?

If the answer is no, the next step is not a longer system prompt. It is a better access boundary.

Would your current network agent still work if every reusable credential disappeared from its environment variables? Reply and let me know which integration would break first.

Thanks,

Sayali

Four live workshops this month, each paired with a book if you want to read ahead.

1. Tomorrow, 24 hours to go live - October 17th, 9:30 AM EDT |
   Multi-Agent Network Operations

Build a team of agents that work an incident together, run only playbook-approved commands, and wait for your sign-off before touching the network. You’ll work in browser labs against an incident simulator, plus a live demo on Arista cEOS and Containerlab.

Use code FINAL40 for 40% off, exclusively for you

VIEW THE WORKSHOP →

Do you want to build the first agent before you scale to several? Definitely, Building AI Agents for Network Operations walks you through it. A must read.

P.S. Includes a VIP Pass as well, pick 2 e-books of your choice

2. Next Saturday, October 24th, 9 am EDT |
   Build AI Agents: From Local to Production Cloud

Build an agent on your laptop with open models, then take it to Cloud Run with Gemini, MCP and A2A.
Led by Abdel Sghiouar, Cloud Developer Advocate at Google and KubeCon Co-Chair.

Use code FINAL40 for 40% off, exclusively for you

VIEW THE WORKSHOP →

Build the agent in the workshop, then point it at your network. Eric Chou’s AI Networking Cookbook covers local LLMs with Ollama, a network co-pilot, and MCP-based monitoring.

P.S. Includes a VIP Pass as well, pick 2 e-books of your choice

3. Next Week - October 27th, 9 am EDT
   Agentic AI for Platform Engineering: Building Agent-Ready Platforms

AI agents are becoming users of your platform. Learn to give them their own identity and scoped permissions, control their tool access, and observe every action, with approval gates and recovery built in. You’ll finish with an agent triaging an alert inside defined security boundaries.

Use code AINET35 for 35% off, exclusively for you

VIEW THE WORKSHOP →

Decide what agents are allowed to do before they touch your platform. Agentic AI for Platform Engineering shows how to have them open pull requests instead of pushing changes, with audit trails, identity controls and human approval where the risk calls for it.

4. Next week - October 28 | 9:00 AM EDT
Building Endpoint Security with Intune, Security Copilot & AI Agents

Find out where Copilot shortens the investigation of device issues, policy conflicts and vulnerabilities, then build your own custom Security Copilot agent. Includes a 70-minute hands-on Intune Copilot lab. Led by Saurabh Sarkar, Product Manager 2 at Microsoft.

Use code AINET30 for 30% off, exclusively for you

VIEW THE WORKSHOP →

Keep reading

• An agent is only as good as the data behind it. Modern Network Observability shows you how to collect, normalize and alert on network data with Telegraf, Prometheus and Grafana.

• Your agents will also run on Linux boxes. Agentic AI for Linux Administration covers command verification, rollback planning and human approval before anything changes.

Which model should we use?

That is often the first question when someone proposes adding AI to a network operations workflow. But before we compare models, there is another question worth asking: Does this alert need an AI-generated answer at all?

Okay, so a BGP session goes down and an alert fires. The obvious next step might be to send that alert to an LLM and ask what went wrong.

But the device may be inside a maintenance window. The session may already have recovered by the time the workflow runs. The alert may describe a state that is expected, which means the correct response is not a carefully written root-cause analysis. It is to stop the workflow and say nothing.

This came up during our recent session on building a network observability stack with Christian Adell and David Flores. One of the workflows they demonstrated began with an alert that a BGP session was not up, but the alert did not go directly to the model.

The workflow first checked the intended state, current metrics, relevant logs, and whether the device had been placed in maintenance mode.

Christian’s version of the rule was blunter: if the metric is for an interface in maintenance, please don’t wake up Christian on the weekend.

That is really the value of the Skip path. It prevents an expected condition from becoming an unnecessary diagnosis, notification, or escalation.

The model is the task, not the conditional

If you have worked with Ansible, the pattern should feel familiar.

You gather the facts, evaluate a when: condition, and only then allow the task to execute. You do not ask the task to decide whether it should run.

An AI-assisted troubleshooting workflow should work in much the same way. The model is the task. It should receive the incident only after deterministic checks have established that the incident is real, unexpected, and still active.

In the demonstrated workflow, those checks produced three normal outcomes:

• Proceed meant the BGP peer was expected to be up, the observed state did not match that intention, and the device was not under maintenance. Only this path called the LLM.

• Skip meant the condition was expected, such as when the device was under maintenance. No root-cause analysis was needed.

• Resolved meant the session had already returned to its intended state before the workflow reached the investigation stage.

There should also be a fallback for cases that do not match a known condition. Instead of allowing the model to fill in the gap, the workflow should stop and escalate the incident to an engineer.

The policy therefore does more than decide which analysis to perform. In most of its paths, it decides that the model should not be called at all.

Separate the workflow from the prompt

A simple version of the pattern could look like this:

WORKFLOW INPUTS 

Alert: 

Affected device and peer: 

Intended state: 

Observed state: 

Maintenance status: 

Relevant logs: 

DETERMINISTIC POLICY 

If maintenance status is active: 

    Result = Skip 

Else if observed state now matches intended state: 

    Result = Resolved 

Else if the peer should be established but is down: 

    Result = Proceed 

Else: 

    Result = Stop and escalate to a human 

Only when the result is Proceed does the workflow construct the model request:

MODEL INSTRUCTION 

Using only the evidence provided, explain the most likely 

cause and give the next three verification steps. 

Do not propose or execute a configuration change. 

The same structure works beyond BGP. An interface-flap workflow could check whether the link recovered and whether sufficient alternate capacity exists. An ACL-drift workflow could confirm that the running configuration still differs from the approved baseline before asking a model to explain the impact.

Adding an LLM to every alert does not automatically make an observability workflow more intelligent. Sometimes intelligence means recognizing that the alert is expected, recovered, or irrelevant and ending the workflow before the model has anything to say.

How many alerts in your environment could be closed by a deterministic Skip or Resolved check before they reach an engineer, let alone an LLM? Reply and let me know which one you would filter first.

Next week, we’ll look at another observability problem: what happens when SNMP reports an interface state as 1, gNMI reports it as up, and your AI workflow treats them as two different signals.

Thanks,

Sayali

Editor-in-Chief

Three live workshops this month, each paired with a book if you want to read ahead.

1. October 17th, 9:30 AM EDT | Multi-Agent Network Operations

Build a team of agents that work an incident together, run only playbook-approved commands, and wait for your sign-off before touching the network.

Use code AINET35 for 35% off, exclusively for you

VIEW THE WORKSHOP →

Do you want to build the first agent before you scale to several? Definitely, Building AI Agents for Network Operations walks you through it. A must read.

2. October 24th, 9 am EDT | Build AI Agents: From Local to Production Cloud

Build an agent on your laptop with open models, then take it to Cloud Run with Gemini, MCP and A2A.

Use code AINET35 for 35% off, exclusively for you

VIEW THE WORKSHOP →

Build the agent in the workshop, then point it at your network. Eric Chou’s AI Networking Cookbook covers local LLMs with Ollama, a network co-pilot, and MCP-based monitoring.

3. October 27th, 9 am EDT | Agentic AI for Platform Engineering: Building Agent-Ready Platforms

AI agents are becoming users of your platform. Learn to give them their own identity and scoped permissions, control their tool access, and observe every action, with approval gates and recovery built in. You’ll finish with an agent triaging an alert inside defined security boundaries.

Use code AINET35 for 35% off, exclusively for you

VIEW THE WORKSHOP →

Decide what agents are allowed to do before they touch your platform. Agentic AI for Platform Engineering shows how to have them open pull requests instead of pushing changes, with audit trails, identity controls and human approval where the risk calls for it.

Keep reading

• An agent is only as good as the data behind it. Modern Network Observability shows you how to collect, normalize and alert on network data with Telegraf, Prometheus and Grafana.

• Your agents will also run on Linux boxes. Agentic AI for Linux Administration covers command verification, rollback planning and human approval before anything changes.

Before today’s piece, here is the upcoming event:

Upcoming: Architecting Agentic Development Platforms with Harness Engineering

Sept 30th · Wednesday · 10 AM EDT | Use code FINAL40 & get 40% off

Grab your pass now

Use code FINAL40 for 40% off, this is the last call to join the event, limited seats left!

Kaspar von Grünberg, author of Thinking in Platforms, makes the case that agentic AI failures (hallucination, runaway cost, security incidents) almost always trace back to the platform underneath, not the model. He walks through the three layers of an Agentic Development Platform, tooling, paths, and agent infrastructure, and where to start if you’re already running an IDP. Same argument as today’s issue, just one layer up from Git.

Collect less data if you want better AI answers

You ask an agent why a BGP session dropped, and what comes back is a paragraph that could describe almost any BGP problem. Check the peer. Verify the configuration. Confirm reachability.

The instinct at that point is to blame the model, so you try a bigger one, rewrite the prompt, and add a line about being a senior network engineer.

That is usually not where the problem is.

What a specific answer actually needs

At our workshop last week, David Flores walked through an incident workflow he built with Christian Adell. Both are network developers at CoreWeave, and Christian is one of the authors of the Network Automation Cookbook. An alert fires for a BGP session that is down, and before any model gets called, the workflow assembles an evidence bundle.

It pulls from three places. The source of truth says whether the device is in maintenance, whether this peer is intended, and whether the established state is expected. The metrics say the session is sending routes and receiving none. The logs say the session is failing to establish.

That is six or seven facts. The root cause summary that came out the other end was specific because the retrieval was specific, not because the model was clever. Hand the same model an undifferentiated dump of everything collected in that window and you get the vague paragraph back.

So the question worth asking is not which model writes the best RCA. It is whether your stack can assemble those six facts at all.

Plenty cannot, and the reason is the opposite of what most people assume

The pile nobody queries

Christian’s advice on where these stacks fall over is to ask three things before you collect anything:

• Do we actually need this data?

• Is it healthy in terms of the cardinality it adds to the database?

• Are we going to use it for something?

His observation is that out of everything available to collect, you are usually answering your real operational questions with twenty or thirty metrics. Everything else is there because someone turned on a plugin and nobody turned it off.

David’s version is blunter. He said teams need to be frugal with data, that people always assume they need all of it, and that most of what gets collected ends up looked at roughly once in a lifetime. He also pointed out where that lands you. Query performance degrades, and then you are no longer having a networking conversation. You are in a conversation with your platform team about federated ingestion, indexing tiers, and object storage costs.

None of that is an AI problem. But it becomes one the moment you want an agent to explain an incident, because an agent explaining an incident is a retrieval problem wearing a hat. It can only reason over what it can find, and it can only find what you kept and can query quickly. A landfill and a well-chosen set of twenty metrics look identical on a dashboard. They do not look identical to something trying to answer a question.

You already narrow this in Ansible

If you run Ansible at any scale, you have already made this trade deliberately at least once.

You do not gather every fact on every play. You set gather_subset, or you turn fact gathering off entirely for plays that do not need it, because facts you never reference still cost you time on every single run against every single host. The same logic drives fact caching. You decide up front what the play actually needs, and you stop paying for the rest.

Telemetry is that decision made permanent. The difference is that the cost does not show up as a slow playbook. It shows up eighteen months later as a storage bill, a query that times out, and an agent that cannot find the one log line that mattered.

The useful reframe

Start from the question, not from the device.

David and Christian both make the point that alert design should run top down. You decide what healthy means for your network, usually in terms of availability, reliability, and capacity, and you work backwards to what you need to measure. Christian’s example is that for a backbone you care about session state with your transits and peers, while for a data center fabric you care about how many uplinks are active and whether the fabric sessions are holding.

Do the same thing for the questions you would want an agent to answer. Take the incident type you get paged for most often, and write down the fields you would need in front of you to explain it properly to someone else. Not the data you have. The data the explanation requires.

That list is short. It is almost always shorter than what you are currently collecting, and it almost always includes one or two things you are not collecting at all.

Try this

Pick the alert your team sees most often this month. Write the evidence list for it in five lines: what your source of truth needs to say, what metric confirms or contradicts it, what log lines are relevant, what the expected state is, and what would tell you this is not worth acting on.

Then go and check whether you could actually retrieve all five in under a minute.

If you can, you are in a good position to point a model at it. If you cannot, that gap is the work, and it is work worth doing whether or not an agent ever touches it. Every one of those five fields makes a human faster at three in the morning too.

The stack that makes AI troubleshooting useful is the same stack that makes on-call survivable. That is the part of this nobody puts in a keynote.

Christian and David’s book, Modern Network Observability, goes much deeper on the collection and processing layers if you want the full version of this.

Thanks,
Sayali
Editor-in-Chief

Humble Bundle season is back, and this one’s for anyone building or running infrastructure today.

We’ve just put together 24 books covering Linux, cloud, Kubernetes, Terraform, AWS, Azure, and the AI-native tooling now showing up in DevOps and NetOps workflows. And that’s just the beginning.

Get the entire bundle, worth $976, for just $29.50. Part of what you pay goes to Little Free Library.

Deals like this don’t stick around. Grab yours before it’s gone.

GRAB YOUR BUNDLE

Before today’s piece, here is the upcoming event:

Tomorrow: Build a Network Observability Stack with Tools, Automation, and AI

AI-assisted troubleshooting is only as useful as the telemetry behind it.
Join Christian Adell & David Flores this Saturday, i.e. 19th September

In this hands-on workshop, you’ll build a network observability stack using Telegraf, Prometheus, and Grafana, use PromQL and LogQL to investigate network issues, and see where AI can help with anomaly detection and root-cause analysis.

You’ll also learn where traditional observability methods still do the job better.

Use code FINAL40 for 40% off, this is the last call to join the event, limited seats left!

GRAB YOUR PASS NOW

The AI-Generated Script Looked Fine. The Workflow Wasn’t.

Okay, so here’s a small automation problem that is very easy to miss because nothing in the generated output looks obviously reckless.

You ask Aider to create a system-check script and schedule it to run every ten minutes. It creates the script here:

/home/user/Documents/system_check.sh

Then it adds a cron entry that tries to run this:

/usr/sbin/system_check 

That second file does not exist.

There is another problem hiding in the same response. The command that modifies the crontab sits inside the script being scheduled, which means every run could add the cron entry again.

So one mistake prevents the job from running at all, while the other could keep duplicating the schedule. Neither comes from obscure Bash syntax or an unusual edge case. The individual commands look reasonable when you read them separately.

The workflow breaks because the steps do not agree with one another.

The same shape shows up in network work. An agent writes a candidate ACL to one path while the playbook loads the previously approved file from another. Or the change runs against one inventory group while the validation step checks a different device. The configuration can be perfectly valid and the verification command can return cleanly, but neither tells you whether the correct artifact reached the correct target.

The mistake is between the steps

We tend to review AI-generated automation one command at a time.

Does the loop work? Yes.

Is the cron expression valid? Yes.

Does the file have execute permission? Yes.

That kind of review is still necessary, but it misses the actual failure here. Aider creates one file and schedules another. It also treats a one-time setup action as part of a recurring task.

The problem lives in the handoff between generation, scheduling, and execution.

Network automation has more of these handoffs than a small Bash script. A model generates a configuration, a playbook reads it, an inventory decides where it goes, a device applies it, and another tool verifies the result. Every component can behave exactly as designed while the complete workflow does the wrong thing.

That is why reviewing the generated command is not enough. You also need to review what each step leaves behind for the next one.

You already check this in Ansible

If you use Ansible, none of this is a new discipline.

You would not review the syntax of each task and ignore the inventory, variable resolution, conditions, handlers, or task order. A task can be correct and still run against the wrong hosts. A template can render successfully and still use the wrong source data. A handler can work exactly as written and still fire more often than intended.

The AI version has the same problem, except the generated output often looks complete enough that you are tempted to review it as one finished answer.

It is not one answer. It is a chain of assumptions.

The model assumes where the file will be created, which account will run it, how often it should execute, what permissions it will inherit, and how success will be checked. If those assumptions are not made explicit, you end up reviewing syntax while the real risk sits somewhere else.

I found the Aider example in Imran Afzal’s newly published Agentic AI for Linux Administration. What I liked about the example was that the incorrect output was not quietly replaced with a perfect version. The book keeps the generated draft visible, points out the two broken handoffs, and corrects them before the commands are approved.

That is much closer to how these tools behave in real work. The first response may be useful, but useful is not the same as ready to run.

Ask for a handoff review

Before approving a generated workflow, ask the model to review the connections between its own steps. You can use this after it produces a script, playbook, or multi-step troubleshooting plan:

Review this workflow as a chain of dependent steps. 

For each step, identify: 

- what it creates or changes 

- the exact filename, path, variable, inventory group, host, or device involved 

- which later step consumes that output 

- whether the action should happen once or repeatedly 

- whether rerunning it is safe 

- how the result will be verified independently 

Flag any mismatch between what one step produces and what the next step expects. 

Do not rewrite or execute the workflow yet. 

The last line matters. You want the model to expose its assumptions before it tries to repair them, because an immediate rewrite can hide the original mismatch and make the new answer look safer than it has been shown to be.

You can then compare that handoff map with the environment you control: the real inventory, paths, accounts, device groups, approval gates, and validation sources.

For a network change, I would pay particular attention to three questions:

• Did the workflow generate the intended artifact?

• Did it apply that artifact to the intended devices?

• Did the verification step inspect those same devices after the change?

If any one of those answers is unclear, the workflow is not ready.

A confirmation prompt does not solve this

Aider did ask for approval before creating the file and running the shell commands. That is good behavior, but the approval gate only works if the reviewer can see what is wrong.

A human clicking Yes does not make the workflow safe. It only records that someone approved what was shown to them.

This is why human-in-the-loop cannot mean placing a confirmation box in front of every command. The human needs enough context to review the effect of the complete workflow, including the paths, targets, frequency, dependencies, and verification method.

You already apply this thinking when reviewing a playbook or a change request. The useful shift is to stop treating AI-generated automation as a collection of plausible commands and start treating it as a system whose handoffs need to agree.

The model may get every command right and still build the wrong workflow.

If you have seen an automation fail because two individually correct steps did not line up, hit reply and tell me what happened. These small handoff failures are usually where the useful lessons are hiding.

Thanks,

Sayali

Editor-in-Chief

Also coming up

Architecting Agentic Development Platforms with Harness Engineering

Sept 30 · Wednesday · 10 AM EDT | Use code AINET35 & get 35% off

GRAB YOUR PASS NOW

This one’s about building the actual platform underneath agentic AI, not just bolting a coding agent onto your existing setup. You’ll walk through the three layers, tooling, paths, and agent infrastructure, that separate a team that’s genuinely running agents in production from one that’s just demoing them. If you’ve been told to “figure out AI” for your platform, this gives you the real architecture to do it.

One last note, Humble Bundle is back, bigger than ever covering Linux, networking, cloud infrastructure, Entra ID - all for just $25, so what are you waiting for

Grab your bundle

Okay, so imagine this.

You update the model your network incident agent runs on. The change goes through, every automated check passes, and all the dashboards stay green, so on the surface, everything looks good.

Then you replay an incident the agent had handled correctly before. This time, it overlooks part of the device state and reaches a different (and worse) conclusion.

This is where things get a little tricky because nothing technically failed. The API responded, the tools ran, and the output matched the expected format. The agent simply wasn’t doing the job as well as it had before.

The dashboard did its job

To be fair, this isn’t a monitoring failure.

Your dashboard can tell you whether the model responded, how long the call took, how many tokens it consumed, and whether any tools failed. It can warn you about retries, timeouts, or a sudden increase in cost.

All of that matters. If the agent takes 40 seconds to answer or one of its tools keeps returning a 403, you need to know.

But a schema can only confirm that the response contains a likelyCause field. It cannot confirm that the cause is supported by the evidence.

That gap is where evaluation comes in.

Monitoring and evaluation answer different questions

Monitoring asks whether the system is healthy. Evaluation asks whether it is still doing the job you trusted it to do.

The distinction matters because the model isn’t the only moving part. Editing the system prompt may change how the agent weighs evidence. Updating a tool description may affect whether it calls that tool at all, while adding more context may bury the signal that previously led it to the right conclusion.

Change the model, the prompt, a tool, or the context sources, and you have a different agent to evaluate, even when the workflow keeps passing.

Trevoir Williams draws this line clearly in Agentic AI for DevOps Engineers. In the book’s multi-agent incident-response workflow, the usual operational signals are captured through OpenTelemetry, but the generated response also goes through a separate evaluation checkpoint.

One detail I appreciated is that the book doesn’t treat a structurally complete response as proof that it is factually correct. An answer may contain the incident ID, likely cause, rollback recommendation, validation plan, and stakeholder message, yet still be poorly grounded or unsafe.

In other words, all the boxes can be ticked and the answer can still be wrong.

Give the updated agent incidents you already understand

The simplest place to start is with a small replay pack built from network incidents whose outcomes are already known.

One case could be a sub-interface shut during a change window that took a BGP session down with it, the kind of incident Lumen has described publicly. Add an ACL that has drifted from its approved baseline, an interface flap where the available signals do not support a confident root cause, a pyATS collection that returns no usable result, and a routing-policy change that unexpectedly suppresses prefixes.

These cases test very different behaviors. Between them they ask whether the agent can connect a local configuration change to wider routing impact, whether it can spot drift without treating every difference as a problem, and whether it can admit when the evidence is inconclusive. The pyATS case is the most revealing of the set: when the collection returns nothing, does the agent report the missing evidence or quietly interpret the empty result as a healthy device?

For each case, record:

• The evidence the agent must identify

• The conclusion that evidence supports

• Claims or actions it must not recommend

• The tools it should use

• The safest acceptable next step

Now run the same cases through the current agent and the updated version. You’re not checking whether both responses use the same wording. You’re checking whether the new version missed important evidence, called an unnecessary tool, became more certain than the data allowed, or recommended an action the old version correctly avoided.

If you use Ansible, you already know this instinct. You wouldn’t upgrade a collection, confirm that it installed successfully, and assume every existing playbook still behaves exactly as before. You rerun the workflows that matter and compare the results.

An AI agent deserves the same regression discipline.

Don’t let a good average hide a dangerous answer

An overall quality score can hide the failure you most need to see.

An updated agent might perform slightly better across four cases and still recommend an unsafe change in the fifth. The average may look respectable. The fifth answer is still a release blocker.

Assess whether the agent followed the task, selected the right tools, and grounded its conclusions in the supplied evidence. Then ask the two that matter most: did it stay within its safety boundaries, and would an engineer actually find the answer useful?

Some failures should carry more weight than others. A slightly longer response is not equivalent to an unsupported rollback recommendation, and a critical safety failure should block the update regardless of the average score.

Over time, track how engineers respond to the agent’s recommendations. If more answers are being corrected, rejected, or escalated after an update, that tells you more than the response time does. An agent that produces wrong answers faster is not improving.

Treat the model change like a production change

The easiest time to discover a regression is before the updated agent is involved in a real incident.

Whenever you change the model, prompt, tools, context sources, or orchestration logic, replay the cases that matter. Compare the results with the version you already trust, review what changed, and decide whether the difference is acceptable.

The deployment being green only means the new version is running.

It doesn’t mean it is ready.

Before you go, take one agent workflow your team is experimenting with and ask this: if the model changed tomorrow, which past network incidents would tell us whether the agent had become better or worse?

If you don’t have an answer yet, that is probably the first evaluation set you need to build.

Thanks,

Sayali

Editor-in-Chief

Also coming up

We’ve got three genuinely exciting sessions happening this September, in case you have registered already, here’s what you might be missing out on

A quick note: We’ve moved registration for this workshop to Luma. Nothing else about the workshop is changing, the same speaker, same time, and same content. If you’ve already registered through Eventbrite, your existing booking remains valid.

Engineering GPU Infrastructure for AI Workloads

Sept 17 · Thursday · 10 AM EDT | Use code AINET30 & get 30% off

Most platform engineers know how to deploy an application. Things get more complicated when that application needs a GPU.

This hands-on workshop covers the infrastructure behind serving an LLM on Amazon EKS, from GPU-aware scheduling and the NVIDIA stack to autoscaling, monitoring, and controlling idle GPU spend.

Grab your pass on Luma →

Building a Network Observability Stack with Tools, Automation, and AI

Sept 19 · Saturday · 10 AM EDT | Use code AINET30 & get 30% off

Grab your pass on Luma →

This one’s about building a real observability stack from scratch, Telegraf, Prometheus, and Grafana, so you can actually see what’s happening on your network instead of guessing. You’ll also get into where AI genuinely speeds up troubleshooting, and where the old-school methods still win. If you’ve ever wanted a cleaner, faster way to catch problems before they turn into incidents, this is that session.

Architecting Agentic Development Platforms with Harness Engineering

Sept 29 · Tuesday · 10 AM EDT | Use code AINET30 & get 30% off

Grab your pass on Luma →

This one’s about building the actual platform underneath agentic AI, not just bolting a coding agent onto your existing setup. You’ll walk through the three layers, tooling, paths, and agent infrastructure, that separate a team that’s genuinely running agents in production from one that’s just demoing them. If you’ve been told to “figure out AI” for your platform, this gives you the real architecture to do it.

One last note, Humble Bundle is back, bigger than ever covering Linux, networking, cloud infrastructure, Entra ID - all for just $25, so what are you waiting for

Grab your bundle

Before we get into today’s issue, here’s one upcoming session for anyone thinking about what it takes to run AI workloads beyond a laptop.

Engineering GPU Infrastructure for AI Workloads

Sept 17 · Thursday · 10 AM EDT | Use code AINET30 & get 30% off

Most platform engineers know how to deploy an application. Things get more complicated when that application needs a GPU.

This hands-on workshop covers the infrastructure behind serving an LLM on Amazon EKS, from GPU-aware scheduling and the NVIDIA stack to autoscaling, monitoring, and controlling idle GPU spend.

Grab your pass on Luma →

Quick heads up: we’ve moved registration to Luma for the upcoming ones, just working through a few things on our end with Eventbrite. Nothing else about the workshop changes, same speaker, same time, same content.

Now, back to where I think a local LLM should begin.

Your first local LLM should never touch the network

I understand why running an LLM locally feels like the safe option. The configuration stays on your machine, there is no external API call, and you have more control over where the data goes.

That is already better than pasting a production configuration into a random chat window.

But it only answers one question: Where is the model running?

It does not answer the more important question: What is the model allowed to reach?

A local model with device credentials, an SSH tool, or access to a network controller can still cause plenty of trouble. It just causes that trouble from inside your environment.

Give it a Prime Directive

There’s a reason the Prime Directive from Star Trek works so well as a mental model here. The rule is not simply “be careful.” It draws a clear line between observing something and interfering with it.

That is the line I would use for a first local-LLM experiment:

Observe the configuration, document the configuration, and change nothing.

If someone on my team wanted to try a local model for network work, I would not begin by connecting it to a router. I would give it a sanitized copy of a configuration and one fairly boring job: turn that configuration into draft documentation.

And honestly, boring is exactly what we want here.

The model gets a real networking task, the engineer gets something useful, and nobody has to hand an experimental workflow the keys to the network.

Start with a configuration copy

Take a configuration from a lab device or a sanitized copy from your repository.

The model can extract details that are already present, such as the hostname, local AS, router ID, configured neighbors, and advertised networks, and arrange them into readable Markdown.

A starter prompt could look like this:

You are reviewing a static BGP configuration. 


Using only the supplied configuration, document: 

- The hostname 

- The local AS number and router ID 

- Each configured BGP neighbor and remote AS 

- Neighbor descriptions, where present 

- Networks configured for advertisement 
 

Rules: 

- Do not infer operational state or reachability 

- Do not invent missing values 

- Label anything that cannot be confirmed as unknown 

- Quote the relevant configuration line under each finding 

- Return the result as Markdown 

Then paste the configuration beneath it.

The important part is not the Markdown. It is the phrase “using only the supplied configuration.”

A static configuration can tell the model what has been configured. It cannot tell the model whether a neighbor is currently established, whether a route is being advertised successfully, or whether an interface is up.

If you ask it to diagnose a live fault from a static file, it has two choices: admit that it does not know or fill in the gap.

Your prompt and review process should make the first option much easier.

Give the workflow an observation-only contract

Before running the experiment, I would write down the boundary in plain language:

Input: 
A sanitized, read-only configuration copy  

Output: 
Draft Markdown documentation  

The model can: 
Read the copied file 
Extract configured facts 
Organize those facts into documentation 

The model cannot: 
Connect to a live device 
Use SSH, CLI, or controller APIs 
Access credentials 
Modify the source configuration 
Apply configuration commands 

Human checkpoint: 
An engineer verifies every claim against the source before the document is approved 

This may feel obvious, but write it down anyway.

Small experiments have a habit of growing. Someone adds another file, then a script, then a token. Six weeks later, the “documentation assistant” can query a controller and nobody remembers when that decision was made.

The contract gives the team something concrete to review whenever the workflow changes.

Local is a deployment choice, not a security model

This is the mistake I keep seeing in conversations about local AI. People use local and safe as though they mean the same thing. They DON’T.

Running the model locally can reduce the need to send configuration data to a third-party service. That is useful. But you still need to think about where copied configurations are stored, whether prompts and outputs are logged, who can access the host, and whether the model endpoint is exposed elsewhere on the network. Most importantly, you still need to control its tools.

If the model has no credentials and no path to a live device, a bad answer becomes a bad draft. If it can execute commands, that same bad answer can become an incident.

That is why I would keep the first workflow deliberately limited: let the model read a copy, let it write a draft, and let an engineer decide whether that draft is correct.

You can always add capability later, because removing it after people depend on it is much harder.

Try this with one configuration today

Pick one lab configuration your team already understands. Remove credentials, customer details, public addresses, and anything else that does not belong in the test.

Run the documentation prompt through a local model and review the result line by line. As you review it, keep three questions in mind:

• Which statements came directly from the configuration?

• Which statements were reasonable but unsupported?

• Which important details did the model miss?

Save the prompt, source file, output, and corrected version together. That gives you a small, repeatable test you can run again when you change the model or prompt.

The goal is not to prove that the model can produce a nice-looking document; it is to learn whether it can stay inside the evidence you gave it.

Once it can do that consistently, you can consider a slightly more demanding task.

And no, I still wouldn’t give it the router password.

Cheers,

Sayali

Editor-in-Chief

Also coming up

We’ve got two more genuinely exciting sessions happening this September, and both are worth your evening.

P.S. we’ve moved registration to Luma for the upcoming ones, just working through a few things on our end with Eventbrite. No changes on your end.

Building a Network Observability Stack with Tools, Automation, and AI

Sept 19 · Saturday · 10 AM EDT | Use code AINET30 & get 30% off

Grab your pass on Luma →

This one's about building a real observability stack from scratch, Telegraf, Prometheus, and Grafana, so you can actually see what's happening on your network instead of guessing. You'll also get into where AI genuinely speeds up troubleshooting, and where the old-school methods still win. If you've ever wanted a cleaner, faster way to catch problems before they turn into incidents, this is that session.

Architecting Agentic Development Platforms with Harness Engineering

Sept 29 · Tuesday · 10 AM EDT | Use code AINET30 & get 30% off

Grab your pass on Luma →

This one's about building the actual platform underneath agentic AI, not just bolting a coding agent onto your existing setup. You'll walk through the three layers, tooling, paths, and agent infrastructure, that separate a team that's genuinely running agents in production from one that's just demoing them. If you've been told to "figure out AI" for your platform, this gives you the real architecture to do it.

If you have already tried a local model with network configurations, hit reply and tell me what it got wrong. Those failure cases are usually more useful than another polished demo.

Before today’s piece, if you’re building toward IT infrastructure, support, or cloud ops roles, tomorrow’s session is worth your Saturday evening.

Aug 29th - Hybrid Identity with Active Directory & Microsoft Entra ID

LAST 24 HOURS - CLAIM 50% OFF WITH FINAL50

GET YOUR SEAT →

Robert McMillen walks through how AD and Entra ID work on their own and together, so you can actually explain identity concepts in interviews instead of just recognizing the terms.

Last chance to grab a seat before it goes live tomorrow, so don’t miss out!

Now, onto today’s piece.

In Minority Report, police arrest people for murders that three psychic “Precogs” predict they will commit. The system’s mistake is not making predictions. It is treating a projected future as fact.

Your LLM has a prompt containing seven days of bandwidth measurements.

Don’t make the same mistake.

The problem in Minority Report was not prediction itself. It was acting as if one predicted future had already happened.

It is easy to give a model recent utilization data and ask:

Predict bandwidth usage for the next 24 hours. Identify when this link will reach capacity 

The answer may look convincing. You may get an hourly forecast, a predicted peak and a confident recommendation to increase capacity. But you might want to check the following:

• But where did those numbers come from?

• What forecasting method was used?

• Did the model account for seasonality?

• Was the recent spike an actual trend or the end-of-month backup window?

• What is the expected error?

• How wide is the uncertainty range?

If the response cannot answer those questions, you do not have a forecast. You have plausible numbers in forecast-shaped prose.

Separate calculation from explanation

A general chat model can be useful in a performance-analysis workflow. It just should not quietly become the entire workflow.

There are four different jobs here.

• Observed data comes from your monitoring system. Interface utilization, latency, packet loss and error rates belong here, together with the collection period and sampling interval.

• Calculated metrics come from code. Baselines, peaks, rolling averages, rates of change and threshold breaches should be reproducible from the same input.

• Forecasts come from an identified method that has been tested on historical data. The result should include its horizon, assumptions and uncertainty, not only a line climbing towards 100%.

• Interpretation is where the LLM can help. It can summarize the calculated evidence, explain operational implications and suggest what an engineer should inspect next.

Collapsing those jobs into one prompt hides which parts are measured, which are calculated and which are merely predicted.

The research is more careful than the prompt

There is credible research showing that language models can perform time-series forecasting.

But those experiments involve more than pasting a column of numbers into a chat window.

The LLMTime research presented at NeurIPS preprocesses and rescales numerical sequences, controls how they are tokenized and evaluates the results against established forecasting methods.

A later study of LLM forecasting behaviour found that performance varies with the structure of the data. Models performed better when patterns and seasonality were clear, struggled more with multiple periods and were particularly sensitive to the most recent part of the input.

That is useful work. It is also very different from asking a general chat model to inspect recent telemetry and trusting the next 24 numbers it produces.

The architecture may contain an LLM. That does not remove the need for a forecasting method, evaluation data and error measurement.

Give every prediction a contract

Before a prediction reaches an engineer, or another agent, make the workflow return a structured record:

Metric: 
Device or interface: 
Observation window: 
Sampling interval: 

Observed: 
Calculated: 

Forecast method: 
Forecast horizon: 
Expected range: 
Uncertainty or error: 
Assumptions:  

LLM interpretation: 
Recommended checks: 
Unknowns: 

The boundaries matter more than the format.

• Observed must contain only values present in the source telemetry.

• Calculated must contain values your code can reproduce.

• Forecast must name the method that produced it. If the workflow cannot provide an uncertainty estimate or historical error measure, it should say so.

• LLM interpretation can explain the result, but it cannot introduce a new peak, probability or time-to-capacity that the forecasting layer did not supply.

That one restriction prevents the explanation layer from quietly rewriting the evidence.

Constrain the model to its actual job

Instead of asking the model to predict the next outage, give it the output from your approved analytics or forecasting pipeline:

You are reviewing the output of a network-performance 

forecasting pipeline. 

 
Rules: 

- Treat only the supplied values as facts. 

- Do not calculate or invent additional forecasts. 

- Keep observed, calculated and forecast values separate. 

- If uncertainty or historical error is missing, state that 

  the forecast is incomplete. 

- Recommend investigation steps, not configuration changes. 
 
Observed data: 
[INSERT OBSERVED METRICS] 

Calculated metrics: 
[INSERT REPRODUCIBLE CALCULATIONS] 

Forecast: 
[INSERT METHOD, HORIZON, RANGE AND UNCERTAINTY] 

 
Return: 

1. Operational interpretation 

2. Evidence supporting the interpretation 

3. Conditions that could invalidate the forecast 

4. Checks an engineer should perform 

5. Unknowns 

Now the model is doing something it is well suited to do: turning structured technical results into a useful operational explanation.

It is not being asked to disguise extrapolation as telemetry.

Test the pipeline backwards

You would not trust a monitoring alert without knowing which threshold fired. Apply the same discipline here.

When a forecast says an interface will reach capacity within 48 hours, work backwards:

• Which output contains that prediction?

• Which method produced it?

• Which observations influenced it?

• How has that method performed on previous windows?

• What uncertainty surrounds the result?

• What condition would prove it wrong?

If the trail ends at “the LLM said so,” the workflow is not ready to drive an operational decision.

Try this with one metric

Pick one interface and one metric your team already understands.Run the comparison twice.

First, give the raw observations to a general chat model and ask it to predict the next 24 hours.

Then run the data through an existing analytics or forecasting method. Give the resulting baseline, forecast and uncertainty to the model using the constrained prompt above.

Compare the responses.

The first may sound more certain. The second should make it easier to see where every claim came from, and where the unknowns remain.

That is the result you want.

The problem in Minority Report was not that the system made predictions. It was that people began treating a predicted future as an observed fact.

Your NOC needs that distinction too.

Calculate the trend. Validate the forecast. Let the LLM explain it. But keep the future labelled as the future.

Cheers,

Sayali

Editor-in-Chief

A prediction is only as trustworthy as the telemetry beneath it. Before adding forecasting or AI-generated interpretation, you need an observability stack that can show what the network is actually doing.

September 19 – Building a Network Observability Stack with Tools, Automation and AI

LEARN MORE

If the argument above only holds if your observability stack can actually produce clean, structured telemetry in the first place. Most NOCs can’t yet. They have dashboards, but not the pipeline that turns raw signals into something a forecasting method (or an LLM) can trust.

Christian Adell and David Flores are building that pipeline live, with Telegraf, Prometheus, Grafana, PromQL and LogQL.
You’ll walk out with working dashboards, alerts that understand intent instead of just thresholds, and a clear line for where AI-assisted anomaly detection actually earns its place versus where it doesn’t.

If the “contract” idea above made sense to you, this is the workshop that gets you the observed and calculated layers it depends on.

Thanks for reading The AI Network Engineer by Packt. Subscribe for free to receive future issues.

Before today’s piece, one upcoming hands-on session for anyone building AI-assisted NetOps workflows:

Build an AI Assistant for Network Troubleshooting,
August 25th, 9 AM EDT

Last chance to register - Claim 50% OFF

Most AI assistants can write you a plausible-looking config. They can’t tell you if it matches your actual network, because they’ve never seen your runbooks, your device configs, or your topology.

Sif Baksh (Principal Solutions Architect at Tines) spends this workshop building the piece that’s missing: a RAG-powered NetOps assistant that answers from your own runbooks and device configs instead of general training data, with guardrails that make it flag what it doesn’t know instead of guessing.

Four hours. Four labs. You’ll leave with a working assistant and a workflow you can keep testing after the session.

For AI Network Engineer readers: code FINAL50 gets you 50% off all passes.

P.S. Sif wrote the book on this, literally. Building AI Agents for Network Operations goes deeper into the architecture than four hours allows. Bundle it at checkout, or grab it on its own if Tuesday doesn’t work.

Now, back to today’s test.

Your AI model needs an acceptance test

Ask someone which AI model you should use for network operations and you’ll usually get a model name.

That’s the wrong place to start.

A model can perform well on a general benchmark and still miss a requirement your workflow cannot afford to miss. Another may be unnecessary for routine summaries but worth the cost for complex troubleshooting.

You don’t need a favourite model.

You need an acceptance test

Start with the job, not the model

Suppose you give two models this prompt:

Configure OSPF on a router. 

Both responses may look reasonable. But the task is too vague to tell you much.

Which platform? Which area? What should become the router ID? Should the response include verification? What information is the model allowed to assume?

A better test looks like this:

Prepare an OSPF configuration plan for a Cisco IOS-XE router.  

Requirements: 

 - Use area 0 

 - Use Loopback 0 as the router ID 

 - Do not invent interface addresses 

 - List any missing information as an assumption 

 - Include verification commands  

Return: 

 1. Configuration plan 

 2. Assumptions 

 3. Verification steps 

Now the model has constraints it can either satisfy or miss.

That makes the response easier to evaluate. It also makes failures visible before you spend time debating which answer sounds more polished.

Write the answer key first

Before running the prompt, decide what a passing response must contain.

For the OSPF task, you might expect:

• IOS-XE-compatible syntax

• Area 0 applied correctly

• Loopback 0 used as the router ID

• Missing interface or address information identified

• Verification commands included

Then define the hard failures:

• Invented IP addresses

• Syntax from another network operating system

• Unstated assumptions

• Missing verification

• A claim that the configuration is ready to apply without checking the current device state

This is the part most informal model comparisons skip.

If you wait until after reading the responses to decide what matters, the most fluent answer tends to set the standard. Your evaluation becomes a writing contest.

The answer key prevents that.

Run every candidate under the same conditions

Use the same task, context, return format and response settings for every candidate model.

Save the raw responses rather than relying on what you remember from the chat window. Record the model version and the date too. Model behaviour changes, and an evaluation you cannot reproduce is only a snapshot.

A compact test record can look like this:

Task ID:  

Workflow:  

Prompt:  

Expected facts:  

Hard failures:  

Required format:  

Model and version:  

Response:  

Reviewer result:  

Observed cost:  

Test date: 

For a small comparison, a spreadsheet is enough. The important thing is that every model sees the same test and every response faces the same standard.

Don’t ask another model to be the only judge

Using an evaluator model can help sort a large response set. It can identify missing sections, compare formats and produce an initial score.

But it should not own the answer key.

The evaluator is still a model. It can overlook the same plausible-looking technical mistake you are trying to catch.

Let it assist with the comparison. Keep a network engineer responsible for defining the expected facts, reviewing hard failures and approving the result.

For a fairer comparison, hide the model names during review. “Response A” and “Response B” are harder to favour than two brands the reviewer already has opinions about.

Score by workflow, not only by model

Do not collapse every result into one overall number too early.

Separate the tasks by the work the model will perform:

• Configuration planning

• Troubleshooting

• Documentation

• Log or ticket summarization

• Structured data extraction

A model that performs well on documentation may not be the strongest choice for troubleshooting. Another may follow a JSON schema reliably but produce weak operational recommendations.

Different workflows may justify different models.

You already work this way with network tools. The application used for packet analysis does not need to be the same one used for configuration management. Model selection should follow the same logic.

Compare cost only after quality

Cost matters, especially when a workflow will run hundreds or thousands of times.

But a cheap response that fails a mandatory technical check is not better value. It is a failed response delivered inexpensively.

Use the acceptance criteria as the first gate:

• Did the response include the required facts?

• Did it avoid every hard failure?

• Did it follow the requested format?

Only the models that pass should move into the cost comparison.

Then you can ask whether the improvement from a more expensive model is meaningful for this particular workflow.

Try this today

Choose two models and four tasks your team already understands:

• Two configuration-planning tasks

• Two troubleshooting tasks

Use resolved tickets, reviewed MOPs or lab scenarios as your source. Remove credentials, customer data and sensitive infrastructure details before using an external service.

Write the expected facts and hard failures first. Run each model under the same conditions. Save the responses and review them without the model names visible.

You do not need a hundred-question benchmark to learn something useful.

Four well-chosen network tasks will tell you more about your workflow than a general leaderboard ever will.

Stop asking which model is best.

Ask which model passed your acceptance test.

Cheers,
Sayali
Editor-in-Chief

Upcoming live workshops

The tool boundary is only half the production story. The next questions are what happens after an agent proposes a change and how access is governed. These two upcoming sessions approach those problems from different sides.

Aug 27 · Agentic AI for Infrastructure Engineering: From Chatbots to Operators

Ritesh Vajariya, founder and CEO of AI Guru, will show you how to build an infrastructure agent that reads pod events, correlates live metrics and proposes fixes—but executes them only after approval.

You’ll work through seven production-grade failure scenarios injected into your own local Kubernetes cluster, which you can keep and rerun after the session.

Claim 50% Off

Aug 29th · Active Directory and Entra ID in a Modern Hybrid Architecture

Professor Robert McMillen breaks down how traditional Active Directory and Entra ID work together in real hybrid environments: domains and group policy on one side, cloud identity and SSO on the other, and Entra Connect bridging them. Built for IT pros moving into infrastructure or identity-focused roles.

Book Now

If this issue helped sharpen the conversation, please like it or leave a comment so more platform, cloud, security and infrastructure engineers can find it.

Thanks for reading The AI Network Engineer by Packt. Subscribe for free to receive future issues.

Before today’s piece, here’s a practical way to explore the same production-safety questions hands-on.

Build an AI Assistant for Network Troubleshooting,
August 25th, 9 AM EDT

Register now at 40% off

In this four-hour workshop, Sif Baksh will guide you through building a RAG-powered NetOps assistant that answers from your own runbooks and device configurations, with guardrails around how those answers are produced and used.

You’ll leave with four completed labs and a workflow you can continue testing after the session.

Four hours. Four labs. For AI Network Engineer readers, use code LIMITED40 for 40% off all passes.

P.S. Sif’s book, Building AI Agents for Network Operations, covers the same architecture in greater depth. You can bundle it with the event pass at checkout or purchase it separately if the workshop date doesn’t work for you.

Now, back to the tool boundary.

Why run_command () doesn’t belong in your first network agent

You’re building the tool layer for a network agent. You already have a device inventory and an SSH library that works.

There’s an obvious shortcut sitting right there:

run_command(device, command) 

One function. Every command. Every vendor. Anything you haven’t anticipated yet.

You could ship it this afternoon.

Don’t.

Flexibility is the risk surface

A general-purpose tool means you don’t have to enumerate what the agent might need.

That is also what makes it dangerous.

The model composes the action, while your policy layer receives an arbitrary string at runtime and has to decide whether it is safe. Every new command and vendor behaviour expands the set of cases that layer must interpret correctly.

You end up building a command parser and treating it as a security boundary.

Narrow tools move that decision from runtime to design time.

bgp_summary(device) 

That tool has one purpose. It accepts a known argument, returns a defined shape and cannot be persuaded to perform an unrelated operation.

The safety property is structural. It does not depend on the model remembering an instruction or your code correctly interpreting a new command string.

Sif Baksh draws this boundary clearly in Building AI Agents for Network Operations: prompts can guide behaviour, but code enforces policy.

A tool that cannot perform an unsafe action is stronger than a prompt asking the model not to perform one.

Start with the questions your team actually asks

Your first tool set does not need to cover every operational possibility. It needs to cover a small number of useful, low-risk questions.

device_status(device) · Low risk
Allow authenticated users and approved devices.

interface_status(device) · Low risk
Allow and log every call.

bgp_summary(device) · Low risk
Allow and log every call.

show_command(device, command) · Medium risk
Accept commands only from a fixed allowlist.

run_command(device, command) · High risk
Replace with narrower tools.

Configuration-change tools · High risk
Require approval, a ticket reference, a diff, and a rollback plan.

Restart or clear-session tools · High risk
Keep out of scope for the first pilot.

show_command() is the compromise.

It is general enough to support several read-only checks, but the requested command must match an approved list. If it is not on that list, the tool refuses the request and records the decision.

This is different from checking whether an arbitrary command looks safe. The allowlist defines what is permitted before the agent runs.

More tools also mean more possible actions to evaluate, test and audit. Start with tools that map to real operational questions. Add another only when you can define its purpose, inputs, outputs and failure behaviour.

Write down the contract

Once a tool exists, other components start depending on it: the agent, another client and eventually someone else’s workflow.

Treat it as an interface with a documented contract:

tool_contract: 

  name: "bgp_summary" 

  owner: "Network automation team" 

  purpose: "Return BGP neighbor state for an approved device" 

 

  arguments: 

    device: 

      type: "string" 

      required: true 

      validation: "must exist in approved device allowlist" 

 

  returns: 

    device: "string" 

    total_peers: "integer" 

    established_peers: "integer" 

    neighbors: "array" 

 

  blocked_when: 

    - "device is unknown" 

    - "backend times out" 

 

  logs: 

    - "user_or_client_id" 

    - "tool" 

    - "device" 

    - "decision.allowed" 

    - "duration_ms" 

Two fields do disproportionate work:

• blocked_when forces you to decide how the tool fails before you discover its behaviour during an incident.

• owner puts a name against future decisions. If every tool belongs to everyone, nobody owns the boundary when it needs to change.

Once other components depend on the tool’s name, arguments or return shape, changing them becomes a compatibility decision, not a casual refactor.

Log the decision, not only the call

Every invocation should leave a structured record, including blocked requests.

{ 

  "timestamp": "2026-06-28T18:36:38Z", 

  "user_or_client_id": "noc-operator-1", 

  "tool": "show_command", 

  "device": "spine1", 

  "command": "show ip bgp summary", 

  "decision": { 

    "allowed": true, 

    "reason": "read-only command approved", 

    "risk": "low", 

    "requires_approval": false 

  }, 

  "result_summary": "success" 

} 

The decision block is the important part. It records not only what happened, but why the system permitted it.

Blocked calls are useful too. They show which capabilities users keep requesting. That gives you evidence for deciding whether to build a narrow version properly or keep the boundary closed.

Keep secrets and full tool payloads out of these records. Log enough to audit the action without creating another place for sensitive data to collect.

Try this on your tool layer

Start with three checks:

1. Inventory the tool surface

List every tool the agent can call. For each one, describe the worst thing it could do if called with the least helpful arguments the model could construct.

If the answer makes you uncomfortable, narrow or disable the tool.

2. Put tool access in configuration

{ 

  "tools": { 

    "device_status": { 

      "enabled": true 

    }, 

    "bgp_summary": { 

      "enabled": true 

    }, 

    "show_command": { 

      "enabled": true, 

      "read_only_only": true 

    }, 

    "configure_interface": { 

      "enabled": false 

    } 

  }, 

  "global_kill_switch": false, 

  "allowed_environment": "read_only_pilot" 

} 

This lets you enable capabilities for a limited pilot, keep write tools disabled until the approval path exists and reduce the agent’s scope without changing its prompt.

3. Ask how quickly someone else can stop it.

If the agent starts behaving unexpectedly at 4 p.m. on a Friday, how quickly can an operator who did not build it disable its tools?

If the answer involves finding a repository, editing a prompt and waiting for a deployment, the shutdown path is not ready.

The safest agent is not the one you expect to make good decisions every time.

It is the one whose tools make the dangerous decisions unavailable.

Further reading: This issue draws on Building AI Agents for Network Operations by Sif Baksh. Chapter 9 covers authentication, secrets, approval workflows, observability, staged rollout and the production go/no-go review. Appendix A includes tool-contract, runbook and readiness worksheets.

EXPLORE THE BOOK →

What’s next: Narrow tools define what an agent can do. The next production question is who can request an action, and who has the authority to approve it.

Cheers,
Sayali
Editor-in-Chief

Upcoming live workshops

The tool boundary is only half the production story. The next questions are what happens after an agent proposes a change and how access is governed. These two upcoming sessions approach those problems from different sides.

Aug 27th · Agentic AI for Infrastructure Engineering: From Chatbots to Operators

Ritesh Vajariya, founder & CEO - AI Guru, will show you how to build an infrastructure agent that reads pod events, correlates live metrics and proposes fixes—but executes them only after approval.

You’ll work through seven production-grade failure scenarios injected into your own local Kubernetes cluster, which you can keep and rerun after the session.

Book Now

If your Docker and Kubernetes fundamentals need shoring up before you get there, The Ultimate Docker Container Book (4th edition) by Dr. Gabriel Schenker is the deepest single resource we carry on containers through orchestration, and its latest edition adds AI-driven DevOps patterns on top.

Check it out on Amazon

Aug 29th · Active Directory and Entra ID in a Modern Hybrid Architecture

Professor Robert McMillen breaks down how traditional Active Directory and Entra ID work together in real hybrid environments: domains and group policy on one side, cloud identity and SSO on the other, and Entra Connect bridging them. Built for IT pros moving into infrastructure or identity-focused roles.

Book Now

Since the session is already an add-on to the Azure basics going in, Microsoft Azure Fundamentals Certification and Beyond (built around the January 2026 AZ-900 update) is worth having on hand beforehand, especially if you’re eyeing the certification alongside the hands-on identity work.

Check it out on Amazon

If you’re a regular reader, hit reply and tell me which content would be most useful for your team: hands-on agent patterns, security and guardrails, or practical tool reviews.

If this issue helped sharpen the conversation, please like it or leave a comment so more platform, cloud, security and infrastructure engineers can find it.

Thanks for reading The AI Network Engineer by Packt. Subscribe for free to receive future issues.

Today’s story is about an agent that told an engineer a device was fine while its own tool output said otherwise. Before we get into it, Sif Baksh, whose book this issue draws from, is running a hands-on workshop on August 25th that walks through exactly this problem: building the evidence checks that catch an agent lying to you in fluent prose.

Build an AI Assistant for Network Troubleshooting,
August 25th, 9 AM EDT

Claim 40% Off

If you’ve ever had an agent tell you everything looked fine while the raw output said otherwise, you already know why this session exists. In this four-hour hands-on workshop, you build a RAG-powered NetOps assistant that answers from your own runbooks and device configs, then build the evidence record and response checks that stop it from writing a confident summary its own tool output disagrees with. Same failure mode as today’s story, except you’re the one building the guardrail.

Four hours. Four labs. For AI Network Engineer readers, use code LIMITED40 for 40% off all passes.

P.S. My book, Building AI Agents for Network Operations, goes deeper into the same architecture. You can bundle it with your event pass at checkout, or pick it up separately if the 25th does not work for your calendar.

That's the hands-on version. Now, the story.

Your troubleshooting agent finishes its run and gives you this:

leaf2 appears to be functioning normally. It has two established peers and no major connectivity issues. 

Clear. Confident. Well-written.

Now scroll up to the tool output it was summarizing. total_peers: 2. established_peers: 1. Neighbor 10.1.2.2 sitting in Idle with zero prefixes received. And Ethernet3, described as server_rack_2, is down.

The prose is fluent. The evidence disagrees.

Two outputs. Only one is evidence

This is one of the easiest failure modes to miss in agentic troubleshooting, especially in a polished demo.

A tool-using agent produces two distinct things.

• First, the tool results: what get_bgp_summary() actually returned and what ping_device() actually reported.

• Second, the final summary: natural language written about those results.

The second is another model-generated output. It can omit, soften, or contradict facts that appeared in the tool results, even when the underlying investigation was correct.

During an incident, a confident wrong summary is worse than no summary. It sends someone down the wrong path with the reassurance of a second opinion attached.

So the rule is simple: when the summary and the tool output disagree, the evidence wins. Always.

The final model response is never more authoritative than the tool results it describes.

That means you need something in your system that can tell when they disagree.

Better prompts help. They don’t close the gap.

You can reduce the drift with prompt rules, and you should.

Instructions such as always compare established_peers against total_peers, always name any neighbor not in Established state, treat zero prefixes received as possible routing impact, and never claim there are no issues if a tool returned an error or degraded state can meaningfully improve the output.

But prompt rules are guidance, not enforcement.

They make the good answer more likely. They don’t make the bad answer impossible. If your only defence against a wrong conclusion is a bullet point in a system prompt, you don’t have a defence.

The enforcement has to live outside the model.

Build an evidence record

Sif Baksh demonstrates a useful approach in Building AI Agents for Network Operations.

Alongside the conversation history the model reasons over, the application builds a separate, structured record of what was checked and what looked unhealthy. The model doesn’t write this record. Your code assembles it from the raw tool results.

For the leaf2 case, the record holds:

• Device status: up

• Down interface: Ethernet3, described as server_rack_2

• Total BGP peers: 2

• Established BGP peers: 1

• Non-established neighbor: 10.1.2.2, Idle, with zero prefixes received

Once those facts exist as data rather than prose, your application can derive rules from them:

• If established_peers < total_peers, the response must flag a BGP issue.

• If a neighbor isn’t Established, the response must identify that neighbor.

• If a neighbor reports zero prefixes received, the response should flag possible route impact.

• If ping_device() returns unreachable, the response must not describe the target as reachable.

• If a tool returns an error, the response must surface it or request another check.

The evidence checks themselves do not need another model. Your code can derive them directly from the tool results.

To enforce them reliably, require the final response in structured fields before rendering it as prose. That gives the application something deterministic to validate instead of searching an unrestricted paragraph for the right conclusion.

Careful wording is a feature

Say a host at 10.99.99.99 is unreachable and the agent is asked whether BGP on leaf2 might be involved.

The ping tool returns 100% loss. That proves reachability failed. It does not prove why.

If the BGP summary also shows an Idle neighbor, the honest answer is that the failed session may be related to a missing route, not that BGP caused it.

Hedging in a troubleshooting summary can look like weakness. Here, it’s the opposite.

An agent that separates what it confirmed from what it inferred is far more useful during a 2 a.m. incident.

Try this

Give your agent’s final response a fixed shape with five structured parts:

• Finding: The short operational conclusion.

• Evidence: The specific tool results supporting it, recorded as facts. For example: leaf2 status is up, BGP total_peers is 2, BGP established_peers is 1, and neighbor 10.1.2.2 is Idle.

• Likely cause: A clearly labelled hypothesis, using may be or likely rather than certainty the evidence doesn’t support.

• Next checks: What the engineer should verify or approve.

• Unknowns: What the agent did not check.

That last section is the one to fight for. It keeps the agent honest.

If the route table was never inspected directly, the response should say so. If only reachability was tested, it should say that the route itself wasn’t examined.

Then run your evidence checks against those structured fields before displaying the final answer. If a check fails, don’t show the answer. Show the mismatch.

The summary is a view of the evidence. It is not the evidence itself.

When the two disagree, trust what the tools returned; not how confidently the agent described it.

Thanks,

Sayali

Editor-in-Chief

This issue draws on Building AI Agents for Network Operations by Sif Baksh. Chapter 7 follows the leaf2 investigation step by step, including the evidence record and response checks.

Get the Book

The evidence-versus-summary problem doesn't stop at troubleshooting. It shows up the moment an agent proposes a change, or the moment you're deciding who's allowed to approve one. That's the thread running through the next two sessions.

Aug 27th · Agentic AI for Infrastructure Engineering: From Chatbots to Operators

Ritesh Vajariya, founder & CEO - AI Guru, teaches you how to build an infrastructure agent that reads pod events, correlates live metrics, and proposes fixes it only executes once you approve, replacing the documentation chatbot most platform teams already outgrew. Seven production-grade failure scenarios, injected into your own local Kubernetes cluster, yours to keep and re-run after the session ends.

Book Now

If your Docker and Kubernetes fundamentals need shoring up before you get there, The Ultimate Docker Container Book (4th edition) by Dr. Gabriel Schenker is the deepest single resource we carry on containers through orchestration, and its latest edition adds AI-driven DevOps patterns on top.

Check it out on Amazon

Aug 29th · Active Directory and Entra ID in a Modern Hybrid Architecture

Professor Robert McMillen breaks down how traditional Active Directory and Entra ID work together in real hybrid environments: domains and group policy on one side, cloud identity and SSO on the other, and Entra Connect bridging them. Built for IT pros moving into infrastructure or identity-focused roles.

Book Now

Since the session is already an add-on to the Azure basics going in, Microsoft Azure Fundamentals Certification and Beyond (built around the January 2026 AZ-900 update) is worth having on hand beforehand, especially if you’re eyeing the certification alongside the hands-on identity work.

Check it out on Amazon

Before you go, I’d like to hear from regular CloudPro readers. Hit reply and tell me what would be most useful for your team. And if this issue helped sharpen the conversation, please like it and leave a comment so more platform, cloud, security, and infrastructure folks can find it.

You’ve got an LLM turning show interface output into JSON. It works across your test samples, the fields come back clean, and you wire it into the next step.

Now give it this:

GigabitEthernet0/1 is up, line protocol is down (err-disabled) 

If the model returns a neat JSON object describing a healthy interface, json.loads() will accept it without complaint.

Nothing crashed. That’s the problem.

Subscribe now

Model output must clear two gates, not one

When json.loads() succeeds, you’ve cleared the first gate: the response is syntactically valid JSON.

That tells you nothing about whether the content is correct. Valid JSON can still:

• Omit a required field

• Return an MTU as text instead of an integer

• Return a state your schema doesn’t allow

• Invent a value that never appeared in the source output

The second gate is validation, and it belongs in your code.

Sif Baksh draws this boundary clearly in Building AI Agents for Network Operations: the model helps transform the text, but the application owns the trust decision. Until your code validates it, an LLM response is only candidate data.

First ask whether you need the model

If the device returns native JSON, use it.

If a reliable TextFSM template already exists, use it. If the format is stable and under your control, a deterministic parser will usually be faster, cheaper, and easier to test.

LLM-assisted parsing earns its place in a narrower band.

The clearest example is multi-vendor normalization. Cisco IOS might return GigabitEthernet0/1, Arista Ethernet1, and Juniper ge-0/0/1.0. The text differs, but you may want each result mapped into the same shape:

{ 
  "interface": "...", 
  "admin_status": "...", 
  "oper_status": "..." 
} 

The other good fit is text that was never structured to begin with: incident notes, ticket bodies, or mixed log output containing prose.

Use the model where flexibility helps. Keep exact decisions in deterministic code.

Validate the smallest useful shape

Start by checking whether every required field exists:

expected = [ 

    "interface", 
    "admin_status", 
    "oper_status", 
    "ip_address", 
    "mtu", 
] 
missing = [ 

    field for field in expected 
    if field not in result 

] 
if missing: 

    print(f"Missing fields: {missing}") 

else: 

    print("All expected fields present!") 

This is only the first validation check. A production workflow should also confirm:

• Each field has the expected type

• Status values come from an allowed set

• Values agree with the source data where possible

• Missing or unsupported values are handled safely

A typed data model or validation library such as Pydantic makes these checks easier to enforce before anything downstream touches the result.

The key phrase is before anything downstream touches it. Logging a validation warning while still passing the object forward defeats the point.

Validated output is only one part of a safe workflow. If the next step is an infrastructure change, policy enforcement and approvals need to live in the execution layer, not in the prompt.

Turn infrastructure intent into governed execution

Stop managing IaC manually. Spacelift brings AI-native orchestration to Terraform, OpenTofu, Ansible, and more. With automatic drift detection, policy as code, approval workflows, and natural-language provisioning through Spacelift Intelligence, teams can accelerate infrastructure delivery while retaining the governance and visibility production environments require.

TRY SPACELIFT ON AWS MARKETPLACE →

Give uncertainty somewhere to live

Real device output doesn’t always fit neatly into up or down.

An interface may be err-disabled. A port may be flapping, leaving its current state unclear. A command may return % No interface information available.

If your schema only permits clean states, the model has two choices: break the schema or guess. Models are very good at choosing the second option.

Allow states such as unknown, and include fields such as error_reason and warning. Keep those fields empty unless the reason appears in the source data. A failed command should return a controlled error or an explicitly uncertain result – never a confident, invented status.

The same rule applies when the model produces malformed output. It may wrap JSON in Markdown fences or add a helpful sentence before the object. Strip the common wrappers and retry. If it still won’t parse, return a structured error and let the caller decide what happens next.

Never fail silently, and never pass a half-valid result forward.

Try this on one parser

Take one parser you already use (or the next one you’re planning), and check whether it does these five things:

1. Requests only the fields the workflow actually needs.

2. Uses a low temperature for predictable output.

3. Checks required keys, types, and allowed values.

4. Represents missing or uncertain data explicitly.

5. Stops the workflow when validation fails and logs the raw input, model response, and parsed result.

That last check pays for itself the first time something breaks during an incident. You need to know whether the device returned unexpected data, the model produced a bad response, or your code accepted something it should have rejected.

This issue draws on Building AI Agents for Network Operations by Sif Baksh, out now from Packt. Chapter 4 is the one to read if you’re building parsers. The labs run on local models via Ollama with mocked device data, so you can work through the whole thing without touching a production device.

Get the book

Agentic RAG for Network Operations - Tuesday, August 25th, 9 AM EDT

Sif Baksh, Principal Solutions Architect at Tines, himself is teaching you how to build a RAG-powered NetOps assistant that pulls answers from your own runbooks, device configs, and troubleshooting notes instead of the open internet, so it can tell you why a BGP neighbor is stuck in Active, with the actual source cited. You’ll also learn the guardrail patterns that stop the assistant from inventing answers or recommending unsafe production changes, so it’s something you can trust mid-incident.

This one’s capped on seats and going quicker than usual. Use code LIMITED40 for 40% off.

Book Now

P.S. - We have a special bundle for you where along with the event you get access to the book. Avail now

Aug 27th · Agentic AI for Infrastructure Engineering: From Chatbots to Operators

Ritesh Vajariya, founder & CEO - AI Guru, teaches you how to build an infrastructure agent that reads pod events, correlates live metrics, and proposes fixes it only executes once you approve, replacing the documentation chatbot most platform teams already outgrew. Seven production-grade failure scenarios, injected into your own local Kubernetes cluster, yours to keep and re-run after the session ends.

Book Now

If your Docker and Kubernetes fundamentals need shoring up before you get there, The Ultimate Docker Container Book (4th edition) by Dr. Gabriel Schenker is the deepest single resource we carry on containers through orchestration, and its latest edition adds AI-driven DevOps patterns on top.

Check it out on Amazon

Aug 29th · Active Directory and Entra ID in a Modern Hybrid Architecture

Professor Robert McMillen breaks down how traditional Active Directory and Entra ID work together in real hybrid environments: domains and group policy on one side, cloud identity and SSO on the other, and Entra Connect bridging them. Built for IT pros moving into infrastructure or identity-focused roles.

Book Now

Since the session is already an add-on to the Azure basics going in, Microsoft Azure Fundamentals Certification and Beyond (built around the January 2026 AZ-900 update) is worth having on hand beforehand, especially if you’re eyeing the certification alongside the hands-on identity work.

Check it out on Amazon

AI-Powered GitOps & Platform Engineering - Thursday, July 30th · 11:00 AM EDT

This event is happening next Thursday, don’t miss out!
Most AI-in-GitOps failures trace back to one thing: stale context, not a bad model. Taylor Dolezal (Head of OSS at Dosu) has seen this pattern across 100,000+ repos, and this session runs a live query against fresh vs. stale knowledge side by side, so you can see where the token cost actually goes and where the accuracy breaks down before you build anything on top of it.

Use code FINAL50 for 50% off.

Book Now

One signal is not a diagnosis

Your monitoring system reports packet loss on a production link.

An AI agent checks the alert, identifies a recent configuration change, and recommends rolling it back.

The explanation sounds convincing. But is it enough to act on?

Not yet.

A single signal can reveal a symptom. It rarely proves the cause.

Make the agent show its evidence

Before an AI agent recommends a network change, require it to find three things:

• The symptom: What does the telemetry show?

• A plausible cause: What configuration, event, or change could explain it?

• Independent confirmation: What other source supports, or contradicts, the diagnosis?

Consider the packet-loss alert:

• Interface telemetry shows a rise in errors.

• The change log shows that the interface configuration was updated shortly before the errors began.

• A path test confirms that the loss is isolated to that link.

Now the recommendation is supported by evidence, not just correlation.

If the signals do not agree, the agent should not force a conclusion. It should show what is missing and escalate the investigation.

That is the difference between an agent that produces a plausible answer and one that supports a defensible decision.

Thanks,
Sayali
Editor-in-Chief

P.S. When an AI-generated diagnosis looks convincing, what evidence does your team check before acting on it? Reply and tell us.

Here’s our star event for the month of August, Sif Baksh returns again!

Agentic RAG for Network Operations - Thursday, August 25th · 9 AM EDT

Sif Baksh, Principal Solutions Architect at Tines helps you build a RAG-powered NetOps assistant that pulls answers from your own runbooks, device configs, and troubleshooting notes instead of generic internet answers, so it can tell you why a BGP neighbor is stuck in Active, with the actual source cited. You’ll learn with guardrail patterns that stop the assistant from inventing answers or recommending unsafe production changes, so it’s something you can actually trust during an incident.

Use code SPECIAL40 for a 40% off on the event

Book your tickets

Agentic AI for Infrastructure Engineering: From Chatbots to Operators -
Thursday, August 27th - 9:30 AM EDT

Ritesh Vajariya, the AI Guru himself takes you past the documentation chatbot stage: an infrastructure agent that reads live pod events and Prometheus metrics through MCP tools, correlates them against your runbooks, and tells you why your model-serving pods are crashlooping right now, not what a stale doc says about crashloops in general. You’ll build the three-layer architecture (knowledge, live-state, gated action) from the raw LLM API and MCP SDK, work through GPU scheduling failures and rollback scenarios on a disposable local Kubernetes cluster, and walk away with the guardrail patterns (approval gates, dry-run, scoped credentials) that make the difference between an agent that answers and one you’d actually let touch production.

Use code EARLY40 for 40% off.

Book now

Before you go, maybe a few books worth having on your shelf

1. LLMs for Modern Software Delivery and DevOps - Gu Huangliang, Zheng Qingzheng, Niu Xiaoling, Che Xin
   

   Check it out on Amazon →

Taylor’s upcoming GitOps workshop on 30th July is built around one idea: knowledge freshness beats model choice. This book takes that same premise and runs it across the entire delivery lifecycle, log triage, testing, code review, incident response, so if the workshop’s context-engineering angle clicked, this is the fuller playbook behind it.

2. From Cloud Native to AI Native - Pini Reznik, Michael Müller
   

   Check it out on Amazon →

This book is about how the same cloud-native patterns you already trust (GitOps, Kubernetes, observability) extend into AI-native architecture, instead of getting ripped out and replaced.

2. The Self-Taught Cloud Computing Engineer - Dr. Logan Song

   Check it out on Amazon →

If you’re building toward a platform or SRE role and keep bouncing between AWS, Azure, and GCP docs without a throughline, this book takes the “learn one deeply, then compare“ approach instead of three separate crash courses, compute, storage, security, and ML services mapped across all three.

AI agents are moving into prod. Is your secrets strategy ready for that?

Find out at the webinar on July 22.

Register Now

Your AI agent needs a stop condition

Last week, we talked about giving AI agents a context contract: a clear definition of what information they can access, trust, and use.

But even with the right context, an agent can still make the wrong move.

Why?

Because knowing what to consider is not the same as knowing when to stop.

Imagine an agent investigating a network incident. It checks interface status, reviews recent configuration changes, and summarizes the logs.

So far, so good.

But what happens when the evidence is incomplete? Does it keep investigating? Restart a service? Roll back a configuration? Escalate to an engineer?

Without a defined stop condition, the agent has to guess.

Define the boundary before the workflow

For every agentic workflow, answer three questions:

• What can the agent do independently?

• What conditions require human approval?

• What evidence must it provide before escalating?

A stop condition might look like this:

The agent may collect telemetry, compare configurations, and recommend a remediation. It must stop before making a production change unless the diagnosis is supported by two independent signals and an engineer approves the action.

That single rule turns an open-ended agent into a controlled workflow.

A useful stop condition should cover:

• Confidence: How certain must the agent be?

• Impact: Which actions are too risky to perform autonomously?

• Evidence: What must the agent show before recommending or taking action?

• Escalation: Who takes over when the boundary is reached?

The goal is not to prevent autonomy. It is to make autonomy predictable.

Because the safest agent is not the one that can do everything. It is the one that knows when it should hand control back to you.

Before deploying your next AI workflow, run it through this five-minute test.

Download the AI Agent Stop-Condition Checklist

Thanks,

Sayali

Editor-in-Chief

P.S. What is one action your team would never allow an AI agent to take without human approval? Reply and tell us.

Take a look at our upcoming workshops that point to the same issue from different angles.

1. Agentic DevOps with Claude - Thursday, July 23rd · 11:00 AM EDT

Where do agents and MCP servers actually belong in a Kubernetes stack?

Most platform teams are still guessing. Michael Rishi Forrester (Accenture) spends four hours building a 33-component AI-native IDP live, on a provisioned cluster, with Claude Code doing the actual work: scaffolding Helm values, wiring up ArgoCD’s App-of-Apps, resolving sync issues as they come up. You clone the repo when it’s done and it runs.

Use code EMAIL40 for 40% off.

Book Now →

2. AI-Powered GitOps & Platform Engineering - Thursday, July 30th · 11:00 AM EDT

Most AI-in-GitOps failures trace back to one thing: stale context, not a bad model. Taylor Dolezal (Head of OSS at Dosu) has seen this pattern across 100,000+ repos, and this session runs a live query against fresh vs. stale knowledge side by side, so you can see where the token cost actually goes and where the accuracy breaks down before you build anything on top of it.

Use code EMAIL40 for 40% off.

Book now →

Both sessions are teaching the same shift from two different angles, Michael’s building the platform itself, Taylor’s making sure the agents on it aren’t running on stale context. Worth sitting through both if you’re the one who’ll own this stack. Bundle Pass gets you both for 25% off - use code BUNDLE25.

Grab the bundle!

Agentic RAG for Network Operations - Thursday, August 25th · 9 AM EDT

Sif Baksh, Principal Solutions Architect at Tines helps you build a RAG-powered NetOps assistant that pulls answers from your own runbooks, device configs, and troubleshooting notes instead of generic internet answers, so it can tell you why a BGP neighbor is stuck in Active, with the actual source cited. You’ll learn with guardrail patterns that stop the assistant from inventing answers or recommending unsafe production changes, so it’s something you can actually trust during an incident.

Use code SPECIAL40 for a 40% off on the event

Register now

Before you go, maybe a few books worth having on your shelf

1. Operational AI with Docker - Ajeet Singh Raina & Harsh Manvar
   

   Check it out on Amazon →

If Michael’s event - Agentic DevOps with Claude IDP build got you thinking about what it actually takes to run agentic workloads in production, not just prototype them, this is the deeper dive. It covers containerizing, serving, and scaling LLMs and multi-agent pipelines with Docker and Kubernetes, the infrastructure layer most “AI for DevOps“ content skips past.

2. AI Networking Cookbook - Eric Chou
   

   Check it out on Amazon →

2. Pairs naturally with Sif’s session on Agentic RAG for Network Operations. Where the workshop builds you a RAG assistant for troubleshooting, this book is recipes for the other side, using AI to automate config, monitoring, and threat detection across multi-vendor network environments, with working code for each one.

3. Microsoft Intune Cookbook, 2nd Edition - Andrew Taylor

   Check it out on Amazon →

For anyone managing endpoints rather than the network layer, practical recipes for securing and automating devices across Windows, macOS, iOS, Android, and Linux, including where Security Copilot and Graph API now fit into day-to-day admin work.

Check it out on Amazon →

Is your secrets strategy ready for AI agents?

Save your seat for Infisical’s webinar on July 22.

Register now

Now, back to infrastructure and AI.

Last issue, we shared a three-part test for AI workflows:

Fresh context. Clear validation. Reusable workflows.

This week, let’s stay with the first one.

Giving an agent access to current information does not automatically mean it has the right information.

Imagine an agent reviewing an infrastructure change. It can access the live configuration, an old architecture diagram, several runbooks, a recent incident ticket, and a Slack thread describing a temporary workaround.

Technically, the agent has plenty of context. Operationally, it has a trust problem.

• Which source should it prioritize?

• How recent is recent enough?

• What should happen when two sources disagree?

What the agent needs is a context contract.

A context contract defines three things before the workflow starts.

1. Which source is authoritative?

Not every document deserves equal weight. Define a clear order, for example:

1. Live system state

2. Version-controlled configuration

3. Approved runbooks

4. Incident and change records

5. Informal discussions

The agent shouldn’t have to guess what wins.

2. How fresh must the information be?

“Current” depends on the task, for example:

• A troubleshooting agent may need device state from the last few minutes.

• A capacity-planning workflow may be fine with yesterday’s data.

• A policy document may remain valid for months.

Set a freshness threshold. If the available context is older, the agent should flag it rather than quietly reasoning from stale information.

3. What happens when sources conflict?

Conflict is normal. The running configuration may not match Git. A runbook may describe a process that the latest incident proved incomplete.

The safest rule is simple:

If live state and the approved source of truth disagree, stop, show the conflict, and request human review.

Fresh context is not about connecting an agent to everything. It is about giving it the right sources, in the right order, with clear rules for handling uncertainty.

Thanks,

Sayali

Editor-in-Chief

P.S. What source does your team trust when the documentation, Git repository, and live environment disagree? Reply and tell us how you resolve it. Your approach may help shape a future issue.

Both sessions point to the same issue from different angles.

Agentic DevOps with Claude - Thursday, July 23rd · 11:00 AM EDT

Michael Rishi Forrester (Accenture) helps you build a live, 33-component AI-native Kubernetes platform in four hours, so you walk away with a working IDP you can clone and run Monday, not another slide deck. You’ll also get a reference architecture showing exactly where agents and MCP servers belong in your stack, something most teams are still guessing at.

Book Now →

AI-Powered GitOps & Platform Engineering - Thursday, July 30th · 11:00 AM EDT

Taylor Dolezal (Head of OSS at Dosu) helps you pinpoint exactly where AI agents earn their keep in GitOps workflows, drawn from patterns across 100,000+ repos Dosu supports. You’ll also walk through a live demo comparing fresh vs. stale knowledge, so you can see the token-cost and accuracy tradeoffs before you build anything.

Book now →

Given this is fast becoming a must-know skillset for every platform and DevOps engineer, grab the Bundle Pass for both sessions at 40% off instead of buying separately

Grab the Bundle!

Before you go, a book worth having on your shelf

If this issue made you realize your AWS/Azure/GCP fundamentals could use a refresh, The Self-Taught Cloud Computing Engineer by Dr. Logan Song is a solid pick. It’s a structured, hands-on guide covering compute, storage, networking, data, security, and AI across all three major clouds, rated 4.8★ from over 350+ readers.

Check it out on Amazon →

Social engineering is about manipulating people’s emotions. Identify the susceptibilities that hackers use to exploit people.

This NINJIO Insights Report dives into the key emotional susceptibilities that make social engineering work and offers concrete steps that your security team can take to equip your workforce to resist cyberattacks.

Download the Guide

Now, back to infrastructure and AI.

This week, we have two hands-on sessions for teams trying to move AI from side experiments into real platform and DevOps workflows.

The common thread: AI is only useful when the system around it is strong enough to keep it grounded, reviewable, and reusable.

Agentic DevOps with Claude | July 23rd

Early Bird Live Now - 40% Off | Last 48 hours before it’s sold out

Claude Code is not just generating snippets here. It is helping build the platform.

In this 4-hour live workshop, you’ll see a 33-component AI-native internal developer platform built on a real Kubernetes cluster, with ArgoCD, Backstage, kgateway, and an observability stack included.

The cluster is provisioned for you. You leave with the repo and a working reference architecture your team can study, adapt, and build on.

Michael Rishi Forrester from Accenture, previously at KodeKloud, is running this one.

Limited seats available.

📅 Thursday, July 23 | 11:00 AM EDT

Lock in Early Bird

AI-Powered GitOps and Platform Engineering Workshop | July 30th

Early Bird Live Now - 40% Off | Last 4 days before it’s gone

Your AI agent does not know your manifests are stale.

That is the problem.

In this 3-hour hands-on workshop, you’ll work through real ArgoCD and Flux workflows, see live demos comparing fresh versus stale context, and learn how to turn repeated agent tasks into tooling your team actually keeps.

The session covers drift detection, change review, validation, and the parts of platform engineering AI tools usually get wrong because nobody gave them current operational context.

Taylor Dolezal from Dosu, an AI-native knowledge infrastructure company for agents and humans, is running this one, drawing on patterns from 100,000+ repos.

Limited seats available.

📅 Thursday, July 30 | 11:00 AM EDT

Book Your Tickets

Both sessions point to the same issue from different angles.

AI is not hard to demo. It is hard to operationalize.

The moment agents get involved in real DevOps, GitOps, or platform workflows, three questions start to matter more than the prompt itself: does the agent have fresh context, is validation built in, and can the repeated task become reusable team-owned tooling?

So here’s a quick test before you give an agent more responsibility.

Start with these three questions.

1. Is the context fresh?

Most AI failures in operations do not start with a bad model, they start with stale context.

The agent answers based on yesterday’s architecture, last month’s runbook, or a ticket pattern that no longer applies.

For platform teams, context freshness matters because the environment changes constantly: manifests, service owners, policies, dependencies, deployment history, incidents, and rollback procedures.

If the agent cannot retrieve current context, it should not be trusted with current decisions.

2. Is validation built into the workflow?

AI output should not move straight from suggestion to execution, there should be a validation step.

In GitOps environments, that might mean checking manifests, reviewing diffs, validating against policy, detecting drift, or confirming that a proposed change matches the expected deployment state.

The workflow should make review easier, not optional.

The useful question is not “did the agent produce something?”

It is “can the team verify it quickly and confidently?”

3. Can the repeated task become team-owned tooling?

If an engineer keeps asking an agent the same thing every week, that is a signal.

Maybe the task should become a reusable workflow, template, validation check, or internal tool.

The goal is not to collect clever prompts.

The goal is to turn useful AI-assisted work into something the team can maintain, improve, and trust.

That is where AI starts becoming part of platform engineering instead of sitting beside it as an experiment.

The pattern is simple:

• Fresh context

• Clear validation

• Reusable workflows

That is the difference between AI that demos well and AI that survives contact with real infrastructure work.

If your team is working through these questions right now, the two sessions above are worth a look. One shows what it looks like to build an AI-native platform live. The other goes deeper into GitOps, context engineering, validation, and reusable agent workflows.

Book your seat while Early Bird pricing is still live.

Thanks,

Sayali

Editor-in-Chief

P.S. If someone on your team is experimenting with Claude Code, GitOps, platform engineering, or AI-assisted infrastructure workflows, share this issue with them. And if you’re already testing agents in production-adjacent work, reply and tell us where they are helping, or where they are still getting stuck.

Start small. Keep the task scoped. Make the output reviewable. Don’t let an agent near production just because the demo worked once.

This week, let’s make that more specific.

Before you turn a script, playbook, or check into an MCP skill, ask one question:

Should an agent be allowed to call this?

That sounds obvious, but it is where a lot of messy workflows begin.

• A script can be useful and still be a bad MCP skill.

• A playbook can be reliable and still need a human to decide when it runs.

• A troubleshooting step can be routine and still carry too much risk if the agent gets the context wrong.

So here’s a quick test.

1. Is the task repeatable?

Good MCP skills usually start with work your team already does often: interface checks, config comparisons, log summaries, pre-change validation, post-maintenance checks.

If the task happens once a quarter and needs senior judgment every time, it probably should not be your first skill.

2. Are the inputs and outputs clear?

“Check interface health for these devices and summarize errors from the last 24 hours” is clear.

“Find out why the network is slow” is not.

The agent needs a defined job, not a vague investigation.

3. Can it fail safely?

What happens if the device does not respond?

What happens if the data is incomplete?

What happens if the tool call fails?

A good MCP skill does not only define the happy path. It defines when the agent should stop.

That is the difference between exposing a tool and engineering a workflow.

And that is exactly what Engineering Agentic Network Operations is about.

Engineering Agentic Network Operations

June 30th | 9 AM EDT | FLASH SALE - Use SAVE40 for 40% Off

The prior networking event we had, Build Intelligent Networks with AI brought together 150+ attendees from 30+ countries and became AutoCon4’s highest-rated workshop.

Now we’re taking this to the next step, building agentic networking operations at scale.

The focus is not just building an MCP server. It is building MCP skills that are reusable, composable, and practical enough to make sense in real network environments.

In this workshop, William Collins and John Capobianco of Itential will walk through how to build MCP skills that can be reused across your environment instead of becoming one-off scripts only one person can debug.

They’ll also cover spec-driven development, so agents have defined behavior before they start acting, and agentic loops that recover when they get stuck instead of failing silently.

The live lab includes OpenClaw, NetClaw, Selector AI, Containerlab, FastMCP, Python, Claude, and Arista cEOS.

If you’ve been following this newsletter and thinking, “Okay, I understand the theory, but what does this look like in a lab?” this is the session to attend.

BOOK YOUR SEAT NOW

Use code SAVE40 for 40% off while the flash sale is live upto 48 hours.

The useful question is no longer whether an agent can call a tool.

It can.

The better question is whether the workflow has enough structure to be trusted.

That is what this workshop is built around: smaller skills, clearer behavior, reusable workflows, and network automation that engineers can actually reason about.

Hope to see you there.

Thanks,

Sayali

Editor-in-Chief

P.S. If this framework was useful, share this issue with someone on your network or automation team. And if you’re working on MCP, agents, or AI-led network workflows, reply and tell us what you’re trying to build next.

Additionally, a word from a sponsor if relevant to you

Make Claude Your 2nd Brain That works 24/7 By Mastering In It 16 hours- Here’s How

A bit of history before we get into today’s issue.

When we conducted the Build Intelligent Networks with AI in January, we had 150+ attendees from 30+ countries and came out as AutoCon4’s highest-rated workshop. That response is the reason The AI Network Engineer exists. There was clearly an appetite for this conversation, not just for four hours in a workshop, but every week. So we started writing.

We understood that engineers wanted to see how these ideas actually work against real network environments. How MCP skills are built. How agents should behave when they have access to tools. How to keep automation reusable instead of creating another pile of scripts only one person understands.

Six months later, the question has changed. It’s no longer “can this work”, it’s “how do I make this work reliably, without it falling over the first time something unexpected happens.”

In the upcoming session, we’re moving from raw MCP tools to composable, reusable skills, using spec-driven development to actually constrain agent behavior, and designing agentic loops that detect when they’re stuck and recover safely, all hands-on with OpenClaw and NetClaw as production frameworks,

ENGINEERING AGENTIC NETWORK OPERATIONS
June 30th | 9 AM EDT 50% off with code SPECIAL50 (ENDS SOON)

John Capobianco and William Collins are running this one live. If their names sound familiar, it’s because they’re the same two voices behind most of what we’ve covered: John’s trust ramp from Issue #1, the RAG-is-your-vars-file and MCP-is-your-Galaxy-role explainers from Issue #4, William’s “spider-web it out” approach to agent rules from Issue #3.

Cohort 2 picks up where those conversations leave off. The honest version of where most teams get stuck isn’t building an MCP server, it’s running one in production without it becoming another thing to babysit. That’s what this session is built around:

MCP skills that are reusable across your environment, instead of one-off integrations you write once and then maintain forever. Spec-driven development, which is really the production version of the “markdown files that define what the agent can and can’t do” idea from Issue #3, and the five-line job description from Issue #6, role, trigger, tools, job, escalation, except now you’re writing the spec that an agentic loop actually follows. And agentic loops designed to recover when they get stuck, rather than failing in a way nobody can diagnose.

All of it hands-on, in a live lab, with OpenClaw, NetClaw, Selector AI, Containerlab, FastMCP, and Arista cEOS. Same digital-twin philosophy John described back in Issue #1: let the agents break things here, where it doesn’t cost you anything, before they’re anywhere near production.

If Issue #4 made MCP click for you on paper, this is where you build the thing.

Use code SPECIAL50 to get 50% off

BOOK YOUR SEAT NOW

ALSO WORTH A LOOK THIS MONTH

Linux, SysAdmin, and AI Workflows with Imran Afzal
June 27 | 11:30 AM EDT | 40% off with code EARLY40

A quick second one, because it lines up with the “collection job” idea from Issue #5. Imran Afzal, one of Udemy’s best-selling Linux instructors, is running a hands-on session on ChatGPT CLI, Shell Genie, Aider, and Goose CLI, generating commands and scripts from plain English, explaining logs, and reviewing what these tools produce before you run it. If Cohort 2 is the network and agent layer, this is the Linux layer everything underneath actually runs on.

Use code EARLY40 to get 40% off

RESERVE YOUR SEAT NOW

The Network Automation Accelerator is a new 12-week DevNet Academy program that helps traditional network engineers build practical automation skills through hands-on labs, mentoring, code reviews, and a final real-world automation project. Register your interest to learn more.

Register here

The reason we keep coming back to these sessions is simple.

AI in infrastructure is no longer just a curiosity. But it also cannot be treated like a magic layer you place on top of production and hope for the best.

The teams that get value from it will be the ones that know how to scope the work, define the boundaries, test in a lab, review the output, and keep humans in control where it matters.

That was the lesson from our first Networking event, with the second one we’re just getting started

Hope to see you there.

– Packt Conferences

P.S. Engineering Agentic Network Operations is happening on June 30 at 9 AM EDT. Use code SPECIAL50 while the flash sale is live. Goes out in <48 Hours

In this issue, I want to point you toward three sessions worth your time if you work near platforms, networks, Linux, automation, or incident response. They are connected by one theme: AI in operations is only useful when the boundaries are engineered first.

That has been the thread running through recent CloudPro issues, even when the topics looked unrelated. In the last few issues, we’ve kept coming back to the same engineering instinct: slow down, separate the moving parts, and understand what should happen before you let anything change. Whether we were talking about broken containers, MCP design, or admin rights, the point was never just the tool. It was the discipline around the tool.

AI does not remove that habit. It raises the cost of skipping it. The teams that get value from agents will not be the ones that connect the most tools. They will be the ones that know what the agent can see, what it can change, when it must stop, and how a human verifies the next step.

Taken together, these sessions give you a practical way to think about AI where it actually shows up in your work: inside the platform, inside network operations, and at the Linux command line.

FINAL CALL - Building AI-Native Platform Engineering Systems

Last 24 hours!

Start at the platform layer.

If you run an internal platform, you’ve probably felt the pressure: AI in the developer portal, agents that explain failures, golden paths that do more than point to templates.

The challenge is that platform agents need context, telemetry, policy, safe actions, approvals, and clear trust boundaries. AI can’t just be bolted on.

That’s why Building AI-Native Platform Engineering Systems stands out. It covers Backstage, OpenChoreo, CI/CD, golden paths, control planes, policy-as-code, observability, guardrails, and platform intelligence - with AI treated as part of the architecture, not an afterthought.

A solid read for platform engineers, DevOps engineers, SREs, architects, and engineering leaders asking:

“Can we make our platform AI-native?”

The answer may be yes, but only with the right control surfaces.

Use code FINAL50 to get 50% off

BOOK YOUR SEAT NOW

The session is on June 11, i.e., 24 hours from now, and you can use the discount code FINAL50 when booking.

Engineering Agentic Network Operations

Once platform boundaries are in place, the next challenge is network operations, where many AI demos fall apart.

It’s easy to build an agent that summarizes data or suggests a config. It’s much harder to build one that works within real constraints, recovers from failures, and knows when to hand control back to a human.

That’s the focus of Engineering Agentic Network Operations.

The workshop covers MCP, OpenClaw, NetClaw, FastMCP, Containerlab, Arista cEOS, and more. But the real value is learning how to design agentic workflows with clear scope, guardrails, recovery paths, and human oversight.

For network automation, SRE, platform, and infrastructure teams, those questions matter long before an agent is trusted with production operations.

Use code SPECIAL50 to get 50% off

BOOK YOUR SEAT NOW

Agentic Linux – From Commands to AI Agents

Then there’s the base layer.

No matter how advanced the platform, a lot of real work still ends up in Linux - reading logs, checking services, fixing permissions, troubleshooting errors, and deciding whether a generated command is actually safe to run.

That’s why Agentic Linux – From Commands to AI Agents is a useful third piece.

The workshop covers ChatGPT CLI, Shell Genie, Aider, Goose CLI, Bash scripting, troubleshooting, automation, and system administration tasks. What I like is that it treats AI as an assistant, not a replacement for judgment.

Generating a command is easy. Knowing whether it’s safe, spotting destructive actions before they run, and keeping humans in the approval loop is the hard part.

For Linux administrators, DevOps engineers, SREs, and IT professionals, it’s a practical look at using AI at the command line without turning the shell into a roulette wheel.

Use code EARLY40 to get 40% off

BOOK YOUR SEAT NOW

A few books to keep beside these sessions

One more thing worth mentioning: these sessions pair well with a few books from our list if you want to go deeper after the labs.

For the first workshop, I’d start with Platform Engineering for Architects, which is especially relevant because attendees of Building AI-Native Platform Engineering Systems get the book for free. Mastering Enterprise Platform Engineering is the natural follow-up if you’re thinking about platform engineering, delivery workflows, and generative AI at enterprise scale. For the network operations side, AI Networking Cookbook fits neatly with the agentic network operations session. And if the Linux workshop is closer to your day-to-day work,The Ultimate AI Guide for Linux Engineers is a good companions for thinking about what AI-generated commands are actually touching underneath.

The Ultimate AI Guide for Linux Engineers

BUY NOW ON AMAZON

Mastering Enterprise Platform Engineering

BUY NOW ON AMAZON

AI Networking Cookbook

BUY NOW ON AMAZON

Platform Engineering for Architects

BUY NOW ON AMAZON

The useful question is not whether AI can be added to these workflows. It can, and most teams are already experimenting with it somewhere. The harder question is whether the workflow still makes sense once AI is inside it. Can the team understand what happened? Can they review the output? Can they stop a bad change before it lands? Can they trust the system more after adding AI, not less?

That is the thread running through all three sessions. They are not about chasing the newest tool. They are about taking the work many of us already do across platforms, networks, and Linux environments, and asking what has to change when AI becomes part of that work.

That is what makes these sessions useful. They are not abstract AI talks. They sit close to the places many of us actually work: the platform layer, network operations, and the Linux command line.

I’ll be in attendance as well, because these are exactly the conversations CloudPro needs to keep having: what is useful, what is unsafe, what is ready, and what still needs a human in the loop.

This is time-sensitive. The first session is on June 11, and the other two follow later this month. If one of these areas is on your roadmap, I would not skip it.

Cheers,

Apramit