Building AI agents? Learn how Apple, NVIDIA, Coinbase, Microsoft AI and AWS tackle real-time data, analytics performance, governance and cost. Join NEXT QUERY 2026, a free online conference on Nov 4, with 20+ production talks. Register free—recordings included.

Register Free

Hi, welcome to DataPro #193! 👋

Organizations are investing heavily in Microsoft Fabric, Power BI, and modern data platforms. Yet the same problems persist: dashboards nobody trusts, conflicting business metrics, employees returning to Excel, and data quality issues that go unnoticed until they disrupt operations.

So, where is the real problem? Is it the technology, poor governance, or the disconnect between data teams and business needs?

This week, we’re exploring a real-world case where a missing data field nearly disrupted an entire manufacturing operation—and how a structured data quality initiative helped improve overall data quality from 73% to 93%.

In our expert-led edition, Robert Hawker, Microsoft Fabric and Power BI Consultant and Trainer at MELORO LIMITED, joins a conversation with Apeksha Shetty, Portfolio Manager at Packt, to unpack what went wrong, why Power BI adoption often falls short, and how to build trusted, governed analytics without slowing innovation.

From fixing hidden data quality failures to designing scalable Microsoft Fabric architectures, this case study offers practical lessons for anyone responsible for turning enterprise data into reliable business decisions.

Let’s dive in! 🚀

Cheers,

Merlyn Shelley,

Growth Lead, Packt.

Build AI Agents That Recover, Not Just Respond

👉 Reserve Your Spot — Join the Workshop

A Missing Email Field Nearly Shut Down a Factory. The Real Problem Was Data Governance.

How Robert Hawker helped uncover a critical enterprise data failure, improve data quality from 73% to 93%, and rethink what it takes to build trusted analytics with Microsoft Fabric and Power BI.

The Data Problem Nobody Saw Coming

A manufacturing company was on the verge of losing essential supplier services. Payments were delayed, supplier complaints were mounting, and the finance team was struggling to keep up. Senior leadership believed the organization needed more people to manage the growing backlog.

But the real problem wasn’t staffing, financial systems, or even the payment process itself.

It was a missing email address.

Following a hurried data migration, more than 90% of supplier records were missing the email addresses required to send remittance advices. The system was generating payment notifications, but suppliers weren’t receiving them. Without those notifications, suppliers struggled to reconcile incoming payments against outstanding invoices.

Queries multiplied. The payments team became overwhelmed. Payment backlogs grew. Eventually, suppliers threatened to withdraw services, including the electricity supply to a major manufacturing plant in the UK.

A seemingly minor data quality issue had escalated into a serious operational risk.

The most troubling part? The organization had spent considerable time treating the symptoms without recognizing the underlying cause.

This is the kind of problem that Robert Hawker, Microsoft Fabric and Power BI Consultant and Trainer at MELORO LIMITED, has encountered while helping organizations navigate enterprise data governance and analytics challenges.

In a conversation with Apeksha Shetty, Portfolio Manager at Packt, Hawker explores why organizations continue to struggle with unreliable reporting, inconsistent business metrics, and poor analytics adoption despite investing heavily in modern platforms.

Drawing on real-world experiences, including a data quality initiative that improved average overall data quality from 73% to 93%, he examines how organizations can move beyond reactive data fixes and establish sustainable governance practices.

And his perspective challenges a familiar assumption: the biggest obstacle to better analytics isn’t always the technology. It’s the business problems hidden beneath the data.

Build and Stress-Test a multi-asset portfolio with Python

📅 Saturday, November 28 | 9:30 AM–1:30 PM EST

Join Cordell Tanny, CFA, FRM, FDP, to build and stress-test a 12-asset investment portfolio in Python, using portfolio optimization, CVaR, Monte Carlo simulation, and downside risk analysis to evaluate diversification, manage market volatility, and generate AI-assisted investment risk reports.

Join Cordell Tanny Live

🎬 See What’s Waiting for You in This Hands-On Workshop!

Get a sneak peek at the practical skills, real-world challenges, and expert-led learning experience.

1. Why Data Quality Problems Often Disguise Themselves as Business Problems

The supplier payment crisis reveals a recurring weakness in enterprise data management.

Organizations often recognize data quality issues only after they begin affecting business operations.

An incorrect customer address becomes a delivery failure. Incomplete product information creates inventory discrepancies. Inconsistent financial definitions lead to conflicting management reports.

By the time these problems reach leadership, the conversation is rarely about data quality.

It’s about operational efficiency, customer satisfaction, compliance, or rising costs.

Hawker’s experience demonstrates why identifying the underlying cause matters more than immediately addressing the visible symptoms.

At the manufacturing organization, the initial response focused on the payments team’s inability to manage supplier queries.

But adding resources would not have prevented the same queries from recurring.

The missing remittance advice information had created a process failure that continued generating additional work.

From Operational Crisis to Measurable Data Improvement

Once the root cause became clear, data quality became a board-level priority.

Hawker helped establish a structured program focused on identifying, measuring, correcting, and continuously monitoring data quality.

Working with business process owners and data owners, the team developed rules reflecting the organization’s actual operational requirements.

They also built a tool capable of measuring quality across different data objects, quality dimensions, companies, and countries.

This provided visibility into where problems existed and whether corrective actions were producing sustained improvements.

The results were significant.

Average overall data quality improved from 73% to 93% during Hawker’s tenure, contributing to better operational and analytical outcomes.

However, the improvement wasn’t simply the result of introducing another technology tool.

It came from changing how the organization understood and managed its data.

The Practical Challenge: Business Owners Don’t Have Time to Define Every Rule

One of the biggest obstacles was obtaining input from employees who understood the business processes.

The organization was still establishing its systems and workflows, leaving business users with limited time to participate in detailed governance exercises.

Rather than waiting for stakeholders to document every requirement, Hawker’s team took a different approach.

They extracted data from the ERP system, profiled it, and identified potential business rules.

These proposed rules were then presented to process owners for validation and refinement.

This reduced the effort required from stakeholders while allowing the team to establish an initial set of quality rules quickly.

The approach illustrates an important principle for data engineering teams: business experts don’t always need to create requirements from scratch.

Sometimes, the most effective approach is to use the data itself to identify potential problems and then work with stakeholders to determine which ones matter.

The objective isn’t to make every field perfect. It’s to ensure the data is sufficiently reliable for the business processes that depend on it.

LLMs for Quant Research: Build Text Signals & Research Agents You Can Trust

📅 Sunday, December 13 | 9:30 AM–1:00 PM EST

Build LLM-powered financial research agents with Bruno Gonçalves to extract trading signals from earnings calls and SEC filings, verify insights with RAG, and evaluate strategies using reliable, leakage-free backtesting.

👉 Register Now & Save 30%

2. The Power BI Adoption Paradox: Better Dashboards, Same Old Spreadsheets

The same disconnect between technology and business requirements appears in analytics adoption.

Organizations invest in Power BI, establish reporting environments, and publish dashboards across departments.

Yet employees continue using Excel.

Leadership questions why the new reporting platform isn’t delivering the expected value.

Is the technology inadequate? Are employees resistant to change? Does the organization need additional training?

Hawker suggests that the problem often begins much earlier, during report development.

In his experience, one of the most common reasons for poor Power BI adoption is that report developers aren’t sufficiently connected to the business users they’re supporting.

Developers understand the datasets, relationships, and semantic models.

Business users understand the decisions they need to make.

But the two groups don’t always collaborate closely enough to translate those requirements into useful reporting.

When a Technically Correct Report Still Fails

Consider a sales manager investigating declining performance in a particular product category.

The analytics team delivers a dashboard showing revenue trends, regional sales, and monthly growth.

The calculations are accurate, and the visualizations are clear.

But the manager needs to understand whether the decline is caused by inventory shortages, pricing changes, or weakening customer demand.

The report contains information, but it doesn’t answer the question driving the investigation.

This illustrative example captures the problem Hawker identifies.

A technically correct dashboard isn’t necessarily a useful dashboard.

The solution begins with understanding how employees make decisions and what information those decisions require.

Hawker recommends starting with conversations involving the people who should benefit most from the reporting.

Their experiences often reveal the underlying issue.

Slow or outdated reports may indicate inefficient semantic models, failed refreshes, or poorly managed Fabric capacity.

Incorrect numbers may point to data quality problems or insufficient validation.

Multiple reports covering the same subject may indicate weak analytical governance.

And employees who prefer Excel may simply be relying on a workflow that already meets their needs.

Why Forcing Power BI Adoption Rarely Works

Organizations sometimes treat spreadsheet usage as evidence of resistance to modernization.

But employees need a meaningful reason to abandon a familiar and reliable process.

A Power BI report should offer something better: reduced manual work, stronger security, more accessible information, or improved analytical capabilities.

Hawker also cautions against restricting familiar capabilities too quickly.

Preventing users from exporting Power BI data to Excel may eventually support security and governance objectives.

But introducing such restrictions before employees trust the new reporting environment can create frustration and resistance.

The lesson is straightforward.

Analytics adoption isn’t achieved by forcing employees to use a platform. It’s achieved by making that platform more useful than the alternatives.

3. When Five Reports Produce Five Different Answers

Poor adoption is only one symptom of a broader analytical governance problem.

Another emerges when different departments develop independent reporting products for the same business subject.

A finance team creates a revenue dashboard.

Sales builds another.

Operations develops its own version.

Each report uses slightly different calculations, filters, or reporting periods.

When leadership requests a single revenue figure, the organization produces conflicting answers.

The issue isn’t necessarily that any individual report is technically broken.

It’s that the organization has allowed multiple definitions of the same business metric to develop independently.

Hawker sees this as one of the strongest arguments for enterprise semantic modeling.

Building a Shared Foundation for Business Metrics

Rather than allowing every department to recreate important calculations, Hawker recommends maintaining a relatively small number of centrally managed semantic models.

These models should be organized around meaningful business domains and contain reusable metrics, relationships, and logic.

For a streaming organization, for example, a model covering subscribers and engagement may be more useful than a collection of generic departmental models.

Reports built on top of the shared model can answer different business questions while relying on consistent underlying definitions.

This approach reduces duplication and improves confidence in reporting.

But it also raises an architectural question.

How much control should remain centralized, and how much freedom should business users have?

Hawker’s answer is to separate the responsibilities.

Centralize enterprise data engineering and semantic modeling. Decentralize report development where business users have the knowledge and skills to create useful analytical products.

See you next time!

👉 Reserve Your Spot — Join the Workshop

👋 Hi there,

Welcome to DataPro #192!

Building an AI agent is getting easier. But making it reliable when thousands of users, unpredictable tool calls, security requirements, and real-world failures enter the picture? That’s an entirely different engineering challenge.

This week’s big question isn’t how to build smarter agents. It’s how to make them production-ready.

In this week’s expert-led edition, Evandro Franco, Senior GenAI Data Scientist at AWS, joins a conversation with Nilesh Kowadkar from Packt to unpack what enterprises are discovering as they move beyond GenAI experimentation.

From scaling bottlenecks and the complexity of MCP integrations to observability blind spots, granular security controls, and agent evaluations, we explore the engineering decisions that separate impressive prototypes from dependable enterprise systems.

You’ll also find a practical production-readiness playbook covering six critical areas every AI engineering team should think through before deployment.

☕ Grab your coffee, settle in, and discover what it really takes to keep AI agents working beyond the demo.

Let’s dive in! ↓

Cheers,

Merlyn Shelley,

Growth Lead, Packt.

Deterministic AI on CPUs: Build low-latency LLM workflows

📅 Friday, October 30 | 9:30 AM–1:30 PM EDT

Join Ben Auffarth to build deterministic, low-latency AI workflows on CPUs using ModernBERT, LFM2.5, conformal prediction, and LangGraph, with reliable routing, uncertainty checks, and selective LLM escalation for production-ready AI systems.

Join Ben Auffarth Live

Production AI Agent Engineering: Solving Latency, Reliability, and Observability Challenges

AWS GenAI expert Evandro Franco explores the architectural decisions and engineering practices required to take agentic applications from prototypes to reliable enterprise systems.

Imagine spending weeks building an AI agent that works exactly as intended.

It understands natural-language requests, retrieves relevant information, calls external APIs, and completes complex tasks with minimal human intervention. Your development team is impressed. Stakeholders see the potential. The prototype gets approved.

Then someone asks a deceptively simple question:

“Can we deploy this for 10,000 users?”

Suddenly, the conversation changes.

What happens when hundreds of requests arrive simultaneously? What if an external API fails halfway through a workflow? How do you determine why an agent made a particular decision? Who pays when one department generates ten times more model calls than another? And how do you know whether the agent is still performing correctly after deployment?

These are not theoretical concerns. They are the engineering challenges emerging as enterprises move beyond GenAI experimentation and begin deploying autonomous applications into production.

In a conversation with Nilesh Kowadkar from Packt, Evandro Franco, Senior GenAI Data Scientist at Amazon Web Services (AWS), shared what he has observed while working with enterprise customers and focusing on Amazon Bedrock AgentCore.

His central observation reveals an important shift in the AI industry: organizations are increasingly moving from experimenting with agents to confronting the realities of operating them.

And the hardest part is no longer simply getting an agent to work.

It is making sure that the agent continues to work reliably, securely, and efficiently when the real world becomes unpredictable.

The AI Agent That Worked Perfectly — Until Everyone Started Using It

Consider an enterprise building an AI-powered customer support agent.

During development, the agent performs three straightforward operations:

1. Understands a customer request.

2. Retrieves relevant information from internal systems.

3. Generates a response or invokes a tool to complete the task.

With a handful of test users, everything looks promising.

But production introduces conditions that a controlled demonstration rarely captures.

A request might require several model invocations. Some users ask simple questions, while others trigger complex reasoning workflows. External tools introduce additional network latency. A downstream service may become temporarily unavailable. Traffic can increase dramatically during peak hours.

Now imagine the same agent handling thousands of concurrent requests.

Even if the underlying language model performs well, the application surrounding it may struggle.

Evandro highlighted scaling, resilience, and latency as three recurring challenges among enterprises moving their GenAI systems into production.

The distinction matters because model intelligence and application reliability are two different engineering problems.

A powerful model does not automatically provide predictable response times, graceful failure recovery, or efficient resource utilization.

LLMs for Quant Research: Build Text Signals & Research Agents You Can Trust

📅 Sunday, December 13 | 9:30 AM–1:00 PM EST

Build LLM-powered financial research agents with Bruno Gonçalves to extract trading signals from earnings calls and SEC filings, verify insights with RAG, and evaluate strategies using reliable, leakage-free backtesting.

👉 Register Now & Save 30%

Why latency becomes an architectural problem

Traditional applications often follow relatively predictable execution paths.

An agentic application may behave differently.

A single request can trigger multiple reasoning steps, tool calls, retrieval operations, and additional model invocations before producing a final answer.

For example:

User request → Agent reasoning → Knowledge retrieval → API call → Additional reasoning → Response

Every step introduces potential latency.

If one external API takes several seconds to respond, the entire workflow can slow down. If the agent repeatedly invokes tools because it cannot resolve an ambiguous result, response times become even less predictable.

The problem is not necessarily that the model is too slow.

It may be that the application is doing too much work.

Evandro pointed to smaller models and code optimization as potential ways to address latency challenges.

This suggests a broader engineering principle: not every operation in an agentic workflow needs the same level of model intelligence.

For example, a simple classification task might be handled by a smaller model, while a more complex reasoning task could be escalated to a more capable one.

Other practical improvements include reducing unnecessary tool calls, introducing appropriate timeouts, caching reusable results, and separating synchronous tasks from operations that can run asynchronously.

These are architectural strategies rather than specific implementations described in the interview, but they illustrate the trade-offs Evandro identified.

The engineering question is no longer simply, “Which model gives the best answer?” It is, “Which combination of models, tools, and execution strategies produces an acceptable answer within our latency and cost constraints?”

👉 Register Now & Save 10%

When Your AI Agent Becomes a Frankenstein of APIs, Tools, and MCP Servers

One of the most revealing moments in the conversation came when Evandro described the growing complexity of enterprise agents.

As agents accumulate APIs, tools, and Model Context Protocol (MCP) integrations, he compared them to a Frankenstein — a collection of interconnected components that becomes increasingly difficult to troubleshoot.

It is a useful description of a problem many AI engineers will recognize.

A modern enterprise agent might interact with a CRM, a database, a document repository, a search service, an internal application, and multiple external APIs.

Each integration adds functionality.

But every integration also adds another potential failure point.

Imagine an agent tasked with generating a quarterly financial report.

It retrieves sales figures from a database, obtains customer information from a CRM, queries an analytics service, and uses an LLM to summarize the results.

The final report contains incorrect revenue figures.

Where did the failure occur?

Was the database query wrong? Did the CRM return outdated information? Did the model misinterpret the retrieved data? Was a tool response truncated? Or did an earlier reasoning step introduce an incorrect assumption?

Without adequate observability, engineers may see only the incorrect final answer.

They have no reliable way to reconstruct how the agent arrived there.

Harness Engineering: Build an AI Data Analyst You Can Actually Trust

📅 Saturday, October 31 | 9:30 AM–1:30 PM EDT

Join Bruno Gonçalves to build a production-ready AI data analyst in Python using MCP tools, context management, guardrails, output verification, and fault-tolerant execution to create reliable, secure, and trustworthy AI agents.

Join Bruno Gonçalves Live

Observability needs to follow the entire agent workflow

Traditional application monitoring typically focuses on metrics such as CPU utilization, request latency, error rates, and service availability.

Those metrics remain important for AI agents, but they do not fully explain agent behavior.

Engineers also need visibility into the decisions and interactions that occur within an execution.

A practical observability approach can capture:

• Execution traces: The sequence of model invocations, tool calls, and workflow transitions.

• Tool-level latency: Which external services contribute most to response delays.

• Failure information: Where timeouts, invalid responses, or unsuccessful tool executions occur.

• Model usage: How many model invocations and tokens are consumed by a task.

• Outcome signals: Whether the agent successfully completed the intended operation.

Consider a trace showing that an agent called the same search tool five times before returning an answer.

That behavior may indicate poor retrieval quality, ambiguous tool descriptions, or an ineffective planning strategy.

Without tracing, the team may assume that the language model itself is responsible for the delay.

With tracing, engineers can investigate the actual bottleneck.

This is why observability becomes more than a debugging convenience.

It is the mechanism that makes agent behavior understandable enough to improve, govern, and trust.

The Hidden Enterprise Challenge: Who Can Access What, and Who Pays?

Not every obstacle to production AI involves model performance.

Some of the most difficult challenges emerge from enterprise governance requirements.

Evandro described how large organizations often need granular controls beyond what their underlying cloud infrastructure provides by default.

For example, an organization may need to allocate AI-related costs across different business units, restrict access to particular resources, and apply internal policies within a shared environment.

These requirements become especially complicated when multiple teams use the same agent platform.

Consider a financial services company deploying an internal AI assistant.

The finance department needs access to financial reporting systems. The HR department needs access to employee-related information. The engineering team requires access to technical documentation.

All three departments may use the same underlying AI infrastructure.

But they should not have identical permissions.

An agent authorized to retrieve engineering documentation should not automatically gain access to confidential employee records simply because both systems are reachable through the same platform.

Similarly, the organization needs to understand which department is responsible for the infrastructure and model usage costs generated by each workflow.

Build and Stress-Test a multi-asset portfolio with Python

📅 Saturday, November 28 | 9:30 AM–1:30 PM EST

Join Cordell Tanny, CFA, FRM, FDP, to build and stress-test a 12-asset investment portfolio in Python, using portfolio optimization, CVaR, Monte Carlo simulation, and downside risk analysis to evaluate diversification, manage market volatility, and generate AI-assisted investment risk reports.

Join Cordell Tanny Live

🎬 See What’s Waiting for You in This Hands-On Workshop!

Get a sneak peek at the practical skills, real-world challenges, and expert-led learning experience.

Why cloud-level permissions may not be enough

Cloud infrastructure can provide account-level and resource-level security controls.

However, enterprises may also require application-specific authorization.

For example, two users within the same application may have different permissions to invoke a particular tool or retrieve a specific category of information.

This creates a need for additional governance layers that understand user identity, organizational context, tool permissions, and data-access policies.

A production architecture may therefore need to answer questions such as:

• Can this user invoke this particular agent capability?

• Is the agent authorized to access the requested dataset?

• Which tools can the agent execute on the user’s behalf?

• How should usage costs be attributed to a department or cost center?

• Can administrators audit sensitive operations?

Evandro explained that enterprises sometimes build additional platform capabilities on top of AWS services to meet such requirements.

That approach can provide flexibility, but it also introduces engineering responsibilities.

Teams must maintain the controls, ensure their availability, and keep them consistent as the application evolves.

The takeaway is straightforward:

An AI agent’s ability to perform an action should never be confused with its authorization to perform that action.

Production readiness requires both.

Building the Agent Is Easy. Keeping It Reliable Is the Hard Part.

Evandro drew a parallel between today’s agentic AI challenges and the earlier evolution of machine learning operations.

In traditional machine learning, training a model was only one part of the problem.

Teams also needed to deploy models, monitor performance, manage infrastructure, handle changes in data, and maintain acceptable prediction quality over time.

Agentic AI is following a similar pattern.

A successful prototype proves that an agent can complete a task under certain conditions.

A production-ready system must demonstrate that it can perform consistently across a much broader range of conditions.

That includes conditions the original developers may not have anticipated.

Resilience must be designed, not assumed

Imagine an AI agent that processes insurance claims.

It retrieves policy information, validates submitted documents, checks internal records, and prepares a recommendation.

Now suppose the document retrieval service becomes temporarily unavailable.

A poorly designed agent may continue reasoning with incomplete information and generate a confident but unsupported recommendation.

A more resilient architecture would recognize that a required dependency failed.

Depending on the business workflow, it might retry the operation, return a controlled error, request human review, or pause execution until the necessary information becomes available.

The important point is that failure should not silently become an incorrect business decision.

This is where production engineering practices become essential.

Timeout policies, bounded retries, fallback strategies, idempotent operations, and human escalation paths can help manage failures in complex workflows.

Not every application needs every mechanism. The right approach depends on the consequences of failure and the reliability requirements of the system.

But these decisions need to be made deliberately.

An agent should not be considered production-ready simply because its happy-path demonstration succeeds.

The Most Important Question: How Do You Know Your Agent Is Actually Good?

Perhaps the most consequential topic in the conversation was evaluation.

Evandro emphasized that organizations need to assess whether agents are accurately performing the tasks they were designed to complete.

This becomes especially important because evaluating agents is fundamentally different from checking whether a traditional application returned the expected output.

A conventional function might have a deterministic result.

Given the same input, it should return the same output.

An AI agent may generate different wording, choose different tool sequences, or follow alternative reasoning paths while still completing the task successfully.

This makes correctness harder to define.

See you next time!

The harder half is execution, slippage and market microstructure, and knowing what survives a live market. EPAT is QuantInsti's six-month weekend programme covering exactly that, designed for engineers and data professionals. A free one-hour live Q&A walks through the curriculum and the Blueshift backtesting platform, with an open floor for technical questions.

Register Free

Sponsored

👋 Hi there,

Welcome to DataPro #190 | Special FinTech Edition!

AI is transforming financial services. But what happens when intelligent agents meet fragmented data, outdated infrastructure, and disconnected financial systems?

From fraud detection and risk modeling to real-time financial analytics, institutions are racing to adopt agentic AI. Yet the biggest obstacle isn’t necessarily the intelligence of the models. It’s the reliability of the data behind them.

In this special FinTech edition, Tamer Khraisha, FinTech Engineer at Flash Payments, joins Vaideeshwari Roshan, Relationship Lead at Packt, to uncover why legacy data architectures are holding back financial AI and what it takes to build trustworthy, AI-ready financial systems.

Can financial institutions truly scale agentic AI without first fixing their data foundations?

Let’s find out.

Cheers,

Merlyn Shelley,

Growth Lead, Packt.

Deterministic AI on CPUs: Build low-latency LLM workflows

📅 Friday, October 30 | 9:30 AM–1:30 PM EDT

Join Ben Auffarth to build deterministic, low-latency AI workflows on CPUs using ModernBERT, LFM2.5, conformal prediction, and LangGraph, with reliable routing, uncertainty checks, and selective LLM escalation for production-ready AI systems.

Join Ben Auffarth Live

Why Financial Data Architecture Is Holding Back Agentic AI

Tamer Khraisha, FinTech Engineer at Flash Payments, joins Packt’s Vaideeshwari Roshan to explore how fragmented data, legacy systems, and governance gaps are slowing AI adoption in financial services.

The AI revolution has exposed an old problem

Imagine a financial institution that has invested millions in modernizing its technology.

It has cloud infrastructure, sophisticated data warehouses, real-time payment systems, fraud detection tools, and perhaps even an ambitious generative AI roadmap.

Now imagine asking its AI assistant a seemingly straightforward question:

Which customers are showing unusual transaction behavior, increasing credit exposure, and signs of financial distress across their accounts?

The information needed to answer that question probably exists.

The problem is that it may exist in ten different places.

Transaction histories sit in one system. Credit profiles live in another. Market activity is captured elsewhere. Compliance records are maintained separately. Customer interactions may be distributed across several additional platforms.

Each system works. Each serves a business purpose. Yet none necessarily provides the complete picture.

For a human analyst, assembling that picture could take hours or days. For an autonomous AI agent expected to make decisions in seconds, the same fragmentation becomes a fundamental architectural obstacle.

This is the challenge at the center of Packt’s conversation with Tamer Khraisha, whose experience spans financial research platforms, asset management technology, financial data management, and payments infrastructure.

Speaking with Vaideeshwari Roshan, Tamer identified a problem that predates generative AI but has become significantly more urgent because of it.

Fragmentation.

And according to Tamer, it is one of the biggest obstacles standing between financial institutions and meaningful AI adoption.

Design a production-ready RAG architecture for enterprise GenAI applications

📅 Saturday, October 31 | 9:30 AM–1:30 PM EDT
Design a production-ready RAG architecture for enterprise GenAI, covering ingestion, chunking, metadata enrichment, vector search, retrieval tuning, evaluation, governance, and cost control to build grounded, traceable, and reliable AI applications.

Join Brian Live

1. The problem isn’t missing data. It’s disconnected data.

Financial institutions have spent decades collecting information.

Every transaction, customer interaction, trading activity, risk assessment, and compliance event generates data. Over time, this information has become one of the industry’s most valuable assets.

But collecting data and making it usable are two very different achievements.

Tamer describes fragmentation as a challenge that exists both within individual institutions and across financial markets.

A single customer’s information might be distributed across more than ten systems, each managed by different teams, built for different purposes, and operating on different update schedules.

This creates a familiar problem for financial analytics teams: there is no reliable, continuously updated view of the customer.

Traditional reporting systems have learned to work around this limitation. Data is extracted, transformed, reconciled, and eventually delivered to analysts through scheduled pipelines.

AI changes that arrangement.

An intelligent agent investigating suspicious activity cannot always wait for the next batch processing cycle. A risk model responding to changing market conditions may need information from several systems simultaneously.

And an AI-powered financial assistant needs more than isolated records. It needs context.

The important shift is that AI doesn’t simply require more data. It requires connected, trustworthy, and accessible data at the moment a decision is being made.

That is a considerably harder engineering problem.

2. Why yesterday’s data warehouses struggle with tomorrow’s AI agents

To understand why conventional financial data architectures are under pressure, Tamer introduces an important distinction.

There are two broad ways financial institutions use their data: operations and analytics.

Operations concern what the business is doing right now. Processing payments, updating account balances, recording transactions, and executing routine workflows.

Analytics concerns understanding what has happened, why it happened, and what might happen next.

Historically, these environments were designed around relatively predictable requirements.

An analyst knew which question needed answering. A data engineer could construct the necessary query. A reporting pipeline could be optimized to produce the same output repeatedly.

This predictability shaped how financial data platforms were built.

But agentic AI introduces a very different kind of workload.

Consider a traditional financial dashboard that retrieves transaction totals for the previous quarter. Its queries are largely predetermined.

Now consider an autonomous financial analysis agent.

It might begin by examining transaction patterns, identify an unexpected change, retrieve credit exposure information, compare that information with historical behavior, and then request additional compliance data.

Each step can generate a new question.

Each question may require a different combination of datasets.

And the engineering team cannot always predict those queries in advance.

That is where the limitations of conventional architectures become visible.

The data may be available somewhere within the institution, but there is no guarantee that an AI agent can discover it, access it safely, interpret it correctly, or retrieve it quickly enough.

Traditional analytics architectures were often optimized for questions people already knew they wanted to ask. Agentic AI must also support questions that emerge during the analysis itself.

This difference has enormous implications for data platform design.

Harness Engineering: Build an AI Data Analyst You Can Actually Trust

📅 Saturday, October 31 | 9:30 AM–1:30 PM EDT

Join Bruno Gonçalves to build a production-ready AI data analyst in Python using MCP tools, context management, guardrails, output verification, and fault-tolerant execution to create reliable, secure, and trustworthy AI agents.

Join Bruno Gonçalves Live

3. AI has evolved faster than the infrastructure supporting it

One of Tamer’s most important observations concerns the pace of technological change.

AI capabilities have advanced rapidly, but the investment required to modernize enterprise data foundations has not always kept pace.

This imbalance is particularly visible in financial services.

Unlike a startup building its technology stack from scratch, an established financial institution may operate infrastructure accumulated over several decades.

Mainframes coexist with relational databases. Enterprise data warehouses sit alongside cloud storage. New streaming platforms connect to legacy batch processing systems.

Each technology layer was introduced to solve a particular business problem.

Over time, these solutions became deeply embedded in the organization’s operations.

Replacing them is rarely straightforward.

Financial institutions must preserve transaction integrity, maintain regulatory compliance, protect customer information, and avoid disrupting critical services.

Consequently, modernization often means adding another layer rather than removing an existing one.

The result can be an increasingly complex architecture in which the movement, interpretation, and governance of data become harder to manage.

AI exposes these weaknesses because it depends on information flowing across those boundaries.

A sophisticated model cannot compensate for a missing customer record, an inconsistent identifier, or an outdated risk profile.

Nor can an autonomous agent reliably reason about information whose meaning changes from one system to another.

This explains why Tamer places such emphasis on data quality.

The challenge is not simply whether a financial institution has adopted modern AI models.

It is whether the institution has built the foundations those models need to operate reliably.

4. When a financial AI agent encounters fragmented data

Consider a hypothetical fraud investigation.

A payment monitoring system identifies an unusual transaction. An AI agent is asked to determine whether the activity represents a genuine threat.

To reach a useful conclusion, the agent might need to examine several signals.

On paper, the workflow appears straightforward.

In practice, every connection introduces potential failure points.

What happens if the customer identifier in the transaction system does not match the identifier in the credit database?

What happens if the compliance dataset was last refreshed several hours ago?

What happens if the agent retrieves an incomplete record and interprets the absence of information as evidence that no risk exists?

These are not merely database integration issues.

They can affect the quality of financial decisions.

A false positive might unnecessarily interrupt a legitimate payment. A false negative could allow suspicious activity to go unnoticed.

The lesson is that AI reliability depends on more than model accuracy.

It also depends on the freshness, completeness, consistency, and provenance of the information used to produce an answer.

Build and Stress-Test a multi-asset portfolio with Python

📅 Saturday, November 28 | 9:30 AM–1:30 PM EST

Join Cordell Tanny, CFA, FRM, FDP, to build and stress-test a 12-asset investment portfolio in Python, using portfolio optimization, CVaR, Monte Carlo simulation, and downside risk analysis to evaluate diversification, manage market volatility, and generate AI-assisted investment risk reports.

Join Cordell Tanny Live

5. Data quality is becoming an AI reliability problem

For years, organizations have treated data quality as a reporting and governance concern.

Incomplete records lead to inaccurate dashboards. Inconsistent definitions create disagreements between business units. Delayed updates produce outdated reports.

Those problems remain important.

But when AI systems begin making recommendations or initiating actions, poor data quality introduces another level of risk.

An incorrect dashboard may mislead an analyst.

An autonomous agent operating on incorrect data could recommend or initiate a flawed workflow before the underlying inconsistency is discovered.

That distinction changes how financial institutions should think about their data foundations.

The relevant questions are no longer limited to whether the data is technically available.

They also include whether an AI system can determine which source is authoritative, how recently the information was updated, what the information means, and whether the system has permission to use it.

A practical AI-ready architecture therefore needs several capabilities working together: reliable data integration, consistent definitions, appropriate access controls, freshness monitoring, traceable data lineage, and mechanisms for handling uncertainty.

These are not new concepts in data engineering.

What is new is the level of dependence that autonomous AI systems place on them.

As AI becomes more capable of acting independently, the cost of unreliable context increases.

This is why improvements in model reasoning alone cannot solve the architectural challenges Tamer describes.

6. The challenge no longer belongs only to large institutions

Another revealing moment in the conversation comes when Vaideeshwari asks whether architectural failures typically emerge at a particular scale.

Is fragmentation primarily an enterprise problem? Does it become visible only when an organization reaches a certain size or data volume?

Tamer suggests that the answer has changed.

Ten or twenty years ago, data fragmentation might have been viewed primarily as an organization-wide challenge associated with large institutions.

Today, the effects can appear at almost every level.

A small team integrating an AI assistant with several SaaS platforms can encounter inconsistent data definitions.

A growing fintech combining payment processing, customer analytics, and compliance systems can face the same integration problems as a much larger financial institution.

An enterprise deploying agents across multiple departments simply experiences those challenges across a wider operational landscape.

The underlying issue is not necessarily the amount of data.

It is the number of disconnected contexts an AI system must reconcile to complete a task.

This makes data architecture an immediate concern for organizations experimenting with AI, rather than a problem they can postpone until they reach enterprise scale.

7. What should financial institutions build before scaling agentic AI?

Tamer’s observations point toward a broader architectural lesson.

Financial institutions do not necessarily need to replace every legacy system before deploying AI.

But they do need to rethink how those systems expose and govern information.

A useful starting point is to move away from treating each dataset as an isolated asset and toward designing an integrated, governed information layer.

In practical terms, that means prioritizing five areas.

• Connected data foundations: Establish consistent ways to retrieve and combine information across operational systems, analytical platforms, and business units.

• Trusted data definitions: Ensure that critical concepts such as customer identity, account exposure, transaction status, and risk classifications have consistent meanings.

• Freshness and availability: Make data latency visible and distinguish between information suitable for historical analysis and information required for near-real-time decisions.

• Governed agent access: Give AI systems access to the information they need without bypassing authorization, privacy, compliance, or audit requirements.

• Verification and human oversight: Design workflows that can identify missing information, flag uncertainty, and escalate consequential decisions when the evidence is insufficient.

These priorities are an architectural interpretation of the issues raised in the conversation, rather than a framework formally prescribed by Tamer.

Their common objective is simple: ensure that an AI system can reach the right information, understand its limitations, and operate within clearly defined boundaries.

Without that foundation, adding more powerful models may simply make an institution’s existing data problems harder to detect.

8. The next competitive advantage in financial AI may not be the model

The financial services industry has entered an era in which sophisticated AI capabilities are increasingly accessible.

Organizations can experiment with large language models, deploy specialized financial assistants, and build agentic workflows without developing foundation models themselves.

But access to intelligence is not the same as the ability to apply it reliably.

Two institutions may use comparable AI models and achieve dramatically different outcomes.

One may have fragmented customer records, inconsistent data definitions, and slow analytical pipelines.

The other may have well-governed information, reliable integration, and clearly defined controls for AI-driven decisions.

Even with similar models, the second institution is better positioned to produce useful, consistent, and auditable results.

That is the deeper implication of Tamer’s argument.

The next stage of AI adoption in finance will not be determined solely by how capable the models become.

It will also depend on whether financial institutions can modernize the data foundations on which those models rely.

The bigger question: Are we building intelligent systems on unintelligent foundations?

Throughout the conversation, one theme keeps resurfacing.

Financial institutions have accumulated enormous amounts of valuable data. AI has created extraordinary new opportunities to use that information. Yet the architectures connecting the two have not always evolved at the same pace.

The industry is now confronting the consequences.

Agentic AI can formulate new questions, navigate complex workflows, and support increasingly sophisticated analysis.

But it cannot automatically repair decades of fragmented data architecture.

It cannot guarantee that disconnected systems agree.

And it cannot reliably make decisions using information it cannot access, validate, or interpret.

For data engineers, architects, fintech leaders, and AI practitioners, this creates a shift in priorities.

The question is no longer simply, How do we introduce AI into our financial systems?

It is:

How do we redesign our financial data systems so that AI can actually be trusted to work with them?

That may be one of the most important engineering questions facing financial services today.

About the expert

Tamer Khraisha is a FinTech Engineer at Flash Payments, with more than a decade of experience across financial technology, financial research platforms, asset management, and payments. With a background in finance, his work focuses on the intersection of financial systems and data management. He has also authored work on financial data management.

See you next time!

Live Webinar: Query On-Prem Data from Databricks. No pipelines needed.

Every copy of regulated data is another pipeline to secure, govern, and audit. Join MinIO's live webinar on Oct 22 at 10 AM PDT to see how AIStor Table Sharing uses the OpenSharing protocol to let Databricks query on-prem data in place, with zero copies or pipelines to maintain. Reserve your spot.

Register

👋 Hi there,

Welcome to DataPro #189!

Your AI agents work perfectly on their own. So why does everything start falling apart when they work together?

That’s the problem we’re exploring in this week’s Special Expert Conversation Edition.

Building individual AI agents has become easier than ever. But connecting them into a reliable multi-agent system introduces a whole new set of challenges: broken orchestration, inconsistent context, cascading errors, rising costs, and security risks that rarely show up in a polished demo.

So, how do you move from a collection of intelligent agents to an enterprise AI system you can actually trust?

In this special edition, Somil Rastogi, Principal Solutions Architect at SAS, joins Deepesh Patel, Relationship Lead at Packt, to explore the architectural decisions behind production-ready multi-agent AI.

From coordinating agents and managing shared memory to validating decisions, controlling autonomy, and preventing failures, we unpack what it really takes to make multi-agent systems work beyond the prototype.

The big takeaway? More intelligent agents don’t necessarily make a smarter system. Better architecture does.

Let’s dive in.

Cheers,

Merlyn Shelley,

Growth Lead, Packt.

Deterministic AI on CPUs: Build low-latency LLM workflows

📅 Friday, October 30 | 9:30 AM–1:30 PM EDT

Join Ben Auffarth to build deterministic, low-latency AI workflows on CPUs using ModernBERT, LFM2.5, conformal prediction, and LangGraph, with reliable routing, uncertainty checks, and selective LLM escalation for production-ready AI systems.

Join Ben Auffarth Live

THE MULTI-AGENT RELIABILITY PROBLEM

Adding More AI Agents Won’t Fix Your Workflow. Here’s What Might.

Somil Rastogi, Principal Solutions Architect at SAS, joins Deepesh Patel, Relationship Lead at Packt, to explore why multi-agent AI systems become harder to control as they scale, and how better orchestration, shared context, and architectural decisions can turn promising prototypes into reliable enterprise applications.

Imagine you’ve built three AI agents.

The first retrieves information from your enterprise knowledge base. The second analyzes it and recommends an action. The third executes that action through a connected business application.

Individually, each agent performs well. Your tests pass. The demo impresses stakeholders.

Then you connect them.

Suddenly, the retrieval agent returns incomplete context. The analysis agent confidently fills in the gaps. The execution agent acts on that recommendation before anyone notices the problem.

Three capable agents. One unreliable system.

This is the uncomfortable reality behind the excitement surrounding multi-agent AI. While frameworks are making it easier to build autonomous agents, getting those agents to collaborate reliably introduces an entirely different class of engineering problems.

And those problems become significantly more serious when agents move beyond generating responses and begin interacting with enterprise data, business workflows, and production systems.

In a Packt expert conversation hosted by Deepesh Patel, Relationship Lead at Packt, Somil Rastogi, Principal Solutions Architect at SAS, brings an enterprise architecture perspective to the challenge of designing multi-agent AI systems.

The discussion centres on a question that every AI engineer and solution architect should be asking:

If individual agents are becoming more intelligent, why does the architecture connecting them remain so difficult to get right?

To understand the answer, we need to look beyond models and prompts and examine what actually happens when autonomous components must work together.

1. Why Adding More Agents Doesn’t Necessarily Make Your System Smarter

The appeal of multi-agent AI is easy to understand.

Instead of relying on a single model to perform an entire workflow, developers can assign specialized responsibilities to different agents.

One agent handles research. Another performs analysis. A third plans actions. A fourth verifies the results.

It resembles the way human teams divide responsibilities.

But there’s a critical difference.

Human teams rely on shared understanding, established processes, accountability, and the ability to clarify ambiguous instructions.

AI agents don’t automatically possess these coordination mechanisms.

They operate within the information, tools, permissions, and instructions their architecture provides.

Consider a financial research application.

A data retrieval agent collects market information. A forecasting agent generates projections. A reporting agent produces an executive summary.

What happens if the retrieval agent provides outdated financial data?

The forecasting agent might generate a mathematically valid prediction using incorrect inputs. The reporting agent could then present that prediction as a credible business insight.

Each component may appear to have completed its assigned task successfully.

Yet the overall system has failed.

This is one of the most important distinctions in multi-agent engineering:

Individual agent performance and end-to-end system reliability are not the same thing.

The architecture must account for what happens between agents, not just what happens inside them.

That includes how tasks are delegated, how outputs are validated, how context is transferred, and how the system responds when an agent makes a mistake.

For enterprise architects, the challenge is not simply creating more specialized agents.

It is establishing the mechanisms that allow those agents to behave as a coherent, dependable system.

2. The Orchestration Problem: Who Decides What Happens Next?

Suppose a customer asks an AI-powered financial assistant to analyze their portfolio, identify potential risks, and recommend adjustments.

The request involves several activities.

The system must retrieve portfolio information, analyze market exposure, evaluate risk, and generate a recommendation.

But who decides which agent performs each task?

Who determines whether the output is good enough to proceed?

And what happens when two agents disagree?

This is where orchestration becomes central to multi-agent architecture.

There are several ways to coordinate agents, each with different implications for reliability and control.

A central supervisor controls the workflow

In a supervisor architecture, one coordinating component delegates tasks to specialized agents.

For example, a supervisor might send a research task to one agent, pass its output to an analytical agent, and then request validation before producing a final response.

This provides a relatively clear execution structure.

It also gives developers a central point for enforcing workflow rules.

However, it introduces a dependency on the supervisor’s decisions. If the supervisor routes tasks incorrectly or fails to recognize an invalid result, the entire workflow can be affected.

Agents operate in a predefined sequence

A sequential architecture works more like a production line.

Each agent completes a defined task and passes its output to the next stage.

This can be effective for document processing, reporting, and other workflows where the order of operations is predictable.

But it also means that an early mistake can influence every subsequent step.

Without intermediate validation, errors can travel through the workflow unnoticed.

Agents collaborate dynamically

More flexible architectures allow agents to communicate, exchange information, and decide how to approach a task.

This can be valuable for complex research or planning problems where the next step cannot always be determined in advance.

However, flexibility introduces uncertainty.

Agents may repeat work, disagree over responsibilities, or enter unnecessary communication loops.

The result can be higher costs, longer execution times, and more difficult debugging.

The architectural decision therefore depends on the nature of the problem.

The goal isn’t to maximize agent autonomy. It’s to introduce enough autonomy to solve the problem while retaining control over how the solution is produced.

For developers, this means choosing orchestration patterns based on business requirements rather than selecting the most sophisticated framework available.

Design a production-ready RAG architecture for enterprise GenAI applications

📅 Saturday, October 31 | 9:30 AM–1:30 PM EDT
Design a production-ready RAG architecture for enterprise GenAI, covering ingestion, chunking, metadata enrichment, vector search, retrieval tuning, evaluation, governance, and cost control to build grounded, traceable, and reliable AI applications.

Join Brian Live

3. Your Agents May Be Sharing Information, but Are They Sharing the Same Context?

One of the less obvious challenges in multi-agent systems is maintaining a consistent understanding of the task.

Imagine an insurance claims application with three agents.

The first extracts information from a submitted claim. The second checks policy eligibility. The third recommends the next action.

Now imagine the policy information changes between the second and third stages.

The eligibility agent may be operating on an older version of the policy, while the recommendation agent has access to newer information.

Both agents could produce reasonable outputs based on what they know.

But the system may arrive at an inconsistent decision.

This is not primarily a language-model problem.

It is a data consistency and state-management problem.

As multi-agent applications become more sophisticated, developers must decide what information agents should share, where that information should be stored, and which source should be considered authoritative.

Harness Engineering: Build an AI Data Analyst You Can Actually Trust

📅 Saturday, October 31 | 9:30 AM–1:30 PM EDT

Join Bruno Gonçalves to build a production-ready AI data analyst in Python using MCP tools, context management, guardrails, output verification, and fault-tolerant execution to create reliable, secure, and trustworthy AI agents.

Join Bruno Gonçalves Live

Shared memory isn’t the same as reliable memory

Giving agents access to a common memory store might seem like an obvious solution.

But shared memory introduces questions of its own.

What information should be retained?

Which agent can update it?

How are conflicting updates resolved?

What happens when stored information becomes outdated?

And how can developers distinguish between verified enterprise data and an interpretation generated by another agent?

These distinctions matter because an agent-generated summary is not necessarily equivalent to an authoritative business record.

For example, a customer-support agent might summarize a previous interaction as “the customer is eligible for a refund.”

But that summary should not automatically override the organization’s actual refund policy or transaction history.

A more dependable architecture separates temporary conversational context from persistent business state and authoritative data sources.

It also defines explicit rules for reading, updating, and validating shared information.

The important question isn’t how much information an agent can remember. It’s whether the agent is using the correct information when making a decision.

This becomes especially important in enterprise environments where agents interact with databases, analytics platforms, document repositories, and continuously changing operational systems.

4. The Enterprise Data Problem: An Agent Is Only as Reliable as the Information It Uses

One of the biggest differences between an impressive AI prototype and a production-ready enterprise application is the environment in which it operates.

A prototype might use a small collection of documents and a few carefully designed examples.

A production application may need to work across thousands of documents, multiple databases, legacy systems, internal APIs, and business-specific access policies.

The difficulty is not simply retrieving information.

It is retrieving the right information, interpreting it correctly, and using it within the appropriate business context.

Consider a multi-agent system designed to support enterprise risk analysis.

One agent retrieves operational data. Another analyzes historical patterns. A third evaluates potential risks. A fourth prepares recommendations.

If the underlying data is incomplete or inconsistent, the system may produce convincing but misleading conclusions.

Adding more reasoning agents does not fix unreliable source data.

In fact, it can make the problem harder to detect because each agent may transform the original information before passing it downstream.

This is where techniques such as retrieval-augmented generation, structured data access, metadata management, and deterministic validation become important.

Rather than allowing agents to rely entirely on generated interpretations, architects can connect them to governed enterprise information sources.

For example:

• A retrieval agent can locate relevant documents and preserve source references.

• An analytical component can execute structured queries against validated datasets.

• A policy engine can evaluate deterministic business rules.

• A reasoning agent can interpret the combined evidence and explain the result.

• A validation component can check whether the proposed action satisfies the required constraints.

These responsibilities do not necessarily require five separate LLM-powered agents.

Some may be better implemented as conventional software services.

That distinction is essential.

Not every intelligent workflow component needs to be an AI agent. Sometimes the most reliable part of an agentic architecture is the part that doesn’t use an LLM at all.

The practical solution is to combine probabilistic reasoning with deterministic systems wherever correctness and accountability matter.

Build and Stress-Test a multi-asset portfolio with Python

📅 Saturday, November 28 | 9:30 AM–1:30 PM EST

Join Cordell Tanny, CFA, FRM, FDP, to build and stress-test a 12-asset investment portfolio in Python, using portfolio optimization, CVaR, Monte Carlo simulation, and downside risk analysis to evaluate diversification, manage market volatility, and generate AI-assisted investment risk reports.

Join Cordell Tanny Live

5. When an Agent Makes a Mistake, How Does the System Know?

Traditional software applications are generally built around defined execution paths.

Developers know which functions are called, what inputs they expect, and how failures should be handled.

Agentic systems introduce more variability.

An agent may choose a different tool, interpret an instruction differently, or generate an unexpected intermediate result.

In a multi-agent workflow, that variability can influence the behaviour of other agents.

Consider an automated incident investigation system.

A monitoring agent detects an anomaly.

A diagnostic agent investigates potential causes.

A remediation agent recommends corrective action.

What happens if the diagnostic agent incorrectly identifies the root cause?

Should the remediation agent proceed?

Should another component verify the diagnosis?

Should the system request human approval?

These questions reveal why validation and failure recovery need to be built into the architecture.

A successful response isn’t enough

Many AI applications are evaluated primarily on whether they produce a correct final answer.

For multi-agent systems, that approach is incomplete.

See you next time!

📅 Friday, October 30 | 9:30 AM–1:30 PM EDT

Join Ben Auffarth to build deterministic, low-latency AI workflows on CPUs using ModernBERT, LFM2.5, conformal prediction, and LangGraph, with reliable routing, uncertainty checks, and selective LLM escalation for production-ready AI systems.

Join Ben Auffarth Live

👋 Hi there,

Welcome to DataPro #188!

Your AI is only as reliable as the data, decisions, and systems behind it.

Your dashboards are accurate. Your pipelines are running. Your data is well-structured. So why does your LLM still retrieve the wrong information or generate unreliable answers?

That’s the challenge at the heart of this week’s expert conversation.

Jennifer Ratten, VP of Application Engineering | Data Strategy and Practice at NFP, an Aon company, joins Vaideeshwari Roshan, Portfolio Manager at Packt, to explain what it really takes to make enterprise data LLM-ready.

Drawing on a production project involving hundreds of thousands of insurance documents, Jennifer reveals how her team combined Microsoft Fabric, Cosmos DB, RAG, semantic chunking, and AI agents to reduce manual processing time by 97%.

But the biggest challenge wasn’t the LLM. Generic chunking failed, retrieval confused plan types, and traditional BI-ready data structures needed rethinking. The solution came from better semantic modeling, context-aware retrieval, and continuous validation.

The lesson? Clean data isn’t necessarily LLM-ready data, and a better model won’t fix poorly structured knowledge.

That same challenge of making AI more accurate, secure, and reliable runs through this week’s developments.

This Week’s Highlights

• NVIDIA’s PivotOPD: Teaches AI agents to prevent and recover from critical mistakes, improving ALFWorld performance by 5.5% and SWE-Bench Verified resolution by 3.2%.

• Perplexity’s Multimodal Embeddings: Introduces 0.6B and 9B models that search text, images, and PDFs without OCR, trained on 186 million query-document pairs across 46 languages.

• Claude Haiku 5.5 on AWS: Anthropic’s faster, cost-efficient model arrives on Amazon Bedrock at approximately 75% lower cost than Haiku 4.5, supporting coding, tool use, and high-volume AI subagents.

• Google Cloud’s Remote MCP Server: Enables AI agents to execute hundreds of gcloud and BigQuery commands through secure, managed cloud infrastructure.

• AWS Secures Enterprise RAG: Adds real-time permission checks to Amazon Quick and Bedrock Knowledge Bases, preventing unauthorized document retrieval. Mondelēz International uses Amazon Quick across 35,000+ employees.

• AWS Rethinks Agentic AI ROI: Introduces a framework measuring automation through time savings, error reduction, decision quality, and maintenance efficiency, with Kitsa reporting 91% cost savings.

• Google’s MCP Toolbox Java SDK 1.0: Brings secure database access, session management, and type-safe orchestration to enterprise Java agents using AlloyDB, Spring Boot, and LangChain4j.

The bigger picture? From preparing data for LLMs to building agents that recover from mistakes, enforce security, and deliver measurable business value, enterprise AI is moving beyond impressive demos toward systems that actually work in production.

Let’s dive in.

Cheers,

Merlyn Shelley,

Growth Lead, Packt.

Design a production-ready RAG architecture for enterprise GenAI applications

📅 Saturday, October 31 | 9:30 AM–1:30 PM EDT
Design a production-ready RAG architecture for enterprise GenAI, covering ingestion, chunking, metadata enrichment, vector search, retrieval tuning, evaluation, governance, and cost control to build grounded, traceable, and reliable AI applications.

Join Brian Live

Your Data Works for BI. So Why Does Your LLM Keep Getting It Wrong?

How one enterprise transformed hundreds of thousands of unstructured insurance documents into an LLM-ready data platform, reduced manual processing time by 97%, and discovered why clean data alone is not enough for production AI.

Interview hosted by Vaideeshwari Roshan, Portfolio Manager, Packt | Featuring Jennifer Ratten, VP, Application Engineering | Data Strategy and Practice, NFP, an Aon company

Your dashboards are accurate. Your SQL queries return the right numbers. Your data pipelines are running smoothly.

Then you connect a large language model, ask it a seemingly straightforward question, and suddenly the answers are incomplete, inconsistent, or simply wrong.

What changed? The data didn’t. The way it needed to be understood did.

This is one of the most overlooked challenges in enterprise AI. Data engineered for reporting, analytics, and business intelligence is not necessarily ready for retrieval, reasoning, or conversational AI.

And the consequences extend far beyond a disappointing chatbot response. In industries where accuracy, compliance, and trust matter, a poorly grounded answer can undermine an entire AI initiative.

In a conversation hosted by Vaideeshwari Roshan, Portfolio Manager at Packt, Jennifer Ratten, VP of Application Engineering, Data Strategy and Practice at NFP, an Aon company, shared what it actually takes to build an LLM-ready data foundation.

Drawing on a production implementation involving hundreds of thousands of insurance documents, Ratten explained how her team combined document intelligence, retrieval-augmented generation (RAG), semantic modeling, structured databases, and AI agents to transform a highly manual process into a scalable AI-enabled workflow.

The result was striking: approximately 97% less manual processing time, achieved by rethinking how data was prepared, structured, validated, and retrieved.

But the most valuable lesson wasn’t about the model or the technology stack.

It was about understanding the data well enough to make AI trustworthy.

Harness Engineering: Build an AI Data Analyst You Can Actually Trust

📅 Saturday, October 31 | 9:30 AM–1:30 PM EDT

Join Bruno Gonçalves to build a production-ready AI data analyst in Python using MCP tools, context management, guardrails, output verification, and fault-tolerant execution to create reliable, secure, and trustworthy AI agents.

Join Bruno Gonçalves Live

The Business Problem: Thousands of Health Plans, Hundreds of Thousands of Documents, and Too Much Manual Work

Before moving into technology, Ratten spent nearly two decades working with insurance agencies. Her career began on the business side, where operating within a small organization made efficiency a necessity rather than an aspiration.

She started automating repetitive tasks through macros and reporting, gradually moving into analytics, Power BI, and Microsoft’s Power Platform. As AI capabilities evolved, she began applying them to operational problems that traditional automation could not easily solve.

Following her company’s acquisition, she transitioned fully into technology, focusing on building and scaling solutions using Microsoft Fabric and related technologies.

That combination of business experience and technical expertise became particularly important when tackling one of her most demanding projects.

The organization was processing hundreds of thousands of unstructured documents, including PDFs, Word files, and text documents containing detailed health insurance plan information.

Each document could contain different terminology, layouts, coverage rules, and benefit structures.

Insurance agents were responsible for reviewing these documents manually, extracting relevant details, and entering the information into multiple downstream systems.

The challenges were predictable but significant:

• Manual extraction: Employees spent substantial time locating and interpreting information buried inside documents.

• Inconsistent terminology: Similar benefits could be described differently across providers and plan types.

• Frequent updates: Changes to health plans required information to be reviewed and updated repeatedly.

• Limited scalability: Growing document volumes increased operational effort and the risk of errors.

• Fragmented access: Information existed across documents and systems rather than within a consistent, queryable knowledge foundation.

The system supported an operational environment of roughly 70,000 employees, with individual employees responsible for managing thousands of health plans.

At that scale, even minor inefficiencies could become substantial operational bottlenecks.

The goal was not simply to automate document reading.

It was to build a system that could extract, validate, standardize, compare, and retrieve insurance information reliably, while allowing users to interact with that information through natural language.

Figures shared by Jennifer Ratten. The 99% figure represents a project requirement, not a reported final accuracy result.

Why One LLM Was Never Going to Solve the Problem

One of the first architectural decisions was recognizing that the solution could not rely on a single LLM prompt or a standalone RAG application.

The system needed several complementary AI capabilities working together.

Retrieval-augmented generation grounded responses in relevant source documents, reducing dependence on the model’s internal knowledge.

Document classification helped distinguish between different formats, plan categories, and content structures.

Section-level summarization extracted and normalized important details from specific benefit categories, while higher-level summarization made broader plan information easier to interpret.

Conversational AI allowed users to ask questions without navigating multiple documents or manually constructing database queries.

And AI agents supported more complex workflows involving reasoning, plan comparisons, and response generation.

These were not isolated features bolted onto a document repository. They formed an integrated data processing and reasoning architecture.

That distinction matters.

A chatbot might answer a question about a single document reasonably well. A production insurance application must identify the correct plan, retrieve the relevant coverage section, distinguish similar terminology, compare information against other sources, and produce an answer consistent with the underlying records.

Each step introduces another opportunity for error.

The challenge was not getting an LLM to read documents. It was ensuring that every piece of information reaching the LLM was relevant, interpretable, and trustworthy.

Inside the Architecture: From Raw Documents to AI-Ready Intelligence

Ratten described an architecture that separated document ingestion, semantic preparation, structured storage, and AI-driven reasoning.

Conceptual representation based on the architecture described in the interview.

Documents first entered blob storage, where they were parsed and normalized.

The team then applied LLM-driven chunking logic designed to accommodate approximately 25 different document layout variations across insurance plan types.

This was a critical departure from generic document-processing approaches.

Rather than splitting text into arbitrary segments, the pipeline needed to preserve the meaning and relationships between insurance benefits, coverage conditions, and plan categories.

Once extracted and normalized, structured information was stored in Cosmos DB and Fabric SQL databases.

These data stores provided a consistent foundation for downstream querying and AI workflows.

On top of that foundation, agents built using Microsoft AI Foundry handled reasoning, comparisons, and final response generation.

Python and SQL formed the core of the transformation, validation, and orchestration work, with R Studio used selectively for analytical tasks.

The architecture reflects an important principle for modern data engineering: LLM readiness is not a final transformation applied before inference. It is an end-to-end design requirement.

Build and Stress-Test a multi-asset portfolio with Python

📅 Saturday, November 28 | 9:30 AM–1:30 PM EST

Join Cordell Tanny, CFA, FRM, FDP, to build and stress-test a 12-asset investment portfolio in Python, using portfolio optimization, CVaR, Monte Carlo simulation, and downside risk analysis to evaluate diversification, manage market volatility, and generate AI-assisted investment risk reports.

Join Cordell Tanny Live

The Chunking Problem: Why Generic RAG Failed to Deliver Accurate Answers

Chunking is often treated as a routine implementation detail in RAG systems.

Choose a chunk size, add overlap, generate embeddings, and move on.

For Ratten’s team, that approach was not sufficient.

Early experiments with generic chunking produced weaker accuracy because the structure of the documents carried important business meaning.

A health insurance document is not simply a collection of paragraphs. It contains relationships between plan types, coverage categories, conditions, exclusions, and benefit structures.

Splitting the document without respecting those relationships can remove the very context needed to answer a question correctly.

The team found that section-based chunking delivered substantially better results.

Instead of organizing content around arbitrary token boundaries, documents were divided into meaningful sections, such as hospital services or prescription coverage.

Those sections were then refined according to the structural differences between document types.

This approach made retrieval more aligned with how insurance professionals actually reason about coverage.

For example, a question about prescription benefits should retrieve the relevant prescription coverage section for the correct plan, not a semantically similar passage from another plan or benefit category.

The distinction sounds simple.

In production, it can determine whether the system delivers a reliable answer or a misleading one.

When the data existed, but retrieval still failed

One of the clearest examples emerged during user acceptance testing.

Users were asking for information about a specific health plan, but the system returned results covering multiple plan types.

The information was present in the underlying dataset.

The model was not necessarily inventing facts.

The failure was in how the data had been chunked and indexed.

The original indexing strategy did not sufficiently distinguish between plan categories, allowing unrelated plan information to enter the retrieval results.

The team addressed the issue by introducing additional indexing layers aligned with specific plan types.

This exposed a problem that many AI teams misdiagnose.

When an LLM produces the wrong answer, the immediate instinct is often to change the model, rewrite the prompt, or adjust the generation parameters.

But sometimes the model is simply receiving the wrong context.

Before optimizing generation, teams need to verify that retrieval is selecting the correct business entities and relationships.

Accuracy Wasn’t a Final Test. It Was Built into the Pipeline.

Given the sensitive and regulated nature of health insurance information, the project had demanding operational requirements.

Accuracy needed to meet a minimum target of 99%. Responses had to arrive within seconds or minutes rather than hours. Security, stability, and personalization were also essential.

Different users needed different types of outputs, and the system had to support those requirements without compromising consistency.

Interestingly, cost was not the primary constraint.

The organization was prepared to invest in the infrastructure required to deliver a reliable solution.

That shifted the engineering focus toward correctness, performance, and operational stability.

Data Science & ML Research Roundup

⭕ PivotOPD: Learning to Recover from Pivotal Mistakes in Multi-Turn Agents. NVIDIA Introduces PivotOPD to Help AI Agents Learn from Mistakes: NVIDIA researchers have developed PivotOPD, a new training framework that helps multi-turn AI agents identify, prevent, and recover from critical errors during complex tasks. Unlike conventional training methods, it teaches agents how to correct mistakes instead of letting failures compound. The approach improved task performan duction, decision quality, and maintenance efficiency. Real-world deployments demonstrate the impact: Kitsa achieved 91% cost savings, dLocal automated up to 75% of compliance reviews, and Genpact reduced supply-chain disruption analysis from days to minutes.

⭕ Multimodal embeddings beyond a single vector. Perplexity Launches Multimodal Embedding Models That Search PDFs Without OCR: Perplexity has introduced pplx-embed-v2-late, a family of 0.6B and 9B models that improve text and image retrieval using token-level embeddings instead of single vectors. Trained on 186 million query-document pairs across 46 languages, they support direct visual PDF search, shared embedding spaces, and faster, lower-cost querying while delivering competitive benchmark performance.

⭕ Introducing Claude Haiku 5.5 on AWS. Anthropic Launches Claude Haiku 5.5 on AWS at 75% Lower Cost: AWS has introduced Claude Haiku 5.5 on Amazon Bedrock and Claude Platform, offering faster, more efficient AI for coding, tool use, and high-volume agentic workflows. Priced approximately 75% below Haiku 4.5 for most tasks, it introduces adjustable reasoning effort and pairs with Opus 5.5 to handle complex workflows through cost-efficient subagents.

⭕ Google Cloud CLI remote MCP server in preview. Google Cloud Launches Remote MCP Server to Let AI Agents Manage Cloud Infrastructure: Google has introduced its Cloud CLI remote MCP server in preview, enabling AI agents to execute hundreds of gcloud and BigQuery commands without local installations. With secure sandbox execution, IAM-based access, Model Armor protection, and audit logging, it simplifies cloud management, infrastructure diagnostics, and data workflows through natural-language-driven automation.

⭕ Rethinking access control for RAG with Amazon Quick and Amazon Bedrock. AWS Introduces Real-Time Access Controls to Prevent Data Leaks in Enterprise RAG: AWS has enhanced Amazon Quick and Bedrock Knowledge Bases with real-time permission verification, addressing security gaps caused by outdated access controls. Its two-stage approach combines indexed filtering with live checks against sources like Google Drive, SharePoint, and Confluence, ensuring AI-generated answers only use authorized documents. Mondelēz International has deployed Amazon Quick for over 35,000 employees.

⭕ Data Agent Kit is now GA: Bring Google Data Cloud to any coding agent. AWS Strengthens Enterprise RAG Security with Real-Time Access Controls: AWS has introduced real-time permission checks in Amazon Quick and Bedrock Knowledge Bases to prevent unauthorized data exposure. By combining indexed filtering with live verification against original data sources, the system ensures AI responses respect current user permissions. Mondelēz International uses Amazon Quick across 35,000+ employees, highlighting its enterprise-scale adoption.

⭕ Beyond hours saved: Building the business case for agentic automation. AWS Rethinks Agentic AI ROI Beyond Hours Saved: AWS has introduced the Agentic Value Model, measuring automation returns through time savings, error reduction, decision quality, and maintenance efficiency. Real-world deployments demonstrate the impact: Kitsa achieved 91% cost savings, dLocal automated up to 75% of compliance reviews, and Genpact reduced supply-chain disruption analysis from days to minutes.

⭕ Announcing MCP Toolbox Java SDK v1.0: Agentic data access for the enterprise. Google Launches MCP Toolbox Java SDK 1.0 for Enterprise AI Agents: Google has released its production-ready Java SDK, enabling AI agents to securely access enterprise databases through standardized MCP connections. Featuring type-safe orchestration, dynamic authentication, session management, and protected SQL queries, it integrates with AlloyDB, Spring Boot, and LangChain4j to build conversational agents that retrieve information, maintain context, and execute transactions without exposing databases directly.

See you next time!

What happens when one question follows a researcher for more than three decades?

For Yousri El Fattah, founder of Causal Computing and co-author of Causal Inference with Bayesian Networks, that question was deceptively simple: How do we intervene in a system to achieve the outcome we actually want?

It took him from aeronautical engineering and stochastic control to learning systems, Bayesian networks, and eventually causal inference. In our lead story, Yousri takes us behind the book, tracing the ideas, research, and three years of writing that became Causal Inference with Bayesian Networks. It is also a timely reminder of a problem data teams are confronting today: prediction tells us what may happen, but making better decisions requires understanding why it happens and what changes when we intervene.

That same shift from simply observing systems to understanding, deciding, and acting runs through this week’s developments in data and AI.

BigQuery can now move from spotting a metric change to identifying its drivers and estimating causal impact. Google is combining GNNs, digital twins, and AI agents to help telecom networks diagnose failures and move toward autonomous operation. AWS is turning hours of video into something agents can interrogate conversationally, while HEMA is using MCP and Bedrock to turn scattered enterprise knowledge into governed answers.

And as agents take on more work, the infrastructure around them is catching up too: persistent agent storage, faster action ranking, Blackwell-powered Dataflow pipelines, and AI-assisted database operations that allow even a solo founder to operate a platform tracking 210,000+ public contracts.

The common thread is increasingly clear: AI is moving from generating outputs to understanding systems, identifying causes, choosing actions, and operating on the results. Yousri’s three-decade journey into causality suddenly feels remarkably current.

Build Production-Ready AI Agents : From Workflow to Deployment

📅 Saturday, September 26–Sunday, September 27 | 10:00 AM–2:00 PM EDT
Build production-ready AI agents with n8n, from real-world workflows and tool integrations to testing, security, governance, and deployment.

Join Ashley Live →

This week’s highlights

🔰 From portal-hopping to instant answers: HEMA’s journey with MCP and Amazon Bedrock

🔰 Solo founder runs a global tender platform on AlloyDB and MCP

🔰 Contrastive-LM: CLM-v0.1-8B ranks agent actions up to 13× faster

🔰 Google Cloud introduces Filestore agent volumes

🔰 AWS builds agentic conversational video intelligence

🔰 Google scales GNNs for autonomous networks with Distributed GraphFlow

🔰 Gemini 3.8 Flash TTS turns prompts into custom voices and performances

🔰 BigQuery augmented analytics brings causal analysis to SQL and AI agents

🔰 Dataflow adds Pause/Resume and NVIDIA RTX PRO 6000 Blackwell support

Let’s dive in.

Cheers,

Merlyn Shelley,

Growth Lead, Packt.

Designing Data Infrastructure for AI Agents & LLMs

📅 Saturday, September 26 | 9:30 AM–1:30 PM EDT
Design production-ready data infrastructure for AI agents & LLMs, from safe write paths and state management to provenance, tracing, recovery, and production architecture.

Join Sandipan Live

From Control Theory to Causal Inference: A Three‑Decade Journey That Led to My Book

Written by Yousri El Fattah

Some academic journeys begin with a clear plan. Mine began with a question that refused to leave me alone: How do we intervene in a system to achieve a desired effect? That question followed me across continents, across disciplines, and across decades — from aeronautical engineering in Cairo to stochastic adaptive control in Europe and Morocco, and eventually to causal inference and Bayesian networks in Southern California.

A Selection of Early Publications

A selection of early publications that shaped my work in stochastic adaptive control, learning automata, and AI during my years in Cairo, Europe, Morocco, and California.

Only in hindsight do I see the thread connecting it all. Every phase of my work — optimal control, learning automata, stochastic processes, modeling biological morphogenesis, market price formation, and communication flow control — was really an exploration of how systems behave, adapt, and respond when we act upon them. Causality was always present. I simply didn’t yet have the vocabulary for it.

The First Half: Learning to Control What We Don’t Fully Know

My early academic life was devoted to understanding how to control systems whose dynamics were uncertain or partially unknown. I focused on finite Markov chains and semi‑Markov processes, developing active, recursive learning mechanisms that unified control and estimation in real time.

This work introduced several ideas that today echo in reinforcement learning:

• Randomized control policies to ensure continual exploration

• Reward gradient formulas to quantify how policy changes affect long‑run performance

• Recursive stochastic gradient algorithms that adapt policies safely toward optimality

At the time, these were simply solutions to problems I found intellectually irresistible. Today, they resemble early versions of concepts that underpin modern adaptive systems.

The figure below illustrates block diagrams from my book Learning Systems, depicting supervised learning of Bayes’ rule, a game between multiple learning automata, a flow‑control system regulating server activity, and a learning automaton.

The Turning Point: Discovering the Language of Causality

Everything changed in 1986.

While teaching control systems and AI at California State University Long Beach, I attended a lecture by Judea Pearl on Bayesian networks. That single talk reframed my entire intellectual world. It connected the systems I had spent decades modeling with a new, powerful idea: causal structure.

I realized that the question I had been chasing — how do we intervene? — was the beating heart of causal inference.

From that moment on, my research, teaching, and industry work shifted toward causal modeling, Bayesian networks, and decision analysis. When Packt approached me in 2023 to write a book on causal inference with Bayesian networks, I knew it was time to synthesize everything I had learned across my eclectic academic journey.

What It Takes To Build Production-Ready LLM Applications

📅 Saturday, October 3 | 9:30 AM–1:30 PM EDT
Build production-ready LLM applications with DSPy, moving beyond manual prompt tweaking to structured evaluation, automated prompt optimization, failure analysis, and experiment tracking with MLflow

Build with Serj & Brett →

Three Years, Hundreds of Papers, and One Unifying Vision

Writing Causal Inference with Bayesian Networks took three years of deep reading, coding, experimentation, and careful structuring. My goal was ambitious:

Build a book that teaches causal inference not as abstract theory, but as a practical, computational discipline grounded in Bayesian networks, influence diagrams, and relational databases.

The book is organized into three parts:

1. Representation (Chapters 1–5)

Foundations of probability, Bayesian networks, structural causal models, and relational database representations.

2. Inference (Chapters 6–11)

Join tree clustering, probabilistic inference, causal identifiability, potential outcomes, and machine‑learning meta‑learners for heterogeneous treatment effects.

3. Applications (Chapters 12–14)

Real‑world case studies in economics, epidemiology, and social science.

One of the book’s most distinctive contributions is an algorithm that compiles influence diagrams into Query Composition Graphs (QCGs) — a reusable, SQL‑driven workflow for computing maximum expected utility (MEU). This bridges causal modeling with production‑grade decision systems.

About the Book

Causal Inference with Bayesian Networks Build Bayesian Networks and Causal Inference Models with R and Python by Yousri El Fattah and Reza Bagheri

This book provides:

• A unified introduction to Bayesian networks and structural causal models

• Hands‑on inference using R and Python

• Practical methods for causal identifiability and treatment‑effect estimation

• A novel SQL‑based workflow for MEU computation using QCG

• Real‑world applications in economics, epidemiology, and social science

Why I Wrote This Book

Because causal inference is no longer optional.

Whether you work in AI, data science, economics, epidemiology, or decision analysis, understanding why things happen — and how to intervene — has become essential. Bayesian networks and influence diagrams offer a structured, transparent way to reason under uncertainty.

My hope is that this book gives readers not just knowledge, but tools they can use immediately:

• Build Bayesian networks from scratch

• Perform probabilistic and causal inference

• Implement SCMs and meta‑learners

• Deploy causal models inside relational databases

• Compute MEU policies using QCGs

After more than two years of collaboration with my co‑author, Reza Bagheri, the book is now live on Amazon. It represents the culmination of my academic journey — and the beginning of yours.

This book is not just a technical guide — it is the culmination of a lifetime spent asking how systems respond when we intervene, and how understanding causality can help us make better decisions.

Book Link

👉 https://packt.link/QNELF

Design a production-ready RAG architecture for enterprise GenAI applications

📅 Saturday, October 31 | 9:30 AM–1:30 PM EDT
Design a production-ready RAG architecture for enterprise GenAI, covering ingestion, chunking, metadata enrichment, vector search, retrieval tuning, evaluation, governance, and cost control to build grounded, traceable, and reliable AI applications.

Join Brian Live

Data Science & ML Research Roundup

🔰From portal-hopping to instant answers: HEMA’s journey with MCP and Amazon Bedrock. HEMA had plenty of internal technical knowledge, but finding it was the problem. Engineers had to jump between wikis, service catalogs, documentation, and IT portals, sometimes spending hours looking for answers. HEMA built HAL, an internal AI assistant using Amazon Bedrock AgentCore and MCP, to bring that scattered knowledge together. Employees can now ask questions from tools they already use, including Kiro and Claude, and get governed answers in seconds. MCP provides a common interface connecting those AI tools to HEMA’s knowledge bases and internal APIs, while Microsoft Entra ID controls access. What started as a developer assistant is becoming a company-wide knowledge layer, with plans to eventually let HAL take actions as well as answer questions.

🔰Solo founder runs a global tender platform on AlloyDB and MCP: Lucius AI is showing how far a one-person company can scale with AI and managed infrastructure. The tender-intelligence platform tracks more than 210,000 public contracts across global markets, helping businesses analyze opportunities and prepare bids. Instead of running separate databases for structured data, logs, and vector search, founder Davor Jerković consolidated everything on Google Cloud AlloyDB for PostgreSQL. A ScaNN index cut semantic-search latency from 1.14 seconds to 24 milliseconds, a 47x improvement. More interestingly, an AI agent connected through MCP now helps handle database work such as performance analysis, freshness checks, and incident investigation. The result is a global data platform that can operate with remarkably little human overhead.

🔰Contrastive-LM/CLM-v0.1-8B: CLM-8B takes a different approach to AI decision-making: instead of generating an answer, it rapidly evaluates and ranks the possible actions you give it. Built on a frozen Qwen3-8B encoder with lightweight state and action heads, the model learns which action best matches a given situation. That makes it useful for tool selection, computer use, agent decisions, gaming, and verifying generated solutions. Because states and actions are encoded separately, actions can also be cached and reused, making CLM up to 13× faster when ranking around 1,000 candidates. Fine-tuned versions reached 81.6% on DeepSWE and 87.6% on Terminal-Bench 2.1, although those benchmark results do not apply to the base zero-shot checkpoint.

🔰Filestore agent volumes: Google Cloud is tackling a less obvious problem in the rise of AI agents: where all their working files and state actually live. Its new Filestore agent volumes give each AI agent an isolated, persistent workspace that can be automatically created and attached in milliseconds as new tasks begin. This means platforms running thousands or even millions of agent sessions no longer need to manually provision storage or keep expensive resources sitting idle. Agents can pause and resume with sub-second responsiveness, storage scales on a pay-per-use basis, and multiple agents can safely share the same files for collaborative workflows. The goal is to make large-scale coding agents, multi-agent systems, and long-running AI workflows easier and cheaper to operate.

🔰Agentic conversational video intelligence built on AWS: AWS is turning hours of video review into a conversation. Its agentic video-intelligence architecture lets users upload footage and simply ask questions such as “What decisions were made?” or “When did this person appear?” Instead of running every video through a fixed pipeline, an AI agent decides whether it needs Amazon Transcribe for speech, Rekognition for visual analysis, or both, then combines the results into an answer. New videos can take 5–10 minutes to analyze, but follow-up questions can return in under a second by reusing cached results. In one customer deployment involving more than 200 multi-hour recordings, the approach was associated with an approximately 80% reduction in manual review time, based on the customer’s own comparison.

🔰Run GNNs at Scale with Ease: Introducing Distributed GraphFlow: Google Cloud is applying Graph Neural Networks (GNNs) to the challenge of running telecom networks autonomously. Modern networks generate billions of interconnected data points, too much for human operators or AI agents to interpret directly. Google’s approach creates a real-time digital twin of the network in Spanner Graph, then uses its newly open-sourced Distributed Graph Flow (DGF) library to train GNNs on that graph data. These models can detect anomalies, trace the root cause of failures, predict equipment problems, and simulate events such as traffic surges or fiber cuts. AI agents can then use those predictions to understand what is happening and help decide or execute the appropriate response, moving telecom operations from reactive troubleshooting toward proactive, increasingly autonomous networks.

🔰Gemini 3.8 Flash TTS and Gemini 3.8 Flash-Lite TTS: Google is turning text-to-speech into something closer to a full voice-production studio with Gemini 3.8 Flash TTS and Flash-Lite TTS. Flash lets creators design entirely new voices using natural-language prompts, controlling characteristics such as accent, tone, pacing, dialect, and acting style across 100+ languages and dialects. It also offers 2,000+ production-ready voices and can replicate an authorized voice from a 30-second sample, with consent verification and watermarking safeguards. Flash-Lite targets cheaper, high-volume workloads such as dubbing and voice agents. Together, the models move TTS beyond simply reading text aloud toward directing characters and performances line by line.

🔰BigQuery Augmented analytics TVFs: BigQuery is moving beyond simply answering “what happened?” to helping explain “why did it happen?” Google has introduced six augmented analytics functions that run AI, ML, and statistical analysis directly inside BigQuery. They can automatically detect sudden changes in metrics, identify what drove them, measure the causal impact of a business decision, uncover correlations, separate long-term trends from noise, and find seasonal patterns. Because the results come back as structured SQL outputs, these capabilities can also become tools for AI agents, allowing an agent to investigate data conversationally and chain analyses together. For example, it could spot an unexpected revenue jump, identify the customers or products responsible, then estimate how much of that increase was actually caused by a pricing change.

🔰Use open weight models as your AI coding agent with Amazon Bedrock: Google is upgrading Dataflow for bigger, more demanding AI workloads, focusing on two practical problems: wasted compute and insufficient inference power. Pause/Resume, now generally available for batch jobs, lets teams pause long-running workloads or recover failed jobs without starting again from scratch, saving compute and engineering time. Dataflow also now supports NVIDIA RTX PRO 6000 Blackwell GPUs, with 96GB of GPU memory and 1.6 TB/s bandwidth, allowing teams to run inference with 70B+ parameter models directly inside Dataflow pipelines. Combined with Dataflow’s autoscaling and infrastructure management, the changes make large-scale AI data processing and inference more flexible, powerful, and cost-efficient.

🔰New Dataflow features to enable large scale AI workloads: Google is making Dataflow better suited to large AI workloads with two upgrades aimed at saving compute and increasing inference power. Pause/Resume lets teams stop long-running batch jobs and continue them later, including after failures, rather than restarting days of processing from scratch. Dataflow also now supports NVIDIA RTX PRO 6000 Blackwell GPUs, offering 96GB of GPU memory and 1.6 TB/s bandwidth. That allows teams to run inference with 70B+ parameter models directly inside their data pipelines, while Dataflow handles capabilities such as autoscaling and resource optimization. Together, the updates make it easier to run large-scale data preparation and AI inference without wasting expensive compute resources.

See you next time!

🤝 PARTNER SPOTLIGHT

Building a startup? Get in front of the right investors.

Start FREE. Pitch when you’re ready.

Fundraising shouldn’t mean weeks of cold outreach. On September 17, join the VC Pitch Conference and connect with a network of 290+ VCs and angel investors.

🎟️ Register for a FREE ticket →

Ready to pitch? Selected founders can get 20 guaranteed 1:1 investor meetings in one day, matched by sector, stage, and geography.

🚀 Pitch to 20 investors and save 10% with PACKT10 →

👋 Hi there, welcome to DataPro 186, where the best model isn’t the one that tops the benchmark. It’s the one that works in the real world.

Your benchmark says the model is ready. Production may disagree.

Before we dive in, a quick opportunity for founders in our community. Fundraising often means weeks of cold outreach without reaching the right investors. Through our partnership with VC Pitch Conference, selected founders can pitch to 20 matched investors in one day on September 17.

🎟️ Want to see if it is right for you? Register for a free ticket to explore the opportunity. Packt readers can also get 10% off with PACKT10.

Register for a free ticket

Now, onto the bigger question shaping this issue.

What does a great LLM benchmark score actually tell you about how that model will behave in production?

Potentially, much less than you think.

A model can ace standardized evaluations and still struggle when real users introduce unfamiliar prompts, edge cases, strict output formats, latency constraints, or entirely different data distributions.

In our expert conversation, Maxime Labonne, Head of Post-Training at Liquid AI, joins Sanjana Gupta, Relationship Lead at Packt, to unpack that gap between benchmark performance and production reliability.

Maxime explains why benchmarks should be treated as signals rather than verdicts, why teams should design evaluations before training, and when deterministic checks, model judges, and human evaluation each make sense. He also explores a counterintuitive shift worth watching: for narrow, well-defined tasks, a smaller fine-tuned model can outperform a much larger general-purpose model.

The practical takeaway for data and ML teams is simple: don’t optimize for the leaderboard. Build an evaluation system around the behavior your users actually need.

Let’s dive in.

Cheers,

Merlyn Shelley,

Growth Lead, Packt.

How Modern AI Systems Really Find Answers: Build GraphRAG Applications

📅 Saturday, September 19 | 09:30 AM–1:30 PM EDT

Move beyond vector search and build explainable AI with Neo4j, knowledge graphs, Cypher and LLM agents.

Join Alessandro Live →

Why Most LLM Benchmarks Fail in Production: Maxime Labonne on the Future of AI Evaluation

Maxime Labonne, Head of Post-Training at Liquid AI, explains why benchmark scores rarely tell the whole story, why evaluation should start before training, and how data teams can build LLM systems that survive contact with real users.

An LLM tops a benchmark.

It scores well on coding, reasoning, instruction following or general knowledge. The evaluation dashboard looks strong. The team ships it.

Then users arrive.

They phrase requests differently from the evaluation set. They introduce edge cases nobody anticipated. They expect outputs in particular schemas. They combine constraints. They send inputs that look nothing like the carefully prepared examples used during development.

And suddenly, the model that looked excellent on paper does not look quite so capable in production.

This gap between benchmark performance and production performance is becoming one of the most important engineering problems in applied AI.

In a conversation hosted by Sanjana Gupta, Relationship Lead at Packt, Maxime Labonne, Head of Post-Training at Liquid AI, explores why teams need to rethink how they evaluate language models, where automated evaluation works, why smaller specialized models can sometimes outperform much larger ones, and what ML teams should prioritize if they want more reliable AI systems.

Labonne’s argument is not that benchmarks are useless.

It is that a benchmark should be treated as a signal, not a verdict.

For data scientists, ML engineers and AI practitioners building systems for production, that distinction changes almost everything.

The benchmark says the model is better. Your users may disagree.

The traditional model-selection workflow feels reassuringly quantitative.

Run several candidate models against MMLU, HumanEval, a leaderboard or another standardized evaluation. Compare the numbers. Pick the model with the strongest result.

But Labonne points to an uncomfortable problem: the ranking produced by an evaluation does not necessarily reflect how people experience the models in practice.

A model can perform exceptionally well on a standardized test while another is preferred by developers for actual programming work.

That does not necessarily mean the benchmark is wrong.

It means the benchmark measures only part of the system’s behavior.

LLMs now operate across coding, extraction, reasoning, summarization, agents, RAG, tool use, structured generation and countless domain-specific tasks. Expecting one score to represent performance across that entire space is increasingly unrealistic.

Labonne therefore recommends treating evaluations as multiple signals rather than searching for one definitive number.

The same caution applies to human preference systems and automated judge models. Human evaluations introduce their own incentives and biases. LLM-as-a-judge systems can automate comparison at scale, but judge models also have biases.

There is no perfect evaluator.

For practitioners, the more useful question becomes:

What evidence would convince us that this model works for the specific job we are asking it to perform?

That question leads to a very different evaluation strategy.

Why good benchmark performance can collapse in production

One of the most dangerous evaluation failures happens before deployment.

Suppose your training and evaluation examples come from essentially the same distribution.

The evaluation set is technically separate. There is no obvious leakage. Your metrics look excellent.

But both datasets were generated using similar assumptions, similar pipelines or similar source material.

The model can become extremely good at the world represented by your test set without becoming equally good at the world represented by your users.

Labonne highlights contamination as an obvious version of this problem. If a model has effectively encountered the evaluation data during training, extraordinarily high benchmark scores deserve scrutiny.

But the problem can be subtler.

Even without direct contamination, training and evaluation distributions can resemble one another too closely.

Production users do not care about your data distribution.

They will use the system in ways you did not expect.

They will submit strange inputs, omit context, combine requirements, misunderstand instructions and discover failure modes that never appeared in the development environment.

A model can therefore pass an evaluation while the evaluation itself fails to represent reality.

That is why production evaluation should not simply ask:

Did the model answer our test questions correctly?

It should ask:

Does our test set resemble the situations this system will actually encounter?

Build Production-Ready AI Agents : From Workflow to Deployment

📅 Saturday, September 26–Sunday, September 27 | 10:00 AM–2:00 PM EDT
Build production-ready AI agents with n8n, from real-world workflows and tool integrations to testing, security, governance, and deployment.

Join Ashley Live →

Start with evaluation, not training

This is perhaps the most actionable idea from Labonne’s interview.

Many teams follow a familiar sequence:

collect data → train or fine-tune → evaluate → discover problems → rebuild

Labonne argues for reversing part of that process.

Define the evaluation first.

Consider a seemingly straightforward task such as structured data extraction.

You give the model an article or paragraph and expect JSON.

At first glance, the task sounds clear.

It is not.

What fields should the JSON contain?

Can objects be nested?

Which fields are optional?

Which data types are permitted?

How should missing values be represented?

What happens when the source contains contradictory information?

What should the model do when the requested information does not exist?

Before generating thousands of training examples, the team needs to define what success actually means.

Once those expectations become an evaluation dataset, something important happens: the evaluation becomes the specification for the system.

It tells engineers what to optimize.

It tells data teams what examples need to exist.

And it gives stakeholders something concrete to approve.

Labonne even recommends separating the processes used to produce training and evaluation data where possible. Different people, processes or code paths can reduce the risk of accidentally building an evaluation that merely reproduces the assumptions embedded in training.

The lesson for data teams is simple:

If your evaluation is built from the same assumptions as your training data, it may validate your pipeline rather than challenge your model.

Evaluation is also a stakeholder-alignment tool

There is another reason to build evaluations early that has little to do with model metrics.

It forces teams to agree on what the product is supposed to do.

Imagine showing a customer or product team an evaluation set before training:

Here are 100 situations we believe the model needs to handle. Here are the expected behaviors. Do these represent your use case?

Suddenly, abstract requirements become concrete.

Stakeholders can identify missing scenarios.

They can challenge assumptions.

They can decide whether a behavior is acceptable.

They can expose edge cases while changing them is still inexpensive.

Labonne argues that this is far easier at the beginning of a project than after the model has been trained and integrated.

For practitioners accustomed to software engineering, the idea resembles test-driven development.

But for LLM systems, the evaluation suite does more than test code.

It defines expected model behavior.

And once that definition exists, constructing the right training dataset becomes much easier because the team knows what it is actually optimizing for.

Automate what can be measured objectively

Building better evaluations does not mean manually reviewing every output.

Labonne strongly supports automated evaluation because it changes the economics of experimentation.

ML teams operate under a time budget.

If evaluating every model takes days of manual review, fewer models can be trained and fewer ideas can be tested.

Automating repeatable checks reduces that cycle time.

The saved time can then be spent where Labonne believes it matters most: experimentation and data quality.

Some failures are particularly suitable for automation.

A model falling into a repetition loop, for example, should not require a human reviewer to discover it every time. If an output repeats the same phrase thousands of times, an automated evaluator should catch it.

Likewise, many instruction-following requirements are deterministic.

If the instruction says:

Return five bullet points in Italian.

A program can check whether:

• the response contains five bullet points;

• the required format was followed;

• the output satisfies the language requirement.

The evaluator does not need another giant model to decide whether a bullet point exists.

This distinction is increasingly important as teams experiment with reinforcement learning with verifiable rewards (RLVR).

If a machine can verify the requirement, let it

Labonne contrasts verifiable rewards with approaches that depend heavily on learned reward models.

Consider an instruction:

“Answer using bullet points.”

A Python function can inspect the response and determine whether the requirement was satisfied.

Pass the check: reward.

Fail the check: no reward.

More complicated instructions can support partial credit.

If the model must produce bullet points and answer in Italian, satisfying one constraint but not the other could receive an intermediate reward.

This works particularly well for tasks with objectively testable outcomes:

Instruction following. Mathematical answers. Structured output. Formatting constraints. Function calls. Schema compliance.

For data practitioners, the implication is significant.

Before reaching for another LLM to evaluate an LLM, ask:

Can this requirement be expressed as code?

If yes, deterministic verification may be cheaper, faster and easier to debug.

Designing Data Infrastructure for AI Agents & LLMs

📅 Saturday, September 26 | 9:30 AM–1:30 PM EDT
Design production-ready data infrastructure for AI agents & LLMs, from safe write paths and state management to provenance, tracing, recovery, and production architecture.

Join Sandipan Live →

But not everything valuable is verifiable

Now ask the model to write a good poem.

A Python function can check whether the response contains four stanzas.

It cannot reliably tell you whether the poem is moving.

This is the boundary of purely deterministic evaluation.

Open-ended tasks such as writing quality, tone, creativity and some forms of reasoning cannot always be reduced to binary checks.

In those situations, Labonne suggests bringing reward or judge models back into the evaluation architecture.

The useful pattern is therefore not:

deterministic evaluation versus model-based evaluation.

It is:

use deterministic evaluation wherever the requirement is objectively verifiable, and model or human judgment where it is not.

A mature evaluation system can combine all three.

That gives practitioners a useful hierarchy:

Code where possible. Models where necessary. Humans where judgment genuinely matters.

The evaluation suite should evolve with the model

Evaluation is not something a team finishes.

New models reveal new failure modes.

Users uncover behaviors the development team never anticipated.

Labonne gives repetition or “doom loops” as an example: a model may start repeating the same words or sentences endlessly. Once a team discovers a failure like this, it should become part of the evaluation suite.

That creates an important engineering cycle:

discover failure → encode failure → test automatically → prevent regression

Over time, the evaluation harness becomes a record of what the team has learned about the system.

This is much closer to production software testing than leaderboard chasing.

The best evaluation suite is therefore not necessarily the one containing the most famous benchmarks.

It is the one that increasingly reflects your model, your application and your users.

Small models change the economics of production AI

The conversation also moves beyond evaluation into another shift that data practitioners should be watching: the growing usefulness of small language models.

At Liquid AI, Labonne works with models ranging from hundreds of millions to several billion parameters, including models designed to run locally on phones, laptops and other edge hardware.

Why does that matter?

Because cloud-scale LLMs introduce constraints that benchmark leaderboards rarely capture:

Cost. Latency. Connectivity. Privacy. Hardware requirements.

A frontier cloud model might produce the strongest general-purpose answer.

But what happens when the application needs to operate without internet connectivity?

What happens when sensitive information cannot leave the device?

What happens when latency needs to be extremely low?

Or when an agentic workflow consumes enormous numbers of tokens?

At that point, “best model” stops being a meaningful question without context.

The real question becomes:

Best model for what operational constraint?

A smaller model can beat a larger model when the task becomes narrow

Labonne describes a particularly revealing example involving in-car function calling.

A client required a very specific function-calling format.

A large general-purpose model struggled with the task.

A much smaller model was fine-tuned specifically on examples matching that format.

According to Labonne, the specialized model achieved over 90% accuracy, compared with roughly 10–20% for the GPT-4 system tested at the time.

The point is not that small models are universally better.

Labonne explicitly notes that general-purpose tasks become harder for smaller models.

The lesson is about specialization.

When the problem is sufficiently narrow, a small model can be trained directly around the required behavior.

For practitioners, that suggests another reason why application-specific evaluation matters.

A generic benchmark might strongly favor the larger model.

Your private evaluation might strongly favor the smaller one.

And if the smaller model also delivers lower latency, local inference and dramatically lower inference cost, the production decision can look very different from the leaderboard.

Agentic AI makes model economics impossible to ignore

This trade-off becomes particularly important for agents.

Agentic systems can be extremely token hungry.

A conventional chatbot may make one model request and return one answer.

An agent can reason, call a tool, inspect the result, generate another request, call another tool, revise its plan and repeat the process.

Token consumption compounds.

Labonne expects this to strengthen the case for local and smaller models, particularly for tasks where frontier-model quality is unnecessary.

That does not mean cloud models disappear.

Instead, production architectures may become more selective.

Use a specialized local model for predictable, repetitive or narrow operations.

Escalate to a larger cloud model when the problem actually requires the additional capability.

For data and AI teams, model routing may therefore become as important as model selection.

The future of LLM evaluation is probably not one benchmark

So what happens as models become more capable?

Labonne expects evaluation to move away from dependence on a handful of dominant benchmarks toward many specialized signals.

This is already happening.

Researchers and practitioners are building evaluations for increasingly narrow domains and behaviors.

Individually, each benchmark sees only a small part of the model.

Collectively, they can provide a much richer picture.

This matters because improvements between successive frontier models may not always appear as dramatic gains on everyday tasks. Instead, Labonne argues that much of the progress comes from models becoming competent across an expanding range of domains.

The capability surface gets wider.

Evaluation therefore has to get wider too.

The interesting infrastructure opportunity is an aggregation layer capable of combining many heterogeneous signals into a more useful picture of model behavior.

Until then, practitioners should resist compressing model quality into one leaderboard position.

What should an ML team do in the next 30 days?

Labonne’s recommendation is surprisingly simple:

Start with evaluations.

Not after the next fine-tuning run.

Not immediately before launch.

Now.

For a team building an LLM application, a practical 30-day version of that advice could look like this:

Week 1: Define production behavior

Write down the tasks the system actually needs to perform. Include output formats, constraints, edge cases and unacceptable failures.

Week 2: Build a private evaluation set

Create examples representing realistic user behavior rather than merely reproducing the training distribution. Separate evaluation-data creation from training-data creation wherever practical.

Week 3: Automate the obvious

Turn deterministic requirements into executable checks: schema validation, formatting, function calls, numerical correctness, repetition detection and instruction compliance.

Use model-based or human evaluation for qualities that cannot be verified reliably with code.

Week 4: Connect evaluation to experimentation

Run every meaningful model, prompt, fine-tuning or data change through the same harness.

When production reveals a new failure, add it to the suite.

The goal is not a perfect benchmark.

It is an evaluation system that becomes harder for your model to fool and increasingly representative of what users actually need.

If the budget is limited, invest in data and evaluation

When asked where a team with limited resources should invest first, Labonne returns to the fundamentals.

Data quality is one of the strongest drivers of model performance.

But data alone is not enough.

Without evaluation, you cannot reliably determine whether better data actually produced a better system.

Training itself is becoming increasingly automated. Tooling can handle much of the mechanical work.

The harder questions remain human ones:

What data represents the problem?

What behavior counts as success?

What failures matter?

What should be measured?

And does the evaluation actually resemble production?

That makes data and evaluation two sides of the same engineering problem.

Stop asking which model is best

Perhaps the most useful takeaway from the conversation is a change in the question practitioners ask.

Instead of:

Which LLM has the highest benchmark score?

Ask:

Which model performs reliably on the distribution, constraints and failure modes that matter to this application?

That change sounds small.

In practice, it affects model selection, dataset design, fine-tuning, evaluation infrastructure, deployment architecture and cost.

Public benchmarks remain useful. Human preference remains useful. LLM judges remain useful. Private prompts and “vibe checks” remain useful too. Labonne himself says that before shipping, one of his final signals is a private collection of prompts designed to probe different model capabilities.

None of these signals needs to become the truth.

They become more valuable when they are combined.

And the closer evaluation gets to the messy reality of production, the more useful it becomes.

Because the hardest test for an LLM is not MMLU, HumanEval or a leaderboard.

It is what happens after real users start typing.

See you next time!

At petabyte scale, egress and storage costs force teams to trim data before it reaches Databricks. AIStor skips the copy entirely, so nothing has to be cut. Read the technical guide for data and infrastructure leads.

Read the Guide

👋Hi there, welcome to DataPro 185, where better outcomes begin by solving the right problem.

For startups, that problem is often access. Fundraising can mean weeks of cold outreach with no guarantee of reaching the right investors. Through our partnership with VC Pitch Conference, selected founders can pitch to 20 matched investors in one day on September 17, with Packt readers receiving an additional 10% off using code PACKT10.

For data teams, the challenge is trust. Organizations can invest heavily in Power BI, Microsoft Fabric and modern data platforms, yet still face unreliable reports, conflicting metrics and users returning to spreadsheets. The technology often gets the blame, but the real problem may lie in poor data quality, weak governance or analytics disconnected from genuine business decisions.

Leading this edition, Microsoft Fabric and Power BI consultant Robert Hawker joins Apeksha Shetty, Portfolio Manager at Packt, to explore how organizations can build analytics people actually trust. Robert explains why governance must begin with visible business pain, how one organization raised data quality from 73% to 93%, and why trusted self-service requires centralized data engineering, governed semantic models and flexibility for trained business users.

That same shift from fragile processes to reliable systems connects the stories across this edition. From reducing Spark provisioning failures and controlling agent memory to automating Spanner migrations and building Physical AI pipelines, the focus is clear: creating data and AI systems that remain efficient, dependable and ready to scale.

This Week’s Highlights

• Yahoo cuts Apache Spark provisioning failures by 85% using flexible VMs and regional auto-zone placement.

• Reducto’s r-1 cuts document-parsing errors by 20% with a single-pass model priced at one cent per page.

• AWS introduces lifecycle policies for AgentCore memory to score, consolidate and prune outdated agent context.

• AXIS releases 50,129 robot-manipulation trajectories collected through a browser-based data engine spanning 207 tasks.

• AWS builds a multimodal WhatsApp ordering assistant that connects text, voice notes and calls through one shared memory.

• Google automates a Spanner migration across more than 30 DAOs using Antigravity CLI, dual writes and automated parity checks.

• Meta FAIR ranks ML experiments before spending GPU hours with Research Preference Models that select from 15 candidates.

• Google launches its Gen AI SDK for Kotlin 1.0, bringing Gemini to JVM and Android through Kotlin Multiplatform.

• AWS and NVIDIA build a Physical AI model factory using Cosmos 3, SageMaker HyperPod and a shared GPU pool.

👉 Pitch your startup to investors →

Let’s dive in.

Cheers,

Merlyn Shelley,

Growth Lead, Packt.

Live LLM Engineering Masterclass

📅 Saturday, September 12 | 09:30 AM–1:30 PM EDT

Build reliable LLM systems with production evals, RAG, resilient agents, regression testing, observability and cost control.

Save 45% with PRODLLM45

Your Power BI Problem May Not Be Power BI

A company can invest heavily in Microsoft Fabric and Power BI, migrate data, build semantic models, publish dashboards, and train users, yet still watch employees return to their Excel trackers.

It is tempting to call this resistance to change. Often, it is something more fundamental: the analytics environment has not earned users’ trust.

Reports may be slow, incomplete, duplicated, or disconnected from the decisions people actually need to make. Governance may arrive too late, after conflicting metrics have already spread, or too forcefully, turning every improvement into a lengthy approval process. In both cases, the technology is visible, but the business value is not.

In a conversation hosted by Apeksha Shetty, Portfolio Manager at Packt, Microsoft Fabric and Power BI consultant, trainer, author, and experienced data and analytics leader Robert Hawker shared a more practical way to approach the problem. His central argument is that successful analytics transformation does not begin with a platform feature. It begins with the outcome an organization needs, the data problems blocking it, and a governance model designed to produce lasting improvement.

How Modern AI Systems Really Find Answers: Build GraphRAG Applications

📅 Saturday, September 19 | 09:30 AM–1:30 PM EDT

Move beyond vector search and build explainable AI with Neo4j, knowledge graphs, Cypher and LLM agents.

Save 35% with MODERNAI35

Start governance where the business already feels the pain

There is no universal first step for enterprise data governance. The right starting point depends on the organization’s industry, strategy, and immediate pressures.

In financial services, a regulatory return may set the priority. During a major transformation, attention may need to shift to new processes, new systems, and the quality of migrated data. Sometimes the trigger is an operational emergency, such as suppliers threatening action because data failures have prevented timely payment.

When no urgent driver is obvious, Hawker recommends starting with the organization’s strategy. Ask leaders what is preventing them from delivering their objectives, without limiting the discussion to issues already labelled as data problems. A deeper analysis often reveals that apparently operational or commercial blockers have data at their root.

That connection matters. Stakeholders are more likely to support governance when they can see how it improves the probability of achieving an outcome they already care about.

Prioritization can then follow a familiar benefit-versus-effort assessment. Quantitative and qualitative value should be weighed against the cost and complexity of resolving each issue, with high-benefit, lower-effort opportunities rising to the top. But the calculation is not purely mechanical. An early initiative backed by a supportive stakeholder can create proof, credibility, and an internal advocate for the wider programme.

The important distinction is between a quick fix and a quick win. Hawker recalls a proposal to pay a supplier for a one-off cleanse of supplier data using generic industry rules. Instead, he redirected the investment into the foundations of a repeatable data quality process and a tool containing rules specific to the organization. The initiative delivered near-term improvement while building a capability that could sustain it.

The lesson is simple: even early wins should point towards the long-term operating model.

Build Production-Ready AI Agents : From Workflow to Deployment

📅 Saturday, September 26–Sunday, September 27 | 10:00 AM–2:00 PM EDT
Build production-ready AI agents with n8n, from real-world workflows and tool integrations to testing, security, governance, and deployment.

Save 35% with PACKTAI35

When dashboards are ignored, diagnose the experience before blaming the users

Low Power BI adoption can look like a technology problem, a data problem, a governance failure, or a cultural issue. Frequently, several are present at once.

Hawker says the most common cause is a gap between report developers and business users. Developers understand the data and may have built the semantic model, but they are not close enough to the operational decision to create a useful product. Business users understand what the analysis needs to accomplish, yet may lack access to governed data or the Power BI skills required to build it themselves.

The diagnosis should therefore begin with wide-ranging conversations, especially with the people whose roles should benefit most from the reporting.

What users say provides clues about where to look:

“Reports are slow or out of date.” Poor semantic modelling, many-to-many relationships, failed refreshes, insufficient operational oversight, or a throttled Fabric capacity may be involved. What appears to be a performance complaint can expose gaps in skills and governance.

“The figures are wrong or important data is missing.” The source data may be unreliable, but governance is also implicated. Releasing a report should include an explicit assessment of whether its data quality is known and adequate for the intended decision.

“There are five reports for the same topic, and I do not know which one to trust.” This is a sign of weak analytical governance. Decentralized development without shared definitions and endorsement creates competing versions of the truth.

“My Excel tracker is more reliable.” This may indicate a cultural barrier, but dismissing the user would be a mistake. If an established process provides dependable information, the new solution must offer a compelling improvement, such as a better outcome, stronger security, or less manual work.

Adoption is not achieved by asking people to stop using familiar tools. It is achieved by giving them something demonstrably more useful and trustworthy.

Data Science & ML Research Roundup

⭕ How Yahoo Optimizes Apache Spark with Flexible VMs: Yahoo cut Apache Spark cluster provisioning failures by 85% using flexible VMs in Google Cloud’s Managed Service for Apache Spark. Instead of relying on one machine type, clusters rank compatible VM shapes and use auto-zone placement to find regional capacity automatically. Matching cores, memory and CPU-to-memory ratios preserves autoscaling performance, while explicit Spark and YARN settings keep workloads predictable and reliable

⭕ Reducto Releases r-1: A Single Pass Document Parsing Model That Cuts Errors 20% at 1 Cent Per Page. Reducto’s r-1 replaces multi-stage agentic OCR with a single full-page pass, extracting text, tables, figures, layout, formatting and bounding boxes together. The preview reportedly cuts errors 20% and costs one cent per page, up to six times cheaper. It runs through the hosted V3 API, while advanced chart extraction and custom prompts still require slower agentic processing. Benchmarks are vendor-reported.

⭕ Designing lifecycle policies for AgentCore memory: AWS proposes treating agent memory as a managed resource before stale context damages responses or compliance. A nightly AgentCore workflow built with Step Functions and Bedrock classifies episodic, semantic and procedural memories, then applies TTL expiry, relevance-decay scoring and consolidation or pruning. Configurable thresholds balance recency, access and frequency, while the CDK stack and GitHub code offer a deployable blueprint.

⭕ Axis Robotics Releases AXIS: A Browser-Based Data Engine With 207 Robot Manipulation Tasks and 50,129 Trajectories. Axis Robotics’ AXIS turns robot demonstration collection into a browser-based, continuously expanding pipeline. Contributors teleoperate Franka robots while backend GPUs generate tasks, validate success, clean trajectories and randomize scenes. The release spans 207 tasks, 50,129 episodes and 2.36 TB. AXIS-pretrained π0.5 reached 88.8 on LIBERO-Plus versus 83.9 baseline, although replay success fell to 86.2%, with academic-only data and no checkpoints.

⭕ Deploy a multimodal WhatsApp ordering assistant with Amazon Bedrock AgentCore: AWS shows how restaurants can run one multimodal WhatsApp assistant across text, voice notes and calls. Bedrock AgentCore hosts isolated channel agents, Nova 2 Lite and Sonic handle language and speech, and shared memory preserves customer context. An asynchronous API Gateway, Lambda and SQS ingress connects through MCP to DynamoDB-backed ordering tools, while AWS CDK deploys the complete backend stack.

⭕ Using Antigravity CLI to streamline dual-write database migration: Google’s Finance Engineering team used Antigravity CLI to automate a high-risk Spanner migration across 30+ DAOs. A standardized MutationConverter separated schema translation from business logic, enabling headless agents to generate dual-write code and unit tests. Build failures looped back for self-correction, allowing teams to queue ten DAOs overnight while backfills, dual reads and writes, and byte-for-byte parity checks minimized disruption.

⭕ Meta FAIR Introduces AI Research Preference Models (RPMs): Ranking ML Experiments Before Spending GPU Hours. Meta FAIR’s Research Preference Models help AI research agents choose which ML experiments deserve scarce GPU time. Instead of predicting absolute results, RPMs rank 15 unexecuted candidates through pairwise tournaments and run only the winner. One variant judges plans and code; another uses an H200 sandbox for pilot experiments. AIRA-dojo, AIRS-Bench and Qwen3.6-27B are open, though deployment remains partial today.

⭕ Announcing the Google Gen AI SDK for Kotlin 1.0: Idiomatic multiplatform access to Gemini: Google’s Gen AI SDK for Kotlin 1.0 brings native Gemini development to JVM and Android through Kotlin Multiplatform. It replaces raw HTTP clients and Java bridging with Coroutines, Flow streaming and immutable data classes. One client supports Gemini Developer and Enterprise APIs, multi-turn chat, function calling and multimodal analysis with Google Search grounding. The open-source package is on Maven Central.

⭕ Build a Physical AI model factory with NVIDIA Cosmos 3 on SageMaker HyperPod. AWS and NVIDIA show how Cosmos 3 on SageMaker HyperPod can power a continuous Physical AI model factory. Its Mixture-of-Transformers architecture unifies video, images, sound and actions, supporting synthetic world generation, inverse-dynamics labelling and robot policies. HyperPod and EKS schedule these stages across one resilient GPU pool, improving goodput per reserved GPU-hour. The repository includes templates and runnable DROID workflows.

See you next time!

Import a share into Unity Catalog and query on-prem data in place—no copying, no downsampling, results back in seconds. Watch the 7-minute walkthrough of MinIO AIStor Table Sharing connected live to Databricks.

Watch the Demo

👋 Hi there, welcome to DataPro #184 – The AI Breakthrough Is Easy. Proving It Works Is Harder.

AI teams are getting remarkably good at building things that look successful.

The harder question is whether those systems are actually improving the user experience.

We lead this issue with Lorenzo Toni, Lead Data Scientist, AI Product Analytics at Amazon, in conversation with Nilesh Kowadkar, Relationship Lead at Packt, on one of the most overlooked problems in AI product development: how do you measure whether an AI feature is genuinely working?

Using a generalized e-commerce chatbot experiment, Lorenzo shows why the obvious signals can be misleading. Engagement increased and conversations became longer, yet conversion eventually declined. The lesson is bigger than one chatbot: AI experimentation cannot stop at surface-level metrics. Teams need to understand downstream behaviour, account for the inherent randomness of generative AI, and use causal inference alongside experimentation to separate correlation from genuine product impact.

That same shift from impressive AI capabilities to measurable, production-ready systems runs through the rest of the edition.

What Else We’re Tracking

• Google Cloud moves enterprise AI closer to production, with Gemini Enterprise for Financial Services and Legal, tighter agent cost controls, expanded Antigravity access, and continued work on hallucinations and token economics.

• AWS rethinks support operations with GenAI, combining automated video-to-SOP creation, RAG, agentic workflows and ML-based SLA risk prediction to turn fragmented operational knowledge into actionable intelligence.

• Google’s TabFM brings zero-shot predictive ML to BigQuery, using in-context learning to perform classification and regression through SQL without the traditional train, tune and deploy cycle.

• AWS tackles the monitoring problem infrastructure checks cannot see: dashboards that load successfully but contain blank visuals or incorrect numbers. Its Bedrock-powered validation system reduced detection time from as much as 72 hours to under one hour.

• BigQuery Graph is now generally available, bringing graph analytics, GQL, SQL, ML and AI together to uncover connected relationships and provide richer grounding for AI agents and GraphRAG.

• Perplexity open-sources Lily, a purpose-built Rust + Metal inference engine that pushes Qwen3.6-35B-A3B performance on Apple Silicon beyond MLX-LM in its reported benchmarks.

• Code-as-World asks whether AI can model physics, not just pixels, turning real videos into executable MuJoCo worlds through an agentic propose-and-verify loop.

From measuring AI impact and controlling agent costs to predictive ML, graph-grounded agents, local inference, and executable world models, this issue looks beyond what AI can generate to ask the harder engineering question:

Can we make AI measurable, reliable, efficient, and genuinely useful in the real world?

Let’s get into it!

Cheers,

Merlyn Shelley,

Growth Lead, Packt.

Live LLM Engineering Masterclass

📅 Saturday, September 12 | 09:30 AM–1:30 PM EDT

Build reliable LLM systems with production evals, RAG, resilient agents, regression testing, observability and cost control.

Save 45% with PRODLLM45

Your AI Experiment Says the Feature Is Working. Your Users May Be Telling You Otherwise

An AI feature can pass an experiment and still make the product worse.

Engagement can rise. Session duration can increase. Users can interact with the feature more frequently. An A/B test can even produce statistically convincing results.

And yet, a few weeks later, conversion starts falling.

Users stop returning.

Trust begins to erode.

That is the measurement problem AI product teams increasingly need to confront.

Traditional digital products generally give users a relatively deterministic experience. Change a button, ranking algorithm, checkout flow, or interface and teams can establish a reasonably clean treatment and control.

Generative AI changes that assumption.

The same feature can produce different answers for different users, and even different answers for the same user. A seemingly insignificant prompt adjustment can alter the length, tone, relevance, accuracy, and usefulness of an interaction.

So what exactly should teams measure?

And more importantly, how do you know whether an AI feature actually caused the outcome you are seeing?

That question was at the centre of a conversation between Nilesh Kowadkar, Relationship Lead at Packt, and Lorenzo Toni, Lead Data Scientist — AI Product Analytics at Amazon, focused on experimentation and causal inference for AI products. Drawing on a generalized e-commerce case study, Lorenzo unpacked why apparently positive metrics can be misleading and why AI teams need to look beyond conventional A/B testing.

Partner Spotlight | Building a startup? Pitch live to 20 investors in one day.

Fundraising shouldn’t mean spending weeks cold-emailing VCs and waiting for replies.

We’re partnering with VC Pitch Conference to give founders a more direct route to investors. On September 17, selected startups can join 20 guaranteed 1:1 investor calls in a single day, with meetings matched by sector, stage, and geography.

With 290+ VCs and angel investors looking across AI, fintech, B2B, and other sectors, it’s an opportunity to put your startup directly in front of investors actively looking for their next deal.

Packt readers get an additional 10% off the early-bird ticket with code PACKT10.

Pitch Your Startup to Investors →

When More Engagement Is Actually a Warning Sign

Consider an e-commerce platform introducing an AI chatbot designed to help customers discover products.

Instead of navigating through search results, users could ask questions, compare prices, describe what they wanted, and receive recommendations conversationally.

A change was made to the prompt powering the chatbot.

Initially, the numbers looked promising.

Engagement increased. Users performed more actions. They also spent longer interacting with the chatbot.

Under a conventional product analytics framework, those signals could easily be interpreted as evidence that the new experience was working.

Then, roughly two weeks later, another pattern emerged.

Chatbot interactions began declining.

More importantly, overall website conversion started falling.

The metric that initially appeared particularly encouraging, time spent with the chatbot, turned out to be negatively correlated with conversion.

Users were not necessarily spending longer because the experience was valuable.

Some were spending longer because the chatbot was taking too long to help them reach a decision.

A ten-minute conversation might look like excellent engagement on a dashboard. To the customer, it might mean:

I could have found this myself in thirty seconds.

That distinction matters.

Because when AI introduces friction instead of removing it, the damage may extend beyond a single failed interaction. Users learn from that experience. The next time they need something, they may bypass the chatbot entirely.

The apparent engagement win can therefore become a trust problem.

How Modern AI Systems Really Find Answers: Build GraphRAG Applications

📅 Saturday, September 19 | 09:30 AM–1:30 PM EDT

Move beyond vector search and build explainable AI with Neo4j, knowledge graphs, Cypher and LLM agents.

Save 35% with MODERNAI35

The Real Problem Wasn’t the Chatbot. It Was What the Chatbot Was Optimizing For.

Digging deeper revealed another issue.

The chatbot was largely recommending top-ranked, popular products.

At first glance, that seems perfectly rational.

Popular products already convert well. Recommend them more frequently and customers should be more likely to buy them.

Except that assumption ignores why someone chooses an AI assistant instead of ordinary search.

A shopper typing pen into a search box probably expects popular pens.

Someone opening an AI chatbot may instead ask:

What is the best gel pen for writing on rough paper?

That is a fundamentally different intent.

The value of an LLM-powered shopping experience comes from its ability to understand those more specific needs and personalize the discovery process, rather than simply reproduce the first page of search results.

This changed the experimentation question.

Instead of asking:

Does the new prompt increase chatbot engagement?

The team needed to ask:

What kind of conversational behaviour actually helps users make better decisions?

Experimenting With Different AI Behaviours

Lorenzo worked with engineering and data teams to test alternative experiences.

The existing prompt became the baseline.

Another version made the chatbot more inquisitive, encouraging it to gather more information about the customer’s needs before recommending something.

A further approach explored what Lorenzo described as alternative “gems”: products that might not dominate the rankings but had strong reviews, good conversion performance, and a closer fit with a customer’s specific requirements.

The experiments also extended across regions using a switchback approach.

Rather than permanently assigning one experience to a group, different versions could be switched on and off across time periods. A user interacting with the system at one point might encounter one LLM behaviour and later encounter another.

This provided another way of understanding how changes to the AI experience affected behaviour while accounting for the dynamic nature of the system.

But product quality was only one side of the equation.

A more sophisticated recommendation experience could require access to a much larger product catalogue and richer context. That means additional memory, infrastructure, retrieval complexity, and ultimately cost.

The winning experiment therefore could not simply be the one with the highest conversion rate.

It had to be statistically meaningful, commercially valuable, and operationally practical.

Build Production-Ready AI Agents : From Workflow to Deployment

📅 Saturday, September 26–Sunday, September 27 | 10:00 AM–2:00 PM EDT
Build production-ready AI agents with n8n, from real-world workflows and tool integrations to testing, security, governance, and deployment.

Save 35% with PACKTAI35

Data Science & ML Research Roundup

⭕ What Google Cloud announced in AI this month – and how it helps you. Google Cloud is pushing AI from experimentation into practical enterprise use. August brought Gemini Enterprise for Financial Services and Legal, stronger agent billing and cost controls, and expanded Antigravity access. Alongside this, Google is tackling two stubborn challenges: AI hallucinations and runaway token costs, while helping teams move prototypes into production and start building agents from the ground up.

⭕ Modernizing and scaling support operations with generative AI on AWS: Support teams are drowning in rising ticket volumes, fragmented knowledge, outdated SOPs, and looming SLA breaches. AWS proposes a GenAI-powered operating model that turns training videos into SOPs, uses RAG and agentic workflows to guide ticket resolution, maps process bottlenecks, and applies ML to predict SLA risk, creating a continuous loop where every resolution makes the next one smarter.

⭕ TabFM adds predictive ML to BigQuery: Predictive ML in BigQuery is getting a major shortcut. TabFM, Google’s new tabular foundation model, uses in-context learning to deliver zero-shot classification and regression with a single SQL query. It removes model training, tuning, deployment, and manual feature engineering while scaling predictions across millions of rows, bringing fast, high-quality predictive analytics directly into BigQuery and agentic workflows.

⭕ How an AWS team detects dashboard content failures at scale using Amazon Bedrock: A dashboard can be technically healthy and still show blank charts or wrong numbers. AWS closed this last-mile monitoring gap with an Amazon Bedrock-powered validation system that visually scans hundreds of dashboards, checks numeric consistency, and alerts owners automatically. By combining LLMs for semantic detection with deterministic logic for precision, the team cut failure detection from up to 72 hours to under one hour.

⭕ BigQuery Graph: Connecting Data and AI at Scale: Enterprise data becomes more valuable when you can understand how everything connects. Now generally available, BigQuery Graph brings native graph analytics directly into the warehouse, combining GQL, SQL, ML and AI without ETL. Teams can uncover fraud networks, model supply chains, resolve identities, and build knowledge graphs that ground AI agents and GraphRAG, even across multi-cloud data without moving it.

⭕ Perplexity Open Sources Lily: A Rust + Metal Inference Engine for Qwen3.6-35B-A3B on Apple Silicon. Running a 35B-parameter model locally on a Mac usually means compromising on speed or relying on general-purpose frameworks. Perplexity’s newly open-sourced Lily takes another route: a specialized Rust + Metal inference engine for Qwen3.6-35B-A3B on Apple Silicon. By optimizing GPU routing, dequantization and attention, Lily delivers 1.23x faster prefill and 1.35x faster decoding than MLX-LM.

⭕ Meet ‘Code-as-World’: An Agentic Loop That Rewrites Real Videos Into Executable MuJoCo Physics Programs. Video models can recreate what a scene looks like, but do they understand how the physical world actually works? Code-as-World takes a different approach, converting real videos into editable MuJoCo physics programs. An agentic propose-and-verify loop reconstructs mass, motion, collisions and gravity, creating verified worlds for training. Its 9B model even outperformed Gemini 3.1 Flash on QuantiPhy.

See you next time!

Stop forcing your data into rigid tables. MongoDB Atlas is the developer data platform built for modern analytics — flexible documents, multi-cloud on AWS, and Atlas Search out of the box. Free tier available on AWS Marketplace.

Start for Free

👋 Hi there, welcome to DataPro #181 – Building the infrastructure behind production AI

AI is entering a new engineering phase. The focus is no longer just on building smarter models, but on creating the infrastructure, storage, and operational systems that make AI reliable at scale. This edition explores the technologies powering production AI, from next-generation storage architectures and autonomous database operations to coding agents, multi-agent systems, and cloud-native AI platforms.

🌟 Community Spotlight

AI Data Storage: Challenges, Capabilities, and Comparative Analysis

As AI workloads continue to scale, storage is emerging as one of the biggest infrastructure bottlenecks. In this deep dive, JuiceFS explores why traditional storage architectures struggle with GPU-intensive training, billions of small files, and high-throughput data access, before comparing the capabilities modern AI storage platforms need to support today’s data pipelines.

🎙️ Expert Perspective

The Reliability Gap Holding AI Agents Back

What separates an impressive AI demo from a production-ready AI system? AI researcher and Developer Advocate Sonam Gupta shares the engineering practices behind building reliable AI agents, explaining why evaluation, observability, guardrails, and system design matter far more than model benchmarks when deploying AI at scale.

⚡ This Week’s Highlights

• Behind the scenes of Google Agent Skills

• Prime Agent introduces a self-improving coding harness

• How LendingTree built a production multi-agent mortgage assistant

• Google’s latest AI infrastructure updates

• Meta Muse Code enters beta

• Mobileye automates enterprise support with AgentCore

• Google’s Borderless Lakehouse expands multi-cloud AI

• AWS explains its MCP bridge architecture

• NVIDIA launches Alpamayo 2 Super for autonomous driving

• Google introduces Database Operations Agents

Happy reading!

Cheers,

Merlyn Shelley,

Growth Lead, Packt.

AI Data Storage: Challenges, Capabilities, and Comparative Analysis

Over the past five years, AI has advanced rapidly and has found applications in a wide range of industries. As a storage company, we’ve had a front-row seat to this expansion, watching more and more AI startups and established players emerge across fields like autonomous driving, computational biology, and quantitative investment. AI workloads have introduced new challenges to the field of data storage. Existing storage solutions are often inadequate to fully meet these demands.

In this article, we’ll deep dive into the storage challenges posed by AI workloads, the critical capabilities required to address them, and a comparative analysis of the leading storage solutions spanning cloud and on-premises environments.

Storage challenges for AI

AI workloads have brought new data patterns:

• High-throughput data access challenges: In AI training pipelines, the growing use of GPUs by enterprises has outpaced the I/O capabilities of underlying storage systems. Enterprises require storage solutions that can provide high-throughput data access to fully leverage the computing power of GPUs. For instance, in smart manufacturing, where high-precision cameras capture images for defect detection models, the training dataset may consist of only 10,000 to 20,000 high-resolution images. However, each image is several gigabytes in size, resulting in a total dataset size in the scale of tens or hundreds of terabytes. If the storage system lacks the required throughput, it becomes a bottleneck during GPU-intensive AI training.

• Managing storage for billions of files: AI use cases need storage solutions that can handle and provide quick access to datasets containing billions of files. For example, in autonomous driving, the training dataset consists of small images, each about several hundred kilobytes in size. A single training set comprises tens of millions of such images, each sized several hundred kilobytes. Each image is treated as an individual file. The total training data amounts to billions or even 10 billion files. This creates a major challenge in effectively managing large numbers of small files.

• Scalable performance for hot data: In areas like quantitative investing, financial market data is much smaller in size compared to computer vision datasets. However, this data must be shared among many research teams, leading to hotspots where disk throughput is fully saturated but still cannot satisfy the application’s needs. This indicates that we need storage solutions that can steadily serve a lot of hot data with high throughput and low latency.

The infrastructure landscape for AI has also changed dramatically.

These days, with cloud computing and Kubernetes getting so popular, more and more AI companies are setting up their data pipelines on Kubernetes-based platforms. Algorithm engineers request resources on the platform, write code in Jupyter Notebook to debug algorithms, use workflow engines like Argo and Airflow to plan data processing workflows, use Fluid to manage datasets, and use BentoML to deploy models as applications. Cloud-native technologies have become a standard consideration when building storage platforms. As cloud computing matures, AI applications are increasingly relying on large-scale distributed clusters. With a significant increase in the number of nodes in these clusters, storage systems face new challenges related to handling concurrent access from tens of thousands of pods within Kubernetes clusters.

The evolving application workloads and computing environments significantly change the landscape for professionals managing the underlying infrastructure and platforms. Existing hardware-software-coupled storage solutions often suffer from several pain points, such as no elasticity, no distributed high availability, and constraints on cluster scalability. Distributed file systems like GlusterFS, CephFS, and those designed for high-performance computing (HPC) such as Lustre, BeeGFS, and GPFS are typically designed for physical machines and bare-metal disks. While they can be deployed as large capacity clusters, they cannot provide elastic capacity and flexible throughput, especially when dealing with storage demands in the order of tens or hundreds of billions of files.

Key capabilities for AI data storage

Considering these challenges, we’ll outline essential storage capabilities that are critical for AI scenarios, helping enterprises make informed decisions when selecting storage products.

POSIX compatibility and data consistency

In the AI/ML domain, POSIX is the most common API for data access. Previous-generation distributed file systems, except HDFS, are also POSIX-compatible, but products on the cloud have not been consistent in terms of their POSIX support:

• Compatibility: Users should not solely rely on the description “POSIX-compatible product” to assess compatibility. Instead, use pjdfstest and the Linux Test Project (LTP) framework for testing. We’ve conducted a POSIX compatibility test of cloud file systems for your reference.

• Strong data consistency guarantee: This is fundamental to ensuring computational correctness. Storage systems have various consistency implementations, with object storage systems often adopting eventual consistency, while file systems typically adhere to strong consistency. Careful evaluation is needed when selecting a storage system.

• User mode or kernel mode: Early developers favored kernel mode due to its potential for optimized I/O operations. However, in recent years, we’ve witnessed a growing number of developers “escaping” from kernel mode for several reasons

The Reliability Gap Holding AI Agents Back

AI researcher and Developer Advocate Sonam Gupta explores the engineering practices that transform AI agents from impressive demos into dependable production systems.

Every AI demo looks impressive.

Until it meets a real user.

For the last two years, the AI industry has celebrated increasingly capable foundation models, autonomous agents, and sophisticated reasoning systems. Every product launch promises more intelligence. Every benchmark claims higher accuracy. Every new release inches us closer to what once felt impossible.

Yet behind the excitement lies a quieter reality that every engineering team eventually encounters.

The hardest part isn’t getting an AI system to work once.

It’s getting it to work consistently, reliably, and at production scale.

That challenge is quickly becoming one of the defining engineering problems of the Generative AI era. As AI applications evolve from experimental prototypes into products that millions depend on, success is no longer measured by how intelligent a model appears in a carefully orchestrated demo. It’s measured by how predictably an entire system performs when confronted with the messy, unpredictable behaviour of real users.

At Packt, we believe these are the conversations that deserve more attention.

That is why Sanjana Gupta, Relationship Lead at Packt, sat down with Sonam Gupta, a data scientist, Developer Advocate at Telnyx, and host of the AI Chronicles podcast, for an in-depth discussion on what it really takes to build reliable AI systems. Bringing together her experience of working closely with AI authors, practitioners, and technical experts across Packt’s learning ecosystem, Sanjana steered the conversation beyond the usual questions about models and benchmarks, focusing instead on the engineering realities that practitioners face every day.

As I listened to their discussion, one thing became immediately clear.

This wasn’t another conversation about the latest LLM.

It was about something far more important.

Register now. Save 30% with EMTAX30

Why do so many AI applications fail after the demo, and what separates production-ready AI systems from prototypes that never earn users’ trust?

Over the next thirty minutes, Sonam unpacked lessons from building AI agents, evaluating LLM workflows, deploying voice applications, and interviewing leaders across companies such as Google DeepMind, Microsoft, AWS, and emerging AI startups. Rather than offering theoretical advice, she shared the engineering mindset that modern AI teams need if they want their systems to perform reliably in the real world.What follows is not simply an interview.

It’s a masterclass on why the future of AI belongs not only to those who build intelligent systems, but to those who build systems people can trust.

Build with Ben. Save 45% with OMEM45

The industry is asking the wrong question.

If you’ve attended an AI conference recently, you’ve probably noticed a familiar pattern.

Every discussion eventually circles back to the same debate.

Which model is better?

GPT or Gemini?

Open-source or closed-source?

Should developers switch frameworks?

Will AI agents replace traditional software?

These questions dominate keynote stages, LinkedIn discussions, and engineering forums. They’re important, certainly, but they also risk distracting us from a much larger engineering challenge.

As Sanjana opened the conversation, she didn’t ask Sonam which model she preferred.

Instead, she began somewhere much more fundamental.

Before discussing AI agents, evaluation frameworks, or production systems, she wanted readers to understand the perspective from which Sonam approaches the AI ecosystem itself.

Data Science & ML Research Roundup

⭕ Behind the scenes: How we build, test, and scale Google Agent Skills: Google has revealed how it built and governs Google Agent Skills, the open-source framework that gives AI coding agents structured domain knowledge to improve accuracy, reduce hallucinations, and follow best practices. The post explains how the project scaled from a Google Cloud Next 2026 initiative into a 15,000+ star GitHub repository, backed by standardized skill design, automated quality checks, continuous evaluations, clear ownership, and internal DevRel Skills that help teams automate and standardize workflows.

⭕ Prime Intellect Releases Prime Agent: An Open-Source RLM Harness Where Sub-Agents Are Function Calls Inside Persistent IPython Kernel: Prime Intellect has open-sourced Prime Agent, a self-improving AI coding harness that replaces fixed tool schemas with a persistent IPython kernel, allowing sub-agents to operate as function calls and continuously refine their own prompts, skills, and memory. The release details its deployable architecture, broad model compatibility, benchmark results, including 95.5% on ARC-AGI-3, and real-world use cases spanning software engineering, GPU optimization, AI research, and long-running autonomous coding tasks.

⭕ How LendingTree built a multi-agent mortgage assistant on Amazon Bedrock: LendingTree has shared how it built a production-ready multi-agent mortgage assistant on Amazon Bedrock to help borrowers navigate complex home financing decisions with personalized, compliant AI guidance. The post details its three-agent architecture, built with LangGraph, MCP, Amazon Bedrock Guardrails, and Knowledge Bases, along with deployment lessons, production metrics, and best practices for scaling secure, multi-agent systems in regulated industries.

⭕ What’s new in AI infrastructure this month: Google has published its latest AI infrastructure and orchestration roundup, highlighting new products, performance optimizations, and deployment guides for building and scaling AI and agentic workloads on Google Cloud. The update covers infrastructure launches such as Managed Lustre, C4N VMs, GKE enhancements, TPU tooling, AI security projects, customer deployments, and research showing how Google is improving AI performance, efficiency, and production readiness across its cloud platform.

⭕ Meta AI Releases Muse Code (Beta): A Terminal Coding Agent Powered by the New Muse Spark 1.2 Model: Meta AI has released Muse Code (beta), a terminal-based coding agent powered by the new Muse Spark 1.2 model, designed to tackle complex, long-running software engineering tasks across large codebases. The announcement outlines its persistent async agent architecture, replay-safe runtime, bundled planning skills, co-training approach, benchmark methodology, and real-world GPU kernel optimization case study, positioning it as a production-ready coding assistant for enterprise development workflows.

⭕ How Mobileye transformed support operations using Amazon Bedrock AgentCore: Mobileye has shared how it transformed its internal support operations with an AI Support Agent built on Amazon Bedrock AgentCore, reducing ticket response times by 90% while exceeding 95% accuracy. The post explains its hybrid production architecture, MCP-powered real-time data access, enterprise governance, and how the successful deployment evolved into a self-service platform that enables teams across the company to build and deploy production-grade AI agents.

⭕ Introducing the borderless Lakehouse: Google has unveiled major enhancements to its borderless Lakehouse, enabling AI agents to securely query, reason over, and act on data across on-premises systems, multiple clouds, and SaaS platforms without moving it. The announcement introduces Iceberg REST catalog federation, zero-copy cross-cloud analytics, Knowledge Catalog for trusted agent context, Gemini Enterprise integration, and cost optimizations that simplify building governed, multi-cloud AI agents while reducing data transfer and token costs.

⭕ How we built an MCP bridge to give our AgentCore-hosted AI agent access to local MCP tools: AWS has shared how it built an MCP bridge that lets cloud-hosted AI agents securely use tools and files running on a user’s local machine. The post explains the architecture behind the bridge, including AgentCore Runtime, browser extensions, WebSockets, native messaging, FastMCP, and stdio, showing how remote agents can work with local spreadsheets and systems without exposing credentials or requiring complex infrastructure.

⭕ NVIDIA Releases Alpamayo 2 Super: A 34B Open Vision-Language-Action Model for Robotaxis and Autonomous Driving Under OpenMDW-1.1: NVIDIA has released Alpamayo 2 Super, a 34B open vision-language-action model designed to help autonomous vehicles handle rare, complex driving situations. The release combines full-surround video understanding, trajectory planning, causal explanations, and meta-actions in one system, while introducing commercially usable weights, strong benchmark results, safety-focused reasoning traces, and tools for accelerating fleet-data annotation and autonomous-driving development.

⭕ Deep dive on new AI-powered database agents: Google has introduced Database Operations Agents, a pair of AI-powered assistants that automate database onboarding, monitoring, troubleshooting, and optimization across Google Cloud. The announcement details how the Database Onboarding Agent simplifies database selection and deployment, while the Database Observability Agent uses Gemini, telemetry, and MCP tools to diagnose issues, recommend remediations, and streamline database management across services such as Cloud SQL, Spanner, AlloyDB, and Bigtable.

See you next time!

Beyond the AI Hype: Building Systems People Can Actually Trust

Everyone is talking about AI agents. But the real challenge isn’t building them—it’s building systems your business can actually trust.

As organizations race to deploy enterprise AI, many are discovering that the biggest obstacles aren’t model performance or prompt engineering. They’re fragmented data, weak governance, poor infrastructure, and a lack of accountability.

Leading this edition is an insightful conversation between Marijn Markus, AI Leader at Capgemini, and Packt’s Relationship Lead Sanjana Gupta, exploring why AI doesn’t have an ethics problem—companies do. From data quality and enterprise readiness to the limits of LLMs and the future of responsible AI, it’s a timely reality check on what separates successful AI adoption from expensive experiments.

Also in this week’s highlights:

• 🔷 Google unveils 13 hands-on demos for the Gemini Enterprise Agent Platform, covering the complete AI agent lifecycle.

• 🔷 Amazon introduces an autonomous business intelligence architecture powered by Bedrock AgentCore and MCP.

• 🔷 Google releases OKF v0.2, bringing trust, provenance, and lifecycle metadata to agent-generated knowledge.

• 🔷 AWS introduces Task-Aware Knowledge Compression (TAKC), reducing enterprise AI context costs by up to 64×.

• 🔷 Google demonstrates how to take AI agents from prototype to production with Agents CLI.

• 🔷 Andrew Ng launches OpenWorker, a local-first AI coworker that delivers completed work instead of chat.

• 🔷 Google Cloud expands Conversational Analytics across BigQuery, Looker, AlloyDB, Cloud SQL, and Spanner with enterprise-grade governance.

Let’s dive in.

Cheers,

Merlyn Shelley,

Growth Lead, Packt.

Register now. Save 45% with MLTRADE45

Marijn Markus: AI Doesn’t Have an Ethics Problem. Companies Do.

The Capgemini AI Leader joins Packt’s Sanjana Gupta to explain why accountability, data quality, and human decision-making — not algorithms — will determine the future of enterprise AI.

Everyone wants to talk about Artificial Intelligence.

Very few want to talk about accountability.

Over the last two years, boardrooms have rushed to launch AI initiatives, governments have drafted AI regulations, and organizations have published glossy frameworks promising “Responsible AI.”

But according to Marijn Markus, AI Leader at Capgemini, we’re obsessing over the wrong problem.

In a compelling conversation with Sanjana Gupta, Relationship Lead at Packt, Markus challenges some of the biggest assumptions surrounding AI — from ethics and hallucinations to enterprise transformation and the future of work.

His central message is both uncomfortable and refreshing:

AI isn’t exposing technological problems. It’s exposing human ones.

Rather than another conversation about prompt engineering or the latest foundation models, this discussion dives into something far more important: why successful AI adoption depends on organizational maturity, high-quality data, and a willingness to confront uncomfortable truths.

AI Ethics Has Become a Convenient Distraction

Ask any enterprise about responsible AI and you’ll hear familiar phrases.

Bias.

Fairness.

Transparency.

Explainability.

Governance.

These are important topics.

But Markus believes they’re only part of a much bigger conversation.

Businesses are increasingly comfortable discussing AI ethics, while avoiding discussions about business ethics.

That’s a crucial distinction.

When organizations automate jobs, optimize for profit, or deploy systems that affect people’s lives, those decisions are rarely made by algorithms.

They’re made by people.

Yet when something goes wrong, blame quickly shifts to the technology.

As Markus points out, AI has become a convenient lightning rod — allowing organizations to distance themselves from decisions that were always fundamentally human.

The real ethical question isn’t whether an algorithm made a poor recommendation.

It’s who chose to deploy it, accepted its risks, and benefited from its outcomes.

The Curious Lifecycle of Enterprise AI

One of the interview’s most memorable observations comes from Markus’ experience leading AI projects across industries.

He jokes that every AI project follows the same lifecycle.

It starts as Artificial Intelligence during the sales pitch.

During implementation it becomes Machine Learning.

When lawyers become involved, it turns into Statistics.

By the time auditors inspect the system…

“It’s just an Excel sheet.”

The room laughs.

But the point is serious.

Technology doesn’t change.

The language does.

And with every change in terminology, accountability quietly shifts.

Organizations often use labels strategically, but changing the vocabulary doesn’t change the impact those systems have on people’s lives.

That’s why Markus insists these conversations shouldn’t be framed as AI dilemmas.

They’re simply ethical dilemmas.

Hallucinations Aren’t Bugs. They’re Probability.

Few topics dominate AI conversations more than hallucinations.

Markus strips away the mystery with a surprisingly simple analogy.

Large language models are sophisticated prediction engines.

Every response is, in essence, a probability calculation — a highly informed guess based on patterns learned from enormous amounts of data.

“They’re rolling dice,” he explains.

Not random dice.

Extremely well-trained dice.

That distinction matters.

Because it means hallucinations cannot be completely eliminated.

Only reduced.

Guardrails.

Evaluation pipelines.

Human review.

Retrieval systems.

These all lower the probability of failure, but they cannot remove uncertainty entirely.

Ironically, Markus argues, humans operate in much the same way.

People make mistakes.

Forecasts fail.

Judgment isn’t perfect.

The goal isn’t perfection.

It’s understanding acceptable levels of risk and designing systems accordingly.

Build with Ben. Save 40% with OME40

The Biggest AI Problem Isn’t AI

As the conversation shifts toward enterprise transformation, Markus introduces what may be his most important observation.

Companies don’t have AI problems.

They have infrastructure problems.

Many organizations proudly announce ambitious AI roadmaps while relying on fragmented databases, disconnected systems, poorly maintained SharePoint repositories, and years of inconsistent documentation.

Then they wonder why building an enterprise chatbot proves difficult.

“How can you build a company GPT,” Markus asks, “when your data isn’t organized in the first place?”

Artificial Intelligence doesn’t replace weak digital infrastructure.

It exposes it.

The organizations succeeding with AI today aren’t necessarily using better models.

They’re the ones that invested years earlier in digitization, governance, metadata, and clean data pipelines.

AI isn’t creating new weaknesses.

It’s shining a spotlight on existing ones.

We’re Running Out of Human Data

One of the interview’s most fascinating discussions centers on an emerging challenge that receives surprisingly little attention.

Future AI systems need high-quality training data.

But today’s internet increasingly consists of content generated by AI itself.

Articles.

Comments.

Images.

Videos.

Social media posts.

Entire conversations between bots.

As synthetic content grows faster than human-created content, future models risk learning from previous generations of AI instead of authentic human knowledge.

It’s a feedback loop that could gradually reduce the quality of future AI systems.

For Markus, this isn’t a theoretical concern.

It’s one of the industry’s biggest long-term challenges.

The future of AI may depend less on building larger models and more on preserving genuine human knowledge.

The Dangerous Rise of “Magical Thinking”

Perhaps the strongest message from the interview is Markus’ warning against what philosophers call magical thinking.

Organizations encounter a difficult business problem.

Instead of understanding it, they decide AI will somehow solve it.

No diagnosis.

No root-cause analysis.

No understanding of whether AI is even the appropriate tool.

Just hope.

According to Markus, this mindset has fueled countless disappointing AI projects.

Technology should never come before understanding the problem.

Sometimes a sophisticated LLM is the right solution.

Sometimes a forecasting model works better.

Sometimes a simple linear regression provides greater transparency.

And sometimes…

AI isn’t needed at all.

The best engineers don’t force AI into every problem.

They choose the right solution for the right problem.

Living in the Age of Deepfakes

As generative AI becomes capable of creating convincing images, videos, and voices, Markus believes society faces another challenge.

Trust.

The next generation won’t simply need AI literacy.

They’ll need information literacy.

Instead of trusting a single video or viral post, we’ll need to verify information across multiple independent sources.

Truth, Markus argues, survives verification.

Misinformation rarely does.

In an age where almost anything can be fabricated, critical thinking becomes our most valuable technology.

Beyond the Hype

Despite his criticisms, Markus is far from pessimistic about AI.

In fact, he’s remarkably optimistic.

He compares today’s generative AI revolution to the arrival of Google decades ago.

When search engines first appeared, they were considered cutting-edge AI.

Today, nobody thinks of Google as AI.

It’s simply part of everyday life.

That’s what successful technology does.

It becomes invisible.

Generative AI, he believes, will likely follow the same path.

The companies that succeed won’t be the ones making the loudest announcements or posting the most AI content on LinkedIn.

They’ll be the organizations quietly solving real problems with thoughtful engineering, quality data, and responsible decision-making.

As the conversation draws to a close, Markus leaves viewers with a reminder that perfectly captures the spirit of the discussion:

“Let’s focus on technology changing people’s lives, rather than just changing the content of internet feeds.”

In a world captivated by AI hype, it’s a refreshingly human perspective — and perhaps the most important lesson of all.

Data Science & ML Research Roundup

🔷 13 demos on Gemini Enterprise Agent Platform: Google has unveiled 13 hands-on demos for its Gemini Enterprise Agent Platform, showcasing the complete AI agent lifecycle from development to production. Built on the Agent Development Kit (ADK), the tutorials cover agent creation, MCP integration, human-in-the-loop workflows, scalable deployment, governance with built-in security, cross-framework orchestration, and continuous evaluation. With the new Agents CLI, developers can scaffold, deploy, monitor, and optimize enterprise-ready agents directly from their coding environment.

🔷 Generate Autonomous Business Insights with AI Agent and MCP Servers: Amazon has introduced a reference architecture for autonomous business intelligence powered by Amazon Bedrock AgentCore, enabling organizations to query enterprise data in natural language without custom integration code. Using pre-built MCP connectors, unified governance, persistent memory, and a semantic data layer, the platform orchestrates insights across IoT, ERP, analytics, and operational systems. The configuration-first approach simplifies secure, scalable multi-agent deployments while reducing data silos and accelerating enterprise decision-making.

🔷 OKF v0.2 adds trust signals: Google has released Open Knowledge Format (OKF) v0.2, extending its open standard for AI agent knowledge sharing with new metadata for provenance, trust, freshness, lifecycle, and attestation. Designed for agent-generated knowledge at scale, the update lets AI systems verify who created information, when it was validated, whether it’s current, and how it was produced—without sacrificing OKF’s lightweight, vendor-neutral design. All additions remain optional, preserving backward compatibility while enabling more trustworthy multi-agent workflows.

🔷 Beyond RAG: Task-aware knowledge compression for enterprise AI on AWS: AWS has introduced Task-Aware Knowledge Compression (TAKC), a technique that goes beyond traditional RAG by pre-compressing enterprise knowledge into task-specific representations for faster, cheaper AI inference. Supporting multiple compression tiers, TAKC preserves cross-document relationships while reducing token usage by 8x–64x. Built on Amazon Bedrock and serverless AWS services, the open-source reference architecture enables scalable, cost-efficient analysis for complex enterprise workloads such as financial due diligence and regulatory compliance.

🔷 Automate agent development lifecycles with Gemini Enterprise: Google has published a hands-on guide to building production-ready AI agents with the Gemini Enterprise Agent Platform and Agents CLI. The tutorial walks developers through the full agent lifecycle—from scaffolding and deterministic tool creation to secure deployment, governance, evaluation, and publishing—entirely within their coding environment. Using an Industry Watch agent as an example, it demonstrates how enterprise AI can combine live data, deterministic workflows, and built-in security to move beyond prototypes into production.

🔷 Andrew Ng Just Released OpenWorker: An Open-Source, Local-First Desktop AI Coworker That Returns Finished Deliverables Instead of Chat. Andrew Ng has released OpenWorker, an open-source, local-first desktop AI coworker that completes tasks instead of generating chat responses. Running entirely on the user’s machine, it combines a Python FastAPI agent, Tauri desktop app, and flexible model routing to produce finished deliverables while keeping data private. Built-in permission controls, prompt-injection safeguards, and support for local or cloud LLMs make OpenWorker a secure, enterprise-friendly AI productivity assistant.

🔷 Conversational Analytics in Google Data Cloud in Q326: Google Cloud has expanded Conversational Analytics across its enterprise data ecosystem, bringing natural language querying to BigQuery, Looker, AlloyDB, Cloud SQL, and Spanner. With support for multi-cloud data, MCP tools, and Gemini Enterprise integration, organizations can securely query business data using AI. The platform also introduces enterprise-grade governance, including row-level access controls, CMEK, VPC, data residency, and HIPAA compliance, enabling trusted, large-scale AI-powered analytics.

See you next time!

For the past two years, AI conversations have revolved around models.

Today, the conversation is shifting to something far more important: the engineering foundations that make those models reliable in production.

As organizations move beyond experimentation, success depends less on choosing the latest LLM and more on building the systems around it. Reliable data pipelines, trustworthy retrieval, rigorous evaluation, secure development workflows, efficient AI agents, and optimized inference are becoming the capabilities that separate impressive demos from production-ready AI.

We begin today’s edition with an expert perspective from Nikola Ilic, founder of Data Mozart, Microsoft Data Platform MVP, Microsoft Certified Trainer, and O’Reilly instructor. In his insightful article, Nikola explains why data engineering is the real foundation of useful LLM applications, and why the quality of your AI system is ultimately determined by the quality of the data workflows behind it.

Also in today’s highlights:

• Anthropic brings multi-agent security reviews to Claude Code with its new security plugin.

• Amazon introduces Self-Distilled Reasoning, making it easier to fine-tune reasoning models without sacrificing general capabilities.

• Google rethinks AI evaluation with Discovery Bench, revealing why traditional benchmarks often hide an agent’s true strengths and weaknesses.

• Cisco Foundation AI open-sources Antares, compact security models that rival much larger systems for vulnerability localization.

• monday.com shares how it deploys AI teammates at enterprise scale using Amazon Bedrock.

• Discover 11 practical principles for building faster, cheaper, and more token-efficient AI coding workflows.

• Google’s Gemini 3.6 Flash, Flash-Lite, and Flash Cyber push AI agents further with lower costs, improved efficiency, and production-ready performance.

Whether you’re building enterprise copilots, RAG systems, AI coding assistants, or autonomous agents, today’s edition explores the engineering practices that turn powerful models into AI systems people can depend on.

🎙️ This Weekend with Packt

Register Now

Happy reading, and we’ll see you in the workshop!

Cheers,

Merlyn Shelley,

Growth Lead, Packt.

Why Data Engineering Is the Real Foundation of Useful LLM Applications

The engineering decisions behind every trustworthy LLM application.

Everyone talks about choosing the right LLM. Far fewer people talk about the engineering decisions that determine whether the model ever sees the right information in the first place.

Written by Nikola Ilic

Nikola Ilic is the founder of Data Mozart, a Microsoft Data Platform MVP, Microsoft Certified Trainer, Pluralsight author, O’Reilly instructor, blogger, and international speaker. Known for helping practitioners “make music from data,” Nikola brings deep expertise in Microsoft Fabric, Power BI, SQL Server, analytics engineering, and modern data architecture. Through his writing, training, and hands-on workshops, he helps data professionals design production-ready data platforms that power modern analytics and AI applications.

Most conversations about LLM applications begin with the model.

Which one should we use?

How large is it?

How fast is it?

How much does inference cost?

Those are important questions.

But in production systems — especially Retrieval-Augmented Generation (RAG) applications — they’re rarely the questions that determine whether the application succeeds.

The data workflow does.

The quality of an LLM application is ultimately determined by the quality of the data pipeline behind it. Long before a prompt reaches the model, a series of engineering decisions has already shaped what the model can — and cannot — know.

That’s where reliable AI systems are really built.

You’re Not Connecting Documents to an LLM — You’re Engineering Knowledge

When we build a RAG system or an enterprise knowledge assistant, we are not simply connecting documents to an LLM.

We are building a pipeline that decides what the model is allowed to see.

Before a user asks a single question, source documents have already been ingested, parsed, cleaned, transformed, split into chunks, enriched with metadata, embedded, indexed, retrieved, assembled into context, and eventually evaluated.

Every one of those stages influences the final answer.

If any one of them is weak, the application becomes weaker.

That is why data engineering deserves far more attention in conversations about building LLM applications.

👉 Reserve your seat today and use code DATAPRO35 to save an exclusive 35%

Why Good Models Still Produce Bad Answers

Many teams assume that when an LLM gives an incorrect answer, the model must be at fault.

In reality, the problem often starts much earlier.

Perhaps the right document never entered the system.

A PDF parser may have dropped an important table.

Chunks may have split critical context in half.

Metadata may be incomplete, preventing accurate filtering.

The vector database may still contain stale embeddings from an earlier version of the content.

None of these are model problems.

They’re data engineering problems.

The model can only answer from the evidence it receives. If the evidence is incomplete, poorly structured, or outdated, the answer will be too — no matter how advanced the model is.

This is why improving prompts alone rarely fixes production RAG systems.

The biggest improvements almost always happen upstream.

The Engineering Questions That Actually Matter

Building reliable LLM applications means asking practical engineering questions, not just AI questions.

Questions such as:

• What documents should enter the system?

• How do we preserve structure from PDFs, Markdown, HTML, CSV, JSON, and APIs?

• What chunk size and overlap actually improve retrieval quality?

• Which metadata fields support filtering, governance, lineage, and refresh?

• How do we avoid duplicating vectors every time an ingestion pipeline runs?

• How do we measure whether retrieval quality is improving — or quietly getting worse?

These decisions directly affect cost.

Latency.

Reliability.

Compliance.

Maintainability.

And ultimately, user trust.

They are the engineering foundations that separate experimental AI from production AI.

The Best LLM Systems Are Engineered Data Products

The strongest LLM systems I see are not sophisticated prompt wrappers.

They are engineered data products.

They have repeatable ingestion pipelines.

Inspectable chunks.

Meaningful metadata.

Reliable embedding workflows.

Idempotent vector updates.

Clear source attribution.

Evaluation datasets that expose retrieval failures before users ever encounter them.

They are designed to evolve as new data arrives while remaining reliable, explainable, and observable.

Those capabilities don’t come from choosing a different model.

They come from building better data workflows.

That’s the difference between an impressive demo and a production-ready AI application.

The Skill That’s Becoming Essential for Every AI Builder

As organizations move beyond AI experimentation and begin deploying LLM applications across the enterprise, expectations are changing.

It’s no longer enough to know how to call an API or write effective prompts.

Teams increasingly need engineers who understand the complete lifecycle of knowledge pipelines — from raw documents to trustworthy retrieval.

Whether you’re building enterprise search, AI assistants, internal copilots, or knowledge-driven applications, designing reliable data workflows is rapidly becoming one of the highest-value skills in modern AI engineering.

The engineers who understand these workflows will be the ones building the next generation of production-ready AI systems.

👉 Reserve your seat today and use code DATAPRO35 to save an exclusive 35%.

Build the Complete Workflow — End to End

If these challenges sound familiar, that’s exactly what we’ll be tackling in my upcoming live Packt workshop, Designing Data Engineering Workflows for LLM Applications.

This isn’t a slide-heavy overview or another discussion about prompt engineering.

It’s a hands-on session where we’ll build a complete, production-oriented workflow — from raw documents to a working Retrieval-Augmented Generation pipeline.

Together we’ll cover:

• Document ingestion from multiple sources

• Parsing and preserving document structure

• Chunking strategies that improve retrieval quality

• Embeddings and vector database design

• Metadata for governance, filtering, and lineage

• Building Retrieval-Augmented Generation (RAG) workflows

• Evaluating and improving retrieval performance using practical techniques

By the end of the workshop, you’ll have a much clearer understanding of how reliable LLM applications are engineered — and more importantly, how to build those foundations yourself.

Don’t Wait Until Your RAG System Starts Hallucinating

One of the most common patterns I see is that teams only begin thinking seriously about data engineering after users report incorrect answers.

By then, the damage has already been done.

The model wasn’t the real problem.

The pipeline was.

Building trustworthy AI starts long before the prompt reaches an LLM.

It starts with designing the data workflows that determine what the model knows, how it retrieves information, and whether users can trust the answers it generates.

If production-ready LLM applications are part of your roadmap this year, this workshop is designed to give you the practical engineering mindset and hands-on techniques needed to build them with confidence.

Seats are limited to ensure an interactive, hands-on learning experience, and registrations are filling quickly.

If you’re ready to move beyond AI demos and start building reliable, production-grade LLM applications, I’d love to have you join us.

👉 Reserve your seat today and use code DATAPRO35 to save an exclusive 35%.

📚 Weekend Read: Hot Off the Press

Data Science & ML Research Roundup

✅ Anthropic Releases Claude Security Plugin for Claude Code in Beta: A Multi-Agent Vulnerability Scanner That Runs in Your Terminal. Anthropic has introduced the Claude Security plugin for Claude Code in beta, bringing multi-agent security reviews directly into the developer workflow. Instead of simply flagging vulnerabilities, it can scan an entire repository or recent code changes, verify findings through an independent three-agent review process, and generate reviewable patch files without modifying your code automatically. Running locally within your Claude Code session, the plugin complements existing SAST tools by combining AI-driven analysis with transparent verification, giving developers a practical way to identify, validate, and remediate vulnerabilities before code reaches production.

✅ Exploring self-distilled reasoning for supervised fine-tuning with Amazon Nova: Amazon researchers have introduced Self-Distilled Reasoning (SDR), a fine-tuning technique that preserves reasoning capabilities in Amazon Nova 2 models without requiring expensive chain-of-thought annotations. By generating reasoning traces from the base model and using them during supervised fine-tuning, SDR improves domain-specific performance while preventing catastrophic forgetting. Compared with model merging, it retains general reasoning, math, and coding abilities more effectively, offering a practical, low-cost path to customizing reasoning models.

✅ Evaluate agent performance: Traditional AI benchmarks often reduce performance to a single pass-or-fail score, but Google researchers argue this masks where agents truly succeed or fail. Introducing Discovery Bench, an information theory-based evaluation framework, they measure how increasing query ambiguity affects AI retrieval performance. By systematically varying query specificity, the approach exposes hidden failure points, uncovers flaws in benchmark quality, and provides a richer map of agent capabilities—helping developers improve retrieval systems beyond static benchmark scores.

✅ Cisco Foundation AI Releases Antares: 350M and 1B Open-Weight Models That Localize Known Vulnerabilities Inside Real Codebases. Cisco Foundation AI has open-sourced Antares, a family of compact security language models purpose-built for vulnerability localization—identifying vulnerable files in large code repositories from a vulnerability description. Despite its small size, the 1B-parameter model rivals frontier AI systems, outperforming much larger open models while approaching GPT-5.5 on Cisco’s new VLoc Bench. By combining supervised fine-tuning with reinforcement learning, Antares delivers accurate, low-cost vulnerability triage, highlighting how task-specific training can outperform brute-force model scale for practical software security.

✅ AI Teammates: how monday.com runs production AI agents on Amazon Bedrock. monday.com has revealed how it runs AI Teammates in production at enterprise scale using Amazon Bedrock, with AI agents now integrated into engineering workflows alongside human developers. Its multi-agent platform, Sphera, enables agents to pick up tasks, write code, open pull requests, and collaborate through Slack, GitHub, and monday.com, contributing to a 50%+ increase in per-engineer PR throughput. Backed by AWS infrastructure, persistent memory, automated guardrails, and continuous evaluation, the architecture demonstrates how production-grade AI agents can reliably augment software engineering teams at scale.

✅ Guide to AI Tokenomics: Eleven Principles for Token Efficient Software Engineering. Optimizing token usage is becoming an essential skill for developers working with AI coding assistants. Rather than relying on ever-larger context windows, this guide outlines 11 practical strategies—from choosing the right model and using reusable skills to delegating tasks, automating verification, and starting fresh sessions for new topics—to keep AI workflows faster, cheaper, and more accurate. By reducing context bloat and focusing the model’s attention, developers can improve coding quality, lower costs, and build more efficient human-AI collaboration.

✅ Google Releases Gemini 3.6 Flash, 3.5 Flash-Lite, and 3.5 Flash Cyber: A Cheaper, More Token-Efficient Flash Tier Built for Agentic Workloads. Google has expanded its Flash lineup with Gemini 3.6 Flash, Gemini 3.5 Flash-Lite, and Gemini 3.5 Flash Cyber, targeting developers building high-volume AI agents. Gemini 3.6 Flash delivers stronger coding and reasoning performance while reducing token usage and inference costs, Flash-Lite prioritizes low-latency, cost-efficient workloads, and Flash Cyber is purpose-built for automated vulnerability detection and patching. Together, the releases underscore Google’s focus on making AI agents faster, more affordable, and better suited for production-scale software engineering.

See you next time!

👋 Hi there, welcome to DataPro #178

That shift is becoming increasingly clear. Across the industry, the conversation is moving beyond benchmark scores and bigger models toward production-ready AI, efficient infrastructure, multimodal systems, browser-native inference, and intelligent automation that delivers measurable business value.

We’re leading this edition with an expert perspective from Diogo Alves de Resende, Business Analytics and Data Science Instructor and former Data Scientist at Zalando SE, who explores how modern AI and analytics teams can move beyond experimentation to build practical, scalable solutions that create lasting impact. If you’re building AI products or data-driven applications, this is a must-read.

This week’s highlights

• ⭐ Expert Voice: Diogo Alves de Resende on building practical, production-ready AI systems that deliver real business value.

• Claude on Google Cloud: Deploy frontier AI with managed infrastructure, global endpoints, enterprise security, and agentic workflows.

• Google AlphaEvolve: Google’s AI optimization agent is now generally available, helping engineers discover better algorithms across ML, logistics, finance, and scientific computing.

• Thinking Machines’ Inkling: A 975B open-weight multimodal MoE with controllable reasoning, 1M-token context, and enterprise-ready fine-tuning.

• Mistral Robostral Navigate: An 8B embodied AI model enabling robots to navigate complex environments using just a single RGB camera.

• AWS Computer Vision + MCP: Bringing vision, agents, and the Model Context Protocol together to build intelligent multimodal AI applications.

• Built Technologies + Amazon Bedrock: How AI-powered document intelligence is transforming real estate finance and reducing document review from days to minutes.

• Gemini Omni Flash & Nano Banana 2 Lite: Google’s newest multimodal creative models for high-speed image generation, video editing, and agentic media workflows.

• LiteRT.js: Run .tflite models directly in the browser with WebGPU and WebAssembly for fast, private, on-device AI.

Whether you’re building production-ready GenAI applications, optimizing large-scale AI systems, deploying multimodal models, or exploring the next wave of embodied and browser-native AI, there’s plenty to unpack in this week’s edition.

Grab your favorite beverage, settle in, and dive into the stories shaping the future of data science, machine learning, and AI engineering.

🎙️ This Weekend with Packt

If you’re building LLM applications that need to work beyond the demo, don’t miss our live workshop: Build Reliable GenAI Applications with AI Evals, Observability & Testing.

📅 Saturday, July 18 | 9:30 AM – 1:30 PM EDT

Happy reading, and we’ll see you in the workshop!

Cheers,

Merlyn Shelley,

Growth Lead, Packt.

Grab the bundle before the offer ends.

The Next Generation of GenAI Applications Will Be Built Around Workflows

Why production AI is shifting beyond prompts to orchestrated workflows powered by RAG, Python, APIs, evaluations, and guardrails.

Written by Diogo Alves de Resende

For the past two years, much of the conversation around GenAI has focused on prompting.

How do you write better prompts? Which model performs best? How can you make responses sound more intelligent?

Those questions matter, but they miss a larger shift that’s already happening.

The next generation of GenAI applications won’t be evaluated by how well they answer a single question. They’ll be evaluated by whether they can reliably complete an entire workflow from start to finish.

That is a fundamentally different engineering problem.

Register for the Workshop → Build Your Financial AI Analyst

From Answers to Actions

Consider a Financial AI Analyst.

A typical chatbot can answer questions about a company using what it already knows or by retrieving a few relevant document chunks. That might be enough for a demo.

It isn’t enough for a system someone can actually trust.

A production-ready financial assistant needs to do far more than generate text. It needs to gather evidence, perform calculations, validate results, and explain its reasoning before arriving at a conclusion.

A typical workflow might look like this:

• Retrieve the latest annual report.

• Identify the relevant financial statements.

• Extract revenue, cash flow, margins, and debt figures.

• Connect to a live market data API.

• Calculate returns, volatility, valuation ratios, and trends using Python.

• Compare results against previous reporting periods or competitors.

• Generate a clear, source-backed explanation.

Each step depends on a different capability.

RAG retrieves relevant information from enterprise documents. APIs provide live external data. Python performs deterministic calculations that shouldn’t be delegated to an LLM. The language model orchestrates these components, deciding which tools to use, when additional information is required, and how to communicate the final result.

This is what modern AI engineering increasingly looks like.

AI Systems Are Becoming Workflow Engines

Rather than relying on one large prompt and hoping for the best, developers are designing AI applications as structured workflows.

These systems retrieve information, call tools, execute code, validate intermediate results, and adapt their next actions based on what they discover.

The LLM becomes one component within a larger orchestration layer rather than the entire application.

This shift is enabling developers to build AI assistants that are significantly more useful because they can interact with external systems, perform real computations, and produce grounded outputs instead of plausible-sounding guesses.

But it also introduces a new challenge.

Every Additional Step Creates New Failure Points

The more capable an AI workflow becomes, the more opportunities there are for things to go wrong.

The system might retrieve the wrong document.

It could extract an incorrect financial value.

An API might return incomplete data.

A calculation could be performed using outdated inputs.

Or the model might confidently generate a conclusion that isn’t actually supported by the evidence it collected.

These aren’t isolated problems. They’re engineering challenges that emerge whenever multiple tools, data sources, and reasoning steps are combined into a single application.

That’s why modern AI workflows require more than good prompts.

They require evaluations, structured outputs, source verification, guardrails, prompt injection defenses, and mechanisms that make every step observable and testable.

Reliability isn’t something that’s added after deployment. It has to be designed into the workflow from the beginning.

Register for the Workshop → Build Your Financial AI Analyst

Building AI Systems You Can Trust

As enterprises move beyond experimentation, the definition of a successful GenAI application is changing.

It’s no longer enough for a model to produce an impressive answer.

The entire process behind that answer needs to be transparent, repeatable, and reliable enough to support real business decisions.

That’s where workflow-driven AI engineering is headed — and it’s rapidly becoming one of the most valuable skills for data scientists, ML engineers, and AI practitioners building production systems.

Build One Yourself

In my upcoming live workshop, Build Intelligent Assistants with GenAI, Python & AI Tools, we’ll move beyond theory and build a production-ready Financial AI Analyst from scratch.

Together, we’ll build an end-to-end workflow that combines:

• GenAI and modern LLMs

• Python for deterministic financial analysis

• Retrieval-Augmented Generation (RAG)

• Live market data APIs

• Jupyter Notebook

• Cursor

• Lovable

Along the way, we’ll explore how to integrate retrieval, tool use, evaluations, guardrails, and prompt injection defenses into AI workflows that are designed for real-world reliability rather than simple demonstrations.

If you’re looking to move beyond chatbot prototypes and start building production-ready AI assistants, this workshop is designed to give you a practical architecture you can reuse across financial analysis, enterprise copilots, and intelligent business applications.

📅 Live Online Workshop
Build Intelligent Assistants with GenAI, Python & AI Tools
Saturday, July 25 | 7:00 PM–11:00 PM GMT+5

Register for the Workshop → Build Your Financial AI Analyst

Data Science & ML Research Roundup

✅ Claude at scale on Google Cloud: Frontier AI, built for enterprise production: Running frontier AI at enterprise scale is about far more than model quality. This post explores how Claude on Google Cloud combines Anthropic’s advanced reasoning with Google Cloud’s managed infrastructure to help organizations deploy production-ready AI. It covers global low-latency endpoints, built-in security and compliance, data sovereignty, prompt caching, 1M-token context windows, cost optimization, and how the same platform powers the next generation of agentic AI workflows.

✅ Built Technologies builds an AI-powered document intelligence solution on AWS to power agents across real estate finance: Document-heavy industries like real estate are prime candidates for AI transformation. This post explores how Built Technologies uses Amazon Bedrock and the AWS Intelligent Document Processing Accelerator to turn complex real estate documents into production-ready AI intelligence. It shows how scalable document understanding, agentic workflows, and automated extraction are reducing review times from days to minutes while laying the foundation for next-generation AI applications.

✅ Thinking Machines Lab Releases Inkling: A 975B-Parameter Open-Weights Multimodal MoE With 41B Active Parameters And Controllable Thinking Effort. Thinking Machines Lab has unveiled Inkling, its first open-weight foundation model trained from scratch, built for large-scale customization and fine-tuning. This post explores its Mixture-of-Experts architecture, 1M-token context window, multimodal capabilities, controllable reasoning effort, benchmark performance, deployment options, and where it fits in production AI workflows—from voice and vision agents to cost-efficient, domain-specific enterprise applications.

✅ AlphaEvolve is available for everyone: Optimization is becoming one of AI’s most valuable enterprise capabilities. This post explores the general availability of AlphaEvolve on Google Cloud’s Gemini Enterprise Agent Platform, showing how organizations are using AI to optimize algorithms, code, and complex systems across logistics, chip design, genomics, finance, and scientific research. It also explains AlphaEvolve’s four-step optimization workflow and how teams can deploy it to accelerate production-grade AI and engineering innovation.

✅ Agentic vision: Building visual intelligence with Amazon Bedrock and MCP servers. Building AI that can see, reason, and act has traditionally required stitching together multiple tools and APIs. This post explores how AWS combines Computer Vision, Strands Agents, and the Model Context Protocol (MCP) into a unified framework, enabling developers to build intelligent multimodal applications with streamlined integration, secure access, and production-ready image and video understanding powered by Amazon Bedrock, Rekognition, and OpenSearch.

✅ Mistral AI Releases Robostral Navigate: An 8B Model Enabling Robots to Navigate Complex Environments Using a Single RGB Camera. Embodied AI is taking a major step forward with Mistral AI’s Robostral Navigate. This post explores how the 8B vision-language model enables robots to navigate complex real-world environments using only a single RGB camera and natural language instructions. It covers the model’s novel pointing-based navigation, efficient training approach, reinforcement learning, benchmark-leading performance, and practical applications across manufacturing, logistics, hospitality, and autonomous robotics.

✅ Nano Banana 2 Lite and Gemini Omni Flash available: Creative AI workflows are getting faster and more capable with two new additions to Google’s Gemini Enterprise Agent Platform. This post explores Gemini Omni Flash for conversational video generation and editing, alongside Nano Banana 2 Lite, Google’s fastest and most cost-efficient image model. It covers their multimodal capabilities, enterprise governance, price-performance advantages, and how businesses are using them to build next-generation creative and agentic AI applications.

✅ Google Releases LiteRT.js: A JavaScript Binding of LiteRT That Runs .tflite Models in Browsers via WebGPU. Running AI models directly in the browser is becoming faster and more practical. This post explores Google’s new LiteRT.js, a JavaScript binding that brings native LiteRT (.tflite) models to the web using WebAssembly, WebGPU, and experimental WebNN. It covers the architecture, performance gains, model conversion workflow, deployment patterns, and how developers can build private, low-latency, on-device AI applications without relying on cloud inference.

See you next time!

This NINJIO Insights Report dives into the key emotional susceptibilities that make social engineering work and offers concrete steps that your security team can take to equip your workforce to resist cyberattacks.

Download the Guide

👋 Hi there, welcome to DataPro #177 – Special Weekend Edition.

The AI conversation has become increasingly dominated by models, benchmarks, and agents. Every week brings another breakthrough, another framework, or another promise of autonomous intelligence. Yet inside enterprise teams, a far more practical question continues to surface: How do you know whether your AI product is actually delivering value?

Building an AI application is only the beginning. Understanding how users interact with it, where they lose trust, what drives adoption, and whether it improves real business decisions is quickly becoming one of the most valuable skills for data professionals. It’s a discipline where data engineering, analytics, product thinking, SQL, Python, experimentation, and AI all converge.

That’s exactly what we’re exploring in this special weekend edition. In the latest Packt Talks session, hosted by Abhishek Kaushik, Prithvi Shivshankar, Data AI Product Manager at H-E-B, takes us inside the emerging world of AI Product Analytics—showing how modern teams instrument AI products, analyze user journeys, measure product adoption, and use behavioral data to continuously improve AI-powered experiences. Drawing on real enterprise examples, he walks through why successful AI products aren’t built around models alone, but around better decisions.

If you’re building AI applications, working with data products, or simply curious about what happens after an AI product reaches production, this is one read that’s well worth your weekend.

Happy reading!

Cheers,

Merlyn Shelley,

Growth Lead, Packt.

Book Your Spot & Save 35%

Why Every AI Product Needs Better Analytics Before Better Models

How SQL, Python, product analytics, and user behavior are becoming the foundation of successful AI products

Every AI product generates outputs.

Far fewer generate outcomes.

As enterprises race to embed generative AI into applications, copilots, and decision-support systems, the conversation has largely centered around models, prompts, reasoning capabilities, and agents. Yet once these products reach production, many organizations encounter a very different challenge.

Are people actually using them?

Do users trust the recommendations?

Where do they abandon the workflow?

More importantly, did the AI improve a business decision—or simply become another dashboard?

That’s exactly what the latest episode of the Packt Talks YouTube series, hosted by Abhishek Kaushik, set out to answer. Prithvi Shivshankar, Data AI Product Manager at H-E-B, walked viewers through the emerging discipline of AI Product Analytics, demonstrating how modern product teams combine SQL, Python, behavioral analytics, experimentation, and business metrics to understand user journeys, evaluate AI performance, and build AI products that people actually trust and use.

Drawing on his experience building enterprise-scale data products at one of the largest independently owned grocery retailers in the United States, Prithvi offered a practical perspective that many AI discussions overlook. The session wasn’t about choosing the best foundation model or writing better prompts. It was about what happens before an AI product is built—and, more importantly, what happens after it ships.

AI Product Analytics Starts with the Business, Not the Model

One of the first ideas Prithvi challenged was the way many AI projects begin.

Too often, teams start with the technology.

A powerful LLM.

An intelligent assistant.

A sophisticated recommendation engine.

Only afterwards do they begin searching for somewhere to use it.

According to Prithvi, successful AI products reverse that process entirely.

Before introducing AI, teams must first understand how decisions are made today. Who makes those decisions? What information do they rely on? Where do they spend most of their time? Which parts of the workflow create friction? Where does decision fatigue begin?

Only after mapping the existing business process should AI enter the conversation. Otherwise, even the most capable model risks becoming another impressive demonstration that never translates into measurable business value.

This simple shift—from starting with technology to starting with workflow—became the foundation for everything that followed during the session.

A Real-World AI Product: Solving Decision Latency in Retail

Rather than relying on theoretical examples, Prithvi grounded the discussion in a production AI product built for supply chain operations.

Managing inventory across a large grocery network is a constant balancing act.

Products move between suppliers, warehouses, transportation fleets, distribution centres, and retail stores before finally reaching customers. Every stage introduces new variables—supplier delays, transportation constraints, seasonal demand, promotions, warehouse capacity, and product shelf life.

Take everyday essentials like milk or bread.

Running out of stock means disappointed customers and lost revenue.

Holding too much inventory means spoilage, shrinkage, and unnecessary operational costs.

The challenge isn’t that organizations lack data.

It’s that analysts often spend hours navigating multiple dashboards, exporting reports, comparing spreadsheets, writing SQL queries, and stitching together information before they can confidently decide whether inventory should be transferred, expedited, reordered, or held.

That’s where the Inventory Decision Assistant comes in.

Its purpose isn’t to replace analysts.

It’s to reduce the time between identifying a problem and taking action.

Throughout the webinar, Prithvi referred to this as reducing decision latency—helping analysts reach the right decision faster by surfacing risks, explaining root causes, and recommending the next best action.

Book Your Seat & Save 35%

AI Doesn’t Create Value by Making Recommendations

Another important theme emerged as the session progressed.

Generating recommendations has become relatively easy.

Generating recommendations that people trust is much harder.

Prithvi explained that an effective AI product needs several layers working together.

It needs to detect meaningful business signals.

Provide enough context to explain why something happened.

Recommend an action.

Allow users to execute that action seamlessly.

Finally, it needs to learn whether that recommendation actually helped.

Without these layers, organizations often mistake model output for business value.

Recommendations alone create very little impact.

Recommendations that users understand, trust, and act upon are what ultimately improve business outcomes.

Mapping the User Journey Before Writing Any AI

One of the most practical parts of the webinar focused on mapping the decision journey.

Using the inventory assistant as an example, Prithvi broke the process into several distinct stages.

Everything begins with a signal—a stock-out risk, declining inventory, or an unexpected demand spike.

Analysts then begin investigating by opening dashboards, querying databases, reviewing historical demand, and comparing operational reports.

Next comes diagnosis.

Is demand increasing unexpectedly?

Has a supplier delayed shipments?

Is inventory sitting too long in a warehouse?

Is there an imbalance between distribution centres?

Only after answering those questions can analysts make a decision.

Transfer inventory.

Expedite shipments.

Adjust purchase orders.

Escalate operational issues.

Finally comes execution, where those decisions translate into actions across the supply chain.

By visualising every step in this journey, teams can identify precisely where AI removes friction—and where simpler automation or better process design may already solve the problem.

Book Your Seat & Save 35%

Not Every Business Problem Needs AI

One refreshing takeaway from the session was that Prithvi repeatedly cautioned against applying AI indiscriminately.

Some business problems simply don’t require large language models.

Static reporting rarely does.

Rule-based automation often doesn’t.

Basic workflows can frequently be solved without introducing generative AI at all.

According to Prithvi, AI becomes valuable when decisions are repetitive, time-sensitive, involve multiple signals, or require synthesising complex information into actionable recommendations.

That’s where AI shifts from being a novelty to becoming genuine decision support.

Why Product Analytics Matters After Deployment

Perhaps the biggest insight from the webinar came after the AI product had already been deployed.

Shipping the product isn’t the finish line.

It’s the beginning.

Once users begin interacting with an AI assistant, product teams need to understand how that interaction unfolds.

Did users open the recommendation?

Did they read the explanation?

Did they accept the suggested action?

Did they complete it?

Where did they abandon the workflow?

These behavioural questions are often impossible to answer using traditional business intelligence alone.

Instead, they require product analytics.

Prithvi introduced tools such as Amplitude as examples of behavioural analytics platforms that capture user interactions throughout the product lifecycle. Unlike BI dashboards that explain what happened in the business, product analytics helps explain how users behave inside the product itself—and where trust begins to break down.

Where SQL and Python Fit Into AI Product Analytics

While AI dominated the discussion, the webinar repeatedly returned to two technologies that every data professional already knows.

SQL.

Python.

SQL forms the backbone of product analytics by transforming raw event logs into meaningful user funnels.

How many recommendations were generated?

How many were viewed?

How many were accepted?

How many ultimately resulted in completed actions?

Those simple funnel analyses often reveal where products lose users long before sophisticated machine learning becomes necessary.

Python then extends those insights through cohort analysis, segmentation, experimentation, A/B testing, behavioural modelling, and recommendation analysis.

Rather than replacing traditional analytics, AI product development depends on it.

As Prithvi put it, SQL helps measure the funnel, while Python helps explain the patterns behind it.

Measuring What Actually Matters

The webinar concluded with a useful framework for thinking about AI success.

At the bottom sit model metrics.

Precision.

Recall.

Latency.

Cost.

Groundedness.

These remain important.

Above them sits product adoption.

Are recommendations viewed?

Accepted?

Executed?

Are users providing positive feedback?

At the very top sit the metrics every business ultimately cares about.

Did stock-outs decrease?

Was waste reduced?

Were analysts able to make decisions faster?

Did customer experience improve?

Those are the outcomes that determine whether an AI product has truly succeeded.

Final Thoughts

If there was one message that carried through every section of the webinar, it was this:

Great AI products don’t simply generate better answers. They reduce decision friction.

As AI becomes deeply integrated into enterprise products, data professionals will increasingly find themselves working across product management, analytics engineering, experimentation, behavioural analytics, SQL, Python, and machine learning.

The organizations that succeed won’t necessarily be those with the most advanced models.

They’ll be the ones that understand their users best.

Because in the end, successful AI isn’t measured by what the model produces.

It’s measured by the decisions people make because of it—and the business outcomes those decisions create.

See you next time!

This NINJIO Insights Report dives into the key emotional susceptibilities that make social engineering work and offers concrete steps that your security team can take to equip your workforce to resist cyberattacks.

Download the Guide

👋 Hi there, welcome to DataPro #176.

As foundation models become increasingly commoditized, competitive advantage is shifting somewhere less glamorous but far more consequential: the quality of your data, features, and production systems.

This week’s lead story explores exactly that. In an insightful conversation, Cornellius Yudha Wijaya, Chief Product Officer and AI practitioner, shares why the features that deliver the best offline metrics often become the first ones to fail in production. Interviewed by Vaideeshwari Roshan, he explains why point-in-time correctness, feature stability, and engineering discipline matter far more than squeezing out another percentage point of model accuracy.

This week’s highlights:

• 🎙️ Expert Interview: Why production-ready feature engineering is about engineering trust, not just better features

• ☁️ Google Cloud’s Gemini Enterprise Agent Platform brings secure MCP connectivity to external AI agents and developer workflows

• 🐍 Baidu’s CUP toolkit streamlines production Python with logging, concurrency, caching, and reliability utilities

• 🧬 Schrödinger and Google DeepMind achieve a 4× acceleration in molecular simulations using AlphaEvolve

• 🤖 Claude Sonnet 5 pushes agentic coding forward with stronger benchmarks and compelling cost-performance gains

• 🛡️ Amazon Bedrock introduces centralized model governance and resilient LLM inference architectures for production AI

• 🎬 Outpost VFX reduces AI model training from weeks to days using distributed multi-GPU training on AWS

If you’re building machine learning systems that need to survive production, scale across cloud infrastructure, or keep pace with the rapidly evolving AI ecosystem, this edition is packed with ideas, architectures, and lessons worth borrowing.

Let’s get into it.

Cheers,

Merlyn Shelley,

Growth Lead, Packt.

Book Your Spot & Save 35%

The Feature That Looked Perfect… Until It Reached Production

Feature engineering is often described as the secret sauce behind great machine learning models. But after reading through a conversation between my colleague Vaideeshwari Roshan and data scientist Cornellius Yudha Wijaya, I realized the conversation isn’t really about feature engineering. It’s about engineering trust.

Cornell has spent more than seven years building machine learning systems across insurance and AI startups, working with sensitive enterprise data where every prediction has business consequences. His perspective is refreshingly different from the feature engineering tutorials most of us consume. Rather than focusing on creating more features, he focuses on building features that survive production.

Below is an edited version of that conversation, along with a few observations that stood out to me as I read through it.

🎙️ Behind the Conversation

Interviewer: Vaideeshwari Roshan, Portfolio Manager, Packt

Expert: Cornellius Yudha Wijaya, Chief Product Officer, Data Scientist, and Author

“Feature engineering isn’t about creating more features.”

One of the first things that struck me was how quickly Cornell shifted the conversation away from algorithms.

Many of us instinctively think about machine learning in terms of model selection. Should we use XGBoost? CatBoost? Neural networks?

Cornell’s answer reminded me that production ML teams often spend far more time debating the quality of their features than the sophistication of their models.

Vaideeshwari: You’ve worked on enterprise machine learning systems as well as AI products in startups. How has your perspective on feature engineering evolved?

Cornellius: Early in my career, I thought feature engineering was mostly about creating predictive variables from raw data.

Over time, I realized that’s only one part of the job.

Today, I think about whether a feature is reliable enough for production. Can it survive upstream data changes? Is it explainable to business stakeholders? Can another engineer understand and reproduce it? Those questions matter just as much as predictive performance.

Feature engineering has become less about data manipulation and more about building reliable systems.

✍️ Editor’s Reflection

That distinction feels increasingly relevant.

As foundation models and AutoML continue to simplify model building, the competitive advantage is shifting elsewhere.

The harder problem isn’t choosing an algorithm.

It’s deciding which information your model should trust.

That philosophy became even clearer when Cornell described one of his largest production projects.

Building a churn prediction model where time was the biggest challenge

The project Cornell chose to discuss wasn’t unusual on paper.

It was a customer churn prediction system built at Allianz Life Indonesia.

The challenge wasn’t predicting churn.

The challenge was making sure every prediction reflected what the business actually knew at that moment in time.

Vaideeshwari: What made this project particularly challenging from a feature engineering perspective?

Cornellius: The data came from multiple enterprise systems spanning different time periods.

Every customer had to be represented at a specific reference date, and every feature had to be calculated only from information available before that point.

That sounds simple, but it’s one of the easiest places to make mistakes.

One incorrect join or timestamp can accidentally introduce future information into your feature set.

Vaideeshwari: You often emphasize “point-in-time correctness.” Why is it so important?

Cornellius: Because models should never learn from the future.

If a feature accidentally contains information generated after your prediction date, validation results become misleading.

The model appears much better than it really is.

Then you deploy it, and suddenly performance drops because that future information no longer exists.

Every feature should answer one question:

“Would we have known this information at prediction time?”

If the answer is no, the feature shouldn’t exist.

Book Your Seat & Save 35%

✍️ What surprised me most

Data leakage is something every ML engineer has heard about.

But Cornell frames it differently.

He doesn’t describe it as a modeling mistake.

He describes it as a feature engineering mistake.

That’s an important mindset shift because it moves the discussion upstream, where these problems actually begin.

The most valuable features weren’t static

One misconception I had before reading this interview was assuming customer profiles would carry most of the predictive power.

Instead, Cornell kept coming back to one word:

Behavior.

Vaideeshwari: Which features ultimately made the biggest difference?

Cornellius: Behavioral features consistently outperformed static customer attributes.

Instead of describing who customers were, we focused on how they were changing.

Rolling-window aggregates over the previous 7, 30, and 90 days helped capture recent activity.

Trend features showed whether engagement was increasing or decreasing.

Delta features highlighted meaningful behavioral shifts.

For highly skewed financial variables, log transformations also improved robustness.

Customers are constantly changing.

Our features needed to reflect that.

✍️ Editor’s Reflection

That answer stayed with me.

Good features don’t simply describe reality.

They describe change.

Whether you’re predicting churn, fraud, demand, or equipment failure, the strongest signal often isn’t the current state.

It’s the direction of movement.

The feature that improved accuracy… and still got deleted

This was probably the most unexpected moment in the conversation.

Cornell explained that his team deliberately removed features that improved model performance.

Naturally, Vaideeshwari asked why.

Vaideeshwari: Why would you remove a feature that improves your validation metrics?

Cornellius: Because validation metrics aren’t the business objective.

Some features looked extremely predictive because they contained subtle leakage.

Others depended on business rules that changed frequently.

A few simply weren’t stable over time.

Keeping them improved offline accuracy.

Removing them improved production reliability.

Sometimes you have to trade a slightly better benchmark for a much better production system.

✍️ One sentence worth remembering

A feature that boosts validation scores but can’t survive production isn’t an asset. It’s technical debt.

Production doesn’t end with model deployment

One theme kept appearing throughout the interview.

Feature engineering isn’t finished once the model is trained.

In many ways, that’s when the real work begins.

Vaideeshwari: How did your team maintain feature quality after deployment?

Cornellius: Every feature was treated like a managed asset.

We documented feature definitions, tracked version changes, maintained validation rules, and monitored data freshness, missing values, and business-rule consistency.

When upstream systems changed unexpectedly, anomaly detection alerted us before the issue affected production predictions.

Feature engineering is an ongoing operational process.

Vaideeshwari: Did richer features ever conflict with operational requirements?

Cornellius: Definitely.

More sophisticated features often improve predictive performance.

But they also increase computation costs, latency, and pipeline complexity.

We had to balance model quality with operational SLAs and business value.

The best feature isn’t always the most sophisticated one.

It’s the one that delivers reliable value at scale.

Three lessons every ML engineer can borrow

Before wrapping up, Vaideeshwari asked Cornell what he hoped practitioners would remember most from his experience.

Vaideeshwari: If our readers remember only three lessons from this conversation, what should they be?

Cornellius:

1. Point-in-time correctness is non-negotiable.

Never allow your model to learn from information it wouldn’t have during prediction.

2. Stability is a feature.

Reliable, consistent features often outperform clever ones over the long term.

3. Every feature should support a business decision.

Predictions become valuable only when they enable action.

Continue reading the full conversation on Packt’s Medium publication.

Book Your Seat & Save 35%

Data Science & ML Research Roundup

◾ Gemini Enterprise Agent Platform remote MCP server: Google Cloud has introduced the Gemini Enterprise Agent Platform remote MCP server, giving developers a secure way to connect external AI agents and IDEs like Claude Code to Google Cloud resources. Built on the open MCP standard, it enables access to Model Garden, prompt libraries, notebooks, and model management through a single governed interface, combining faster development with enterprise-grade security and IAM-based access controls.

◾ CUP (Common Useful Python): Building Reliable Python Workflows with Baidu’s Utility Toolkit. Baidu’s Common Useful Python (CUP) toolkit brings together production-ready utilities that simplify reliable Python development. From structured logging, configuration management, caching, and concurrency to ID generation, scheduling, networking, and resource monitoring, CUP provides a unified toolkit for building maintainable, scalable applications. The tutorial walks through practical workflows that help developers reduce boilerplate and improve the reliability of real-world Python systems.

◾ How Schrödinger sped up molecular discovery by 4x with Alphaevolve: Schrödinger has accelerated molecular simulations by 4× using Google DeepMind’s AlphaEvolve, an evolutionary AI coding agent that optimizes performance-critical algorithms in machine-learned force fields (MLFFs). By replacing computational bottlenecks with AI-generated parallel implementations, the company significantly sped up model training and inference, enabling faster drug discovery, catalyst design, and materials research while paving the way for AI-optimized GPU kernels in scientific computing.

◾ Anthropic Claude Sonnet 5 vs Sonnet 4.6 vs Opus 4.8: Agentic Coding Benchmarks, API Pricing, and Cost-Performance Tradeoffs Compared. Anthropic has unveiled Claude Sonnet 5, its most capable mid-tier model yet, designed for long-running agentic workflows, autonomous coding, and tool use. It outperforms Sonnet 4.6 across every published benchmark while narrowing the gap with Opus 4.8. With introductory API pricing of $2/$10 per million tokens, Sonnet 5 offers one of the strongest cost-to-performance ratios for AI coding, automation, and enterprise development workloads.

◾ Simplify multi-account access to Amazon Bedrock models with managed entitlements: Amazon Bedrock now supports managed entitlements, allowing enterprises to subscribe to third-party foundation models such as Anthropic Claude and Cohere once and securely distribute access across AWS accounts. By centralizing subscriptions with AWS License Manager, organizations simplify governance, streamline multi-account AI deployments, maintain consistent pricing, and eliminate the need for AWS Marketplace permissions in individual workload accounts.

◾ Implementing resilience patterns with Amazon Bedrock and LLM gateway: As generative AI moves into production, Amazon Bedrock provides a set of resilience patterns to keep LLM inference highly available, scalable, and cost efficient. The guide covers five production-ready approaches, including cross-Region inference, multi-account quota isolation, and intelligent request routing, helping organizations improve availability, handle traffic spikes, reduce throttling, and build reliable multi-model AI applications on AWS.

◾ How Outpost VFX Uses AWS to Accelerate AI Model Training for Visual Effects: Outpost VFX has accelerated AI-powered face replacement training by up to 8× using AWS multi-GPU P5 instances and PyTorch Distributed Data Parallel. By replacing single-GPU workflows with distributed training, the studio reduced model training from 1–2 weeks to just two days, enabling faster creative iteration, higher-quality outputs, and more efficient visual effects production at scale.

See you next time!

Build self-service analytics 4× faster with OpenUI Cloud.

What if every user could turn a question into a dashboard? OpenUI Cloud plugs into your existing data stack, transforming plain-English queries into live charts, KPI cards, reports, and dashboards—without adding to your BI backlog or frontend roadmap.

Start Building Now

👋Hey there, welcome to DataPro #175.

Building AI systems is no longer the hard part. Building AI systems that behave consistently, scale across teams, and are fast enough for production is. Many organizations are discovering that great prompts alone don’t translate into reliable products. What they need are reusable workflows, measurable infrastructure, and architectures designed for production from day one.

That’s exactly where this week’s leading story comes in. Elisa Terumi, PhD, explains why Skills are emerging as a foundational building block for AI agents, showing how they transform one-off prompts into reusable, version-controlled capabilities that make agentic systems easier to build, maintain, and scale.

In this week’s highlights:

• 🧠 Learn how AI Skills are reshaping agent architectures and why they’re becoming the new standard for reusable AI workflows.

• ⚡ Explore DFlash, a new speculative decoding technique delivering up to 15× higher LLM throughput on NVIDIA Blackwell GPUs.

• 📄 Discover Datalab’s lift and Baidu’s Unlimited OCR, two approaches pushing document AI forward with structured extraction and long-document parsing.

• 🎙️ See how speech AI is evolving with Gradium’s real-time translation models and Amazon Nova 2 Sonic powering natural voice agents.

• Discover how OpenUI Cloud brings conversational analytics to your existing data stack.

• 📊 Dive into the latest cloud tooling, from BigQuery Managed Python UDFs and Google Observability Analytics to enterprise-scale AI deployments across banking and telecom.

Whether you’re building agentic applications, optimizing inference, or deploying AI into production, this edition brings together the architectural patterns, infrastructure updates, and engineering breakthroughs shaping the next generation of Data and AI systems.

Cheers,

Merlyn Shelley,

Growth Lead, Packt.

⚙️ This Week's Packt Expert Workshop: Production-Ready GenAI Starts with Evaluation

As GenAI becomes part of AI applications, data pipelines, and ML workflows, evaluating model outputs is no longer optional. Join Amy Chen and Surjeet Mishra to learn practical frameworks for measuring LLM quality, identifying failure modes, building evaluation pipelines, and implementing feedback loops that make GenAI systems more reliable, trustworthy, and production-ready.

🎟 Save 30% today with the registration link.

The workshop is filling up quickly. If GenAI is part of your AI, ML, or data stack, you won't want to miss it.

Join Amy Chen & Sujeet Mishra Live

What Are Skills in AI Agent Systems? And How to Build Your Own

Written by Elisa Terumi, PhD

The term sounds simple. But in modern AI systems, it has a very specific meaning.

And understanding it changes how you build with LLMs.

What are skills?

Skills are modular, reusable instruction sets that teach an AI system how to perform specific tasks or workflows.

In systems like Claude Code, a skill is typically:

A folder

Containing a SKILL.md file

With structured instructions describing how a task should be executed

Once defined, the system can automatically apply that knowledge whenever a relevant request appears.

This is the key shift:

Instead of repeating prompts, you encode behavior once — and reuse it.

I’ve created a repository with practical examples of skills — feel free to explore it here: https://github.com/elisaterumi-ai/agent-skills-in-practice

Skills as structured capabilities (not just prompts)

A common misconception is to treat skills as “saved prompts.”

They are not. A saved prompt is a one-off instruction you reuse manually. A skill is closer to a standard operating procedure (SOP) for AI:

It defines what to do

When to do it

How to do it consistently

The practical difference is significant. A prompt depends on you remembering to use it and applying it correctly each time. A Skill is activated automatically by the system when the context is relevant, follows a testable structure, and can be shared with your team as part of the repository.

Technically, a Skill combines instructions, workflows, and context to handle multi-step tasks — while a standalone tool executes one specific deterministic function, and a one-off prompt has no structure or reuse.

That combination is what makes Skills an architectural pattern, not just a convenience.

How skills work under the hood

The execution model is subtle — and important.

When a system (like Claude Code) runs:

It loads only skill names and descriptions

It receives a user request

It performs semantic matching

It selects relevant skills

It loads the full instructions and executes them

This has two implications:

Skills do not clutter the context window

They activate only when needed

Skills vs prompts vs tools

Understanding this distinction is critical.

Prompts

One-off instructions

Not reusable

No structure

Tools

Execute a specific function

Deterministic behavior

Skills

Combine: instructions + workflows + context

Handle multi-step tasks

In other words:

A tool does one thing. A skill orchestrates how things should be done.

Why skills matter (from experimentation to production)

Skills are not just a convenience feature. They are an architectural pattern.

They enable:

Consistency → same output format every time

Reuse → define once, apply everywhere

Scalability → move from prompts to systems

Collaboration → share workflows across teams

In fact, skills are increasingly used to:

encode coding standards

enforce documentation formats

automate workflows

embed domain knowledge into AI systems

Where skills live

Skills are typically scoped at two levels:

Personal skills

Stored locally

Reused across projects

In Claude systems, personal skills live in ~/.claude/skills in your home directory. These follow you across all your projects — your commit style, your documentation format, how you like code explained.

Project skills

Stored in repositories

Version-controlled

Shared with teams

Project skills live in .claude/skills inside the repository root. Anyone who clones the repo gets these skills automatically. This is where team standards live: coding conventions, brand guidelines, project-specific processes. Because they sit inside the repository, they’re version-controlled alongside the code and shared naturally through Git.

This makes them part of the codebase — not just user configuration.

The Anatomy of a Skill

A Skill is a directory containing a SKILL.md file. The directory name should match the skill name. The file has two parts: a YAML metadata block at the top and Markdown instructions below.

The metadata defines name and description,both required. The description is the most critical field: it’s what Claude uses to decide whether the Skill is relevant. Two optional fields also exist: allowed-tools, which restricts which tools Claude can use while the Skill is active, and model, which specifies which Claude model to use for that Skill.

The instructions define the steps, rules, and output format. This is where the actual procedure lives.

When should you create a skill?

A practical rule:

If you are repeating the same instructions more than once, you should create a skill.

Typical use cases:

Code review guidelines

Commit message formats

Documentation templates

Data processing pipelines

Domain-specific transformations

Practical Example: A PR Description Skill

Let’s build a personal Skill that teaches Claude to write pull request descriptions in a consistent format.

First, create the directory:

mkdir -p ~/.claude/skills/pr-description

Then create the SKILL.md file inside that directory:

---

name: pr-description

description: Writes pull request descriptions. Use when creating a PR,

writing a PR, or when the user asks to summarize changes for a pull request.

---
When writing a PR description:

1. Run `git diff main...HEAD` to see all changes on this branch

2. Write a description following this format:

## What
One sentence explaining what this PR does.

## Why
Brief context on why this change is needed.

## Changes
- Bullet points of specific changes made
- Group related changes together
- Mention any files deleted or renamed

Restart Claude Code.

The next time you say “write a PR description for my changes,” Claude will recognize the request, load the Skill, and follow the template — same format every time.

Dive deeper into the topic on Packt’s Medium handle.

🕸️ Turn Connected Data Into Better AI Answers

Many RAG systems fail not because the model lacks knowledge, but because retrieval lacks structure. Join Bruno Gonçalves for a practical workshop on GraphRAG and learn how to build AI applications that can reason across relationships, answer multi-hop questions, and generate more trustworthy responses.

🎟️ Save 35% on your ticket with the DataPro community offer.

Discover how leading teams are tackling AI hallucinations and building systems that can reason across complex business data.

Save Your Spot Now!

Data Science & ML Research Roundup

◾ How Loka Built a Natural, Low-Latency Voice Agent with Amazon Nova 2 Sonic: Loka built a conversational AI agent with Amazon Nova 2 Sonic to eliminate the slow, robotic experience of traditional voice assistants. By using native speech-to-speech processing, the solution delivers faster responses, higher speech reasoning accuracy, and lower costs. Prompt engineering further boosted conversational quality, making the AI more natural, accurate, and production-ready for customer support at scale.

◾ Baidu Releases Unlimited OCR, a 3B Model That Keeps the KV Cache Flat for Long-Document Parsing: Baidu has open-sourced Unlimited OCR, a 3B-parameter model that solves a major OCR bottleneck by keeping memory usage constant, enabling efficient parsing of long documents in a single pass. Built on DeepSeek OCR, it delivers higher accuracy, faster throughput, and lower latency, making it well suited for large-scale document processing, transcription, and multimodal parsing workflows.

◾ Huntington Bank: Redacting sensitive data from 400M+ documents with AWS Huntington Bank cut a multi-year compliance project down to months by building a scalable AWS-powered pipeline to detect and redact sensitive data across 400 million documents. Using Amazon Textract, SageMaker, Step Functions, and Lambda, the solution achieved over 95% redaction accuracy while securely processing documents at massive scale with high concurrency and PCI DSS compliance.

◾ Datalab Releases lift: A 9B Open-Weights Vision Model That Extracts Structured JSON From PDFs Using Schemas Datalab has introduced lift, a 9B open-weights vision model that extracts structured JSON from PDFs and images using JSON schemas. It achieves 90.2% field accuracy while processing multi-page documents in a single pass, making it one of the strongest self-hostable extraction models. Schema-constrained decoding reduces hallucinations and enables reliable document automation workflows.

◾ Build a healthcare appointment agent with Amazon Nova 2 Sonic AWS has published a reference architecture for building healthcare appointment agents with Amazon Nova 2 Sonic and Bedrock AgentCore. The speech-to-speech AI authenticates patients, confirms or reschedules appointments, collects pre-visit information, and escalates to staff when needed. Built with serverless AWS services and healthcare-specific tools, it enables natural, low-latency voice interactions that can help reduce appointment no-shows at scale.

◾ Gradium Launches stt-translate and s2s-translate, Real-Time Speech Translation Models Beating gpt-realtime-translate on Accuracy and Latency: Gradium has launched stt-translate and s2s-translate, real-time speech translation models that combine transcription, translation, and speech output into a faster two-model pipeline. Supporting five languages and 20 pairs, the models claim stronger BLEU accuracy than GPT and Gemini alternatives, 3-second average latency, live browser streaming, and voice control, including cloning for multilingual meetings, agents, and dubbing.

◾ Open models, global networks: How AT&T and GSMA are accelerating innovation with Gemma Google Cloud and GSMA have introduced Open Telco AI, an initiative built on Gemma models to bring domain-specific AI to telecom networks. Fine-tuned on specialized telecom data, the open OTel models outperform larger general-purpose models on network tasks while reducing hallucinations through RAG. The project aims to accelerate AI-driven network automation, self-healing systems, and telecom-grade AI adoption.

◾ How to Design an OpenHarness Style Agent Runtime with Tools, Memory, Permissions, Skills, and Multi-Agent Coordination: This tutorial breaks down how to build an OpenHarness-style agent runtime from scratch, exposing the full mechanics behind modern agent systems. It walks through tool schemas, permissions, lifecycle hooks, memory, skills, retries, cost tracking, context compaction, and multi-agent coordination, giving developers a runnable framework for understanding how agents reason, call tools, manage state, and complete tasks.

◾ Query logs and traces with SQL in Observability Analytics: Google Cloud has rebranded Log Analytics as Observability Analytics, adding GA support for SQL-based analysis of logs and traces in a unified workspace. Developers can now join telemetry with business data to troubleshoot applications, optimize AI agents, and identify performance bottlenecks using BigQuery-powered SQL, while the new Observability API enables programmatic access for agentic workflows and automation.

◾ DFlash Speculative Decoding Drafts Whole Token Blocks in Parallel for Up to 15x Higher Throughput on NVIDIA Blackwell: Researchers at UC San Diego have introduced DFlash, a speculative decoding method that generates entire token blocks in parallel instead of one token at a time. By combining lightweight diffusion drafting with autoregressive verification, DFlash delivers up to 6× faster lossless inference in research benchmarks, while NVIDIA reports up to 15× higher throughput on Blackwell GPUs for latency-sensitive AI workloads such as coding agents and reasoning models.

◾ Python UDF in BigQuery, now generally available: Google Cloud has announced the general availability of BigQuery Managed Python UDFs, enabling developers to run custom Python code and popular libraries like NumPy, pandas, and scikit-learn directly within BigQuery SQL. The serverless feature eliminates infrastructure management while supporting vectorized execution, configurable compute resources, external API integration, and production-grade monitoring for advanced analytics and machine learning workflows.

See you next time!

Welcome to DataPro #174.

This week, we’re exploring a question that sits at the heart of enterprise AI: Why do RAG systems still hallucinate when the answer is already in the data? Former JPMorgan Chase VP of Data Science Bruno Gonçalves unpacks the retrieval challenges behind many RAG failures and explains why GraphRAG is gaining traction for multi-hop reasoning, corpus-wide analysis, and explainable AI.

If you're building AI applications today, these challenges will likely feel familiar. From unreliable retrieval and reasoning gaps to measuring AI ROI, scaling agents, and managing ever-growing context windows, organizations are moving beyond experimenting with AI and into the harder work of making it reliable, explainable, and production-ready. Many of the stories in this week's edition explore that transition from prototype to practice.

As part of our commitment to bringing practical, practitioner-led learning to the DataPro community, we’re also pleased to partner with GrowthSchool and Outskill. Together with AI communities around the world, we're helping surface high-quality learning opportunities, from free modules and workshops to expert-led training and hands-on sessions hosted by the Packt Virtual Conference team.

One such opportunity this week is a free workshop by Outskill on turning Claude into a 24/7 AI copilot, covering Skills, Connectors, Cowork, vibe coding, and workflow automation. It’s a practical, hands-on way to learn how experienced practitioners are integrating AI into their daily workflows. And if you can’t join live, every session is recorded and shared with registered attendees, so you can learn at your own pace and revisit the material whenever you need it.

Make Claude Your 2nd Brain That works 24/7 By Mastering In It 16 hours

Register Now!

This Week’s Highlights

🔬 OpenAI’s LifeSciBench Reveals How Far AI Still Is From Scientific Reasoning
A new 750-task benchmark built by 173 PhD scientists shows even the best models pass just 36% of real-world life sciences research tasks.

🤖 Vercel Open-Sources Eve, the Agent Framework Behind 100+ Production AI Agents
A filesystem-first framework where agents are simply directories of tools, skills, channels, schedules, and subagents.

📈 Google’s DORA Research Says AI ROI Starts With a Productivity Dip
Why the “J-Curve” of AI adoption is normal, and how leading engineering teams are measuring the business impact of AI.

⚡ Amazon SageMaker Async Inference Removes Mandatory S3 Uploads
A new inline payload feature cuts latency, simplifies architecture, and reduces costs for asynchronous AI workloads.

🧠 MiniMax-M3 Launches With 1 Million Context and Sparse Attention Breakthroughs
The 428B-parameter multimodal model delivers 9× faster prefilling and 15× faster decoding at million-token scale.

🚀 GLM-5.2 Raises the Bar for Open-Source Agentic Coding Models
Featuring a stable 1M-token context window, improved coding benchmarks, and an MIT license.

💻 Run a Powerful Coding Model Locally With Just 4.5GB of VRAM

Gemma4-12B-Coder combines verified reasoning traces and coding expertise in a lightweight, fully local package.

Let’s get into it.

Cheers,

Merlyn Shelley,

Growth Lead, Packt.

Why RAG Hallucinates With the Answer in Hand

Written by Bruno Gonçalves, data scientist, educator, and former Vice President of Data Science and Finance at JPMorgan Chase.

Vector RAG has a strange failure pattern. The answer sits in your documents; the retriever returns something plausible but still gets it wrong. The fault sits with your retrieval scaffolding that cannot see structure, and it shows up in three predictable ways.

1. Multi-hop questions. Ask about the indirect suppliers of Company X. The retriever returns chunks that mention Company X, but the indirect suppliers live in documents that never name it. One document says firm A supplies firm B. Another says firm B supplies firm C. No single chunk holds the chain, so no similarity score will surface it. Cosine similarity has no notion of A → B → C. A knowledge graph stores those links as data: ndes represent Entities and their relationships are edges. With this formalism, the question is just a two-hop traversal that runs in milliseconds.

2. Global questions. “What are the main themes across these 500 documents?” Top-k retrieval grabs the ten chunks closest to the query and ignores the other ten thousand. The GraphRAG paper named the problem: global questions are summarization tasks, not retrieval tasks. The fix is to build the a map first, then compute the answers from the map. Extract entities and relations from every chunk, cluster them into communities, summarize each community, then reduce those summaries into one response. On million-token test corpora, this beat vector RAG on both the comprehensiveness and the diversity of its answers.

3. Explainability. Vector RAG can show you a relevant chunk. It cannot show you a reason. A chunk might score 0.87 on query similarity, but that is the whole story. Graph RAG returns the chain of entities and relations behind the answer, hop by hop, back to the source text. In finance, healthcare, and law, that audit trail decides whether the system ships.

The thread running through all three failure modes is the same. Hallucination in RAG is rarely a generation problem. It is a retrieval problem wearing a generation costume. The model invents connections precisely where the retriever failed to supply them: across hops, across the whole corpus, and across the gap between an answer and its evidence. Hand the model real structure instead of a stack of similar-looking chunks, and the invented connections mostly disappear. The documents had the answer all along. The graph is what lets the system find it.

Full GraphRAG is expensive to index. The pipeline makes LLM calls for every chunk, every entity description, and every community summary. One practitioner account puts the price of indexing a single five-gigabyte legal dataset at $33,000 in early 2024. That number kept plenty of teams on plain vectors, whatever the quality argument said.

LazyGraphRAG attacked the cost in November 2024. It skips upfront summarization entirely, relying instead on noun-phrase extraction and co-occurrence statistics, with zero LLM calls, so it costs about the same as building a vector index, or about 0.1% of full GraphRAG. The graph work shifts to query time, where an iterative search builds only the structure a given question needs. Answer quality matches GraphRAG global search at more than 700 times lower query cost, and the cost objection to graphs largely dissolved.

Want to Build a GraphRAG System?

Reading about GraphRAG is one thing. Building one is another.

Start with [our walkthrough], where you’ll learn how to turn 2,000 news articles into a searchable knowledge graph. Then take the next step on July 11 with our live Production GraphRAG Workshop, where you’ll build a complete GraphRAG chatbot from raw Wikipedia data in just 3.5 hours.

You’ll learn entity and relationship extraction with spaCy and REBEL, graph construction with NetworkX, hybrid graph-plus-vector retrieval, and grounded generation with an LLM. You’ll also receive the recording, source code, slides, and a certificate, so you can revisit the material long after the session ends.

🎟️ Exclusive for DataPro readers: Save 35% on your ticket.

Whether you’re exploring GraphRAG for the first time or looking to improve an existing RAG pipeline, this workshop will give you a practical framework for tackling the multi-hop questions traditional retrieval systems often miss.

Basic Python and Docker knowledge are all you need. Seats are limited.

Book Your Spot & Save 35%

Data Science & ML Research Roundup

◾OpenAI Releases LifeSciBench, a 750-Task Benchmark Grading AI Models on Real Life-Science Research With Expert-Written Rubric: OpenAI has launched LifeSciBench, a new benchmark designed to test how well AI models perform real-world life sciences research, not just fact recall. Built by 173 PhD-level scientists, it includes 750 expert-authored tasks spanning 7 research workflows and 7 biological domains, supported by 1,062 artifacts and graded against 19,020 rubric criteria. Results show significant headroom: the top-performing model, GPT-Rosalind, achieved a 36.1% pass rate and 0.576 normalized score, while GPT-5.5 scored 25.7%. Models struggled most with artifact-heavy tasks, experimental design, and precise sequence generation, highlighting how far AI remains from matching expert scientific reasoning.

◾Vercel Releases Eve: An Open-Source AI Agent Framework Where Each Agent is a Directory of Files Mapped to Capabilities: Vercel has open-sourced eve, the AI agent framework powering 100+ production agents internally. Built around a filesystem-first approach, each agent is simply a directory where files map to capabilities like tools, skills, channels, schedules, and subagents. The framework ships with built-in durability, sandboxed execution, human approvals, secure MCP/OpenAPI connections, multi-channel deployment, and observability. Vercel reports real-world adoption at scale: d0 handles 30,000+ data queries monthly, Vertex autonomously resolves 92% of support tickets, and its autonomous SDR delivers a reported 32× ROI. Agents can be scaffolded with a single command and deployed unchanged from local development to production.

◾How to measure the business value of generative AI: Google’s latest DORA research on AI-assisted software development argues that proving AI ROI requires more than measuring productivity gains. The report highlights a “J-curve” effect, where teams often experience an initial productivity dip due to learning new workflows, increased code-review demands, and bottlenecks in testing and approvals. While 90% of surveyed developers now use AI at work, financial outcomes vary widely, with successful organizations investing in workflow and cultural changes alongside tooling. DORA recommends building explicit ROI models that account for both visible costs and hidden adoption challenges, using frameworks and calculators that link AI investments to productivity, security, developer experience, and business growth.

◾Amazon SageMaker AI Async Inference now supports inline request payloads: Amazon SageMaker AI has added inline payload support for Async Inference, allowing developers to send request data directly through the new Body parameter instead of first uploading inputs to Amazon S3. The feature supports payloads up to 128 KB, eliminates an extra network round-trip, removes S3 upload costs and IAM dependencies, and simplifies client code while maintaining existing output behavior through S3. Designed to be backward compatible, it works with existing async endpoints without model or container changes and is available across 31 AWS commercial regions. AWS recommends inline payloads for smaller JSON and structured-data workloads, while larger inputs such as images and audio should continue using S3-based InputLocation workflows.

◾MiniMaxAI/MiniMax-M3: MiniMax has released MiniMax-M3, a native multimodal foundation model with a massive 1 million-token context window, 428B total parameters, and 23B activated parameters. Unlike models that add multimodal capabilities later, M3 is trained on text, images, and video from the first training step, enabling deeper cross-modal reasoning. Its new MiniMax Sparse Attention (MSA) architecture dramatically improves long-context efficiency, delivering 9× faster prefilling, 15× faster decoding, and reducing per-token compute costs to 1/20th of its predecessor (M2) at 1M-token context. The model also targets advanced coding and agentic workflows, achieving frontier-level results on long-horizon coding and cowork benchmarks, and supports three reasoning modes—enabled, adaptive, and disabled—to balance accuracy, latency, and throughput. Since its release, the open-source model has already recorded 56,000+ monthly downloads on Hugging Face.

See you next time!

Welcome to DataPro #173 👋

As we embark on this week’s edition, we want to extend a special welcome to all our new readers joining us from the AI Skills Conference organized by Community Sprints. We’re excited to have you as part of the Packt DataPro community — one of the industry’s largest and fastest-growing Data Science, Machine Learning, and AI newsletters with 125K+ data professionals worldwide.

At DataPro, we go beyond headlines. Every week, we break down the real operational bottlenecks shaping modern AI systems, explore expert-led solutions, and highlight the most important advancements across Data Science, ML engineering, GenAI, MLOps, and enterprise AI infrastructure.

As part of the Packt ecosystem, you’ll also gain access to exclusive perks, expert sessions, community connections, research-driven insights, curated learning resources, and future goodies designed specifically for AI practitioners, data scientists, engineers, and decision-makers. So, stay tuned — there’s a lot ahead.

This week’s edition also features an insightful expert-led deep dive from AI Engineer Esther Guru, who explains why most ML systems fail in production and what it actually takes to build reliable, scalable machine learning infrastructure beyond model training. From data drift and monitoring to operational reliability and deployment trade-offs, the session delivers practical lessons every ML engineer should understand before shipping AI systems into the real world.

Alongside that, this week’s research and engineering roundup covers some of the most important developments shaping the future of AI infrastructure and agentic systems:

• Stability AI’s Stable Audio 3, introducing scalable latent diffusion for long-form audio generation

• Google’s Agentic Enterprise vision, featuring Gemini 3.5, Gemini Omni, Antigravity, and Gemini Spark

• OmniVoice Studio, an open-source local alternative to ElevenLabs for voice AI workflows

• Google Agent Executor, a new distributed runtime for reliable long-running AI agents

• NVIDIA’s Gated DeltaNet-2, improving long-context reasoning in linear attention architectures

• Google AI Edge Portal, enabling on-device LLM benchmarking across 120+ Android devices

• AWS Bedrock AgentCore, powering scalable multi-agent orchestration, observability, and agentic commerce infrastructure

From multimodal AI and distributed agents to edge inference, voice AI, and production ML systems, this edition captures the accelerating shift toward scalable, operational AI engineering.

Let’s dive in. 🚀

Before we get into this week’s developments, join Amy Chen and Sujeet Mishra this weekend for an intensive hands-on workshop on building reliable GenAI systems with AI evaluations, observability, testing, and production-grade workflows. Learn how leading AI teams debug hallucinations, evaluate prompts and agents, monitor LLM systems in production, and build scalable evaluation pipelines for real-world AI applications.

If you’re building RAG systems, copilots, AI agents, or enterprise GenAI products, this session will provide practical frameworks and workflows you can immediately apply to improve reliability, performance, and shipping confidence.

⚡ Few seats left — claim your 15% discount by registering through the link below (discount already enabled).

Register Now

Cheers,

Merlyn Shelley,

Growth Lead, Packt.

This Week’s Sponsor

Grow your Mac app with Setapp

Get up to 30K unique impressions right after launch while Setapp handles distribution, billing, licensing, taxes, and customer support.

You build great software. Setapp helps you grow revenue and reach the right users faster.

Join Setapp

Why Most ML Projects Fail in the Real World

AI engineer Esther Guru explains what it really takes to build reliable ML systems beyond just training models.

Beyond the Model: Building Real World Machine Learning Systems | From Training to Deployment — YouTube

Machine learning has become one of the defining technologies of modern software. From recommendation engines and fraud detection systems to healthcare applications and financial forecasting, ML is now deeply embedded into the products people use every day.

But despite all the excitement around artificial intelligence, the reality of deploying machine learning systems is far more complicated than most organizations expect.

Many machine learning projects never make it into production. Others reach deployment but slowly fail over time because the real world changes faster than the systems behind them. In many cases, companies spend enormous amounts of money building sophisticated models only to discover that the actual problem was never about the model itself.

That was the focus of a recent Packt DataML Talk hosted by Abhishek Kaushik, where AI engineer and electrical engineer Esther Guru explored the realities of engineering machine learning systems for production.

Instead of focusing only on algorithms, Esther explained how real-world ML systems behave outside research environments, why production systems fail, and what engineers need to understand before deploying machine learning at scale.

The session offered an important reminder:

Machine learning in production is not just about building models. It is about building systems.

When Should You Use Machine Learning?

One of the biggest mistakes organizations make is assuming that every problem requires AI.

According to Esther, machine learning should only be used when a problem satisfies a few important conditions.

First, the system must actually involve learning. If a problem can already be solved through a fixed equation or a simple rule-based system, machine learning may only add unnecessary complexity.

For example:

Ohm’s Law already provides a direct mathematical relationship between voltage, resistance, and current. There is no need for a neural network to solve such a problem.

Similarly, if a sports website simply wants to display the top-performing football players based on statistics, a sorting query is enough. Building a machine learning model for that task would be excessive.

Machine learning becomes useful when relationships between variables are too complex for traditional methods.

Problems such as predicting customer churn, forecasting housing prices, detecting fraud, or predicting the outcome of sports tournaments involve large numbers of interacting variables. These are the kinds of problems where ML systems become valuable.

Esther described machine learning as:

The process of learning complex patterns from existing data and using those patterns to make predictions on unseen data.

That definition highlights the most important requirement for ML systems: the ability to generalize.

A successful model is not one that memorizes training data. It is one that performs reliably on new data it has never seen before.

Research ML vs Production ML

One of the most valuable parts of the session was Esther’s explanation of the difference between research machine learning and production machine learning.

Many engineers assume that if a model performs well during experimentation, it is ready for deployment.

In reality, research ML and production ML operate under completely different conditions.

In research environments, engineers typically work with clean datasets, static files, and controlled experiments. Most datasets are already prepared and structured. Training happens in predictable conditions.

Production environments are very different.

Real-world data is messy, incomplete, noisy, and constantly changing. Data pipelines may fail. User behavior shifts. External events change the environment the model operates in.

This difference becomes especially important when discussing objectives.

In research, the main goal is usually simple: build the most accurate model possible.

In production, however, multiple teams have competing priorities.

An ML engineer may want a highly sophisticated deep learning system with maximum accuracy. Meanwhile, the business team may care more about infrastructure costs, scalability, and speed.

Production systems also prioritize latency.

Users expect predictions instantly. If a recommendation engine or AI assistant takes too long to respond, users abandon the platform.

This means production systems must optimize not only for accuracy, but also for:

• speed

• scalability

• cost efficiency

• reliability

• infrastructure performance

This is why production engineering matters just as much as model design.

Why Most ML Projects Fail

One of the most striking insights from the Packt Talk was how frequently ML projects fail.

Esther referenced industry findings showing that a large percentage of machine learning initiatives never even reach production. Even among deployed systems, many degrade significantly within months.

There are several reasons for this.

Poor Problem Framing

Many organizations begin AI projects without clearly defining the actual business problem.

Teams often say things like:

• “We want to use AI.”

• “We need a machine learning strategy.”

• “Can we automate this?”

But vague business goals lead to vague ML objectives.

Without clear alignment between technical teams and business requirements, organizations end up building systems that never deliver meaningful value.

According to Esther, successful ML projects begin with proper problem framing.

Teams must understand:

• whether ML is even necessary

• what success actually looks like

• which business metrics matter most

• what constraints exist in production

Without this clarity, even technically strong models can fail.

Catch the complete story and technical insights on Packt’s Medium handle.

Data Science & ML Research Roundup

◾Stability AI Releases Stable Audio 3: A Family of Fast Latent Diffusion Models for Audio Generation and Editing: Stability AI has open-sourced Stable Audio 3, a new family of latent diffusion models for generating and editing stereo audio at 44.1 kHz. The release includes scalable models, variable-length generation, fast inference, inpainting, and a novel SAME autoencoder with 4096× compression. SA3 supports music and SFX creation, achieves strong benchmark scores, and enables long-form audio generation efficiently on consumer hardware.

◾Innovations from Google I/O 26 on Google Cloud: At Google I/O 2026, Google expanded its vision for the “Agentic Enterprise,” where AI agents move beyond chatbots to autonomously execute workflows across apps, data systems, and developer environments. The announcement introduced Gemini 3.5 Flash for advanced reasoning and coding, Gemini Omni for multimodal video generation, Antigravity for enterprise-scale agent orchestration, and Gemini Spark, a personal AI work agent integrated across Workspace and business tools. Together, these launches signal Google’s push toward AI-native productivity, software development, and enterprise automation.

◾Meet OmniVoice Studio: A Local, Open-Source Alternative to ElevenLabs. OmniVoice Studio is emerging as a compelling open-source alternative to ElevenLabs, offering fully local voice AI capabilities without subscriptions or cloud processing. The desktop app supports voice cloning, video dubbing, diarization, dictation, and multilingual TTS across 646 languages. Built with FastAPI, WhisperX, Demucs, and Pyannote, it also includes an MCP server for integrations with Claude, Cursor, and custom AI workflows.

◾Agent Executor, Google’s distributed Agent Runtime: Google has open-sourced Agent Executor, a runtime standard designed to make long-running AI agents more reliable, scalable, and production-ready. Built for the emerging “agentic enterprise,” it introduces durable execution, secure sandboxing, session consistency, trajectory branching, and distributed deployment support. Combined with Agent Substrate for Kubernetes-scale orchestration, Google is positioning Agent Executor as foundational infrastructure for running millions of enterprise AI agents across hybrid environments.

◾NVIDIA AI Releases Gated DeltaNet-2: A Linear Attention Layer That Decouples Erase and Write in the Delta Rule. NVIDIA has introduced Gated DeltaNet-2, a new linear attention architecture designed to improve long-context memory handling in AI models. By separating memory “erase” and “write” operations into independent channel-wise gates, the model achieves stronger retrieval accuracy and reasoning performance than Mamba-2, KDA, and prior DeltaNet variants. Trained at 1.3B parameters on 100B tokens, it also maintains efficient linear-time scaling for long-running sequence processing.

◾Benchmark LLMs on-device with AI Edge Portal: Google has expanded AI Edge Portal with new tools for benchmarking and debugging on-device LLMs across more than 120 Android device types. The update helps developers optimize latency, memory usage, and inference performance for mobile AI workloads using metrics like decode speed and initialization time. Google also introduced Model Explorer, a visualization tool for analyzing model graphs, quantization issues, and hardware compatibility in edge AI deployments.

◾Vibe-coded AI Studio apps with Firestore, Firebase, Cloud SQL: Google is expanding AI Studio into a full-stack “vibe coding” platform, enabling developers to build and deploy AI-powered applications directly to Google Cloud without needing billing setup or infrastructure management. The update adds support for Cloud SQL, Firestore, Firebase Auth, and Workspace integrations, while AI agents can now automatically provision databases, generate schemas, configure authentication, and deploy apps through natural language prompts.

◾Technical deep dive: AgentCore payments and innovation in agentic commerce. Amazon has introduced Bedrock AgentCore Payments, a managed infrastructure layer that enables AI agents to autonomously make secure microtransactions for APIs, MCPs, and digital services. Designed for the emerging “agentic economy,” the platform handles payment orchestration, wallet security, stablecoin support, spending guardrails, and observability through a single API. The release positions AgentCore as foundational infrastructure for scalable, enterprise-grade autonomous AI commerce.

◾Build highly scalable serverless LangGraph multi-agent systems in AWS with Amazon Bedrock AgentCore: AWS has introduced a serverless multi-agent AI architecture that combines LangGraph orchestration with Amazon Bedrock AgentCore Memory and Observability. The framework uses AWS Lambda, Step Functions, and Bedrock to build scalable AI workflows with persistent memory, real-time telemetry, and parallel agent coordination. Demonstrated through a marketing campaign review system, the solution highlights how enterprises can operationalize reliable, observable, and production-ready multi-agent AI systems on AWS.

◾Build high-performance generative AI systems with Strands Agents, NVIDIA NIM, and Amazon Bedrock AgentCore. AWS and NVIDIA have introduced a production-ready multi-agent AI architecture that combines NVIDIA NIM for GPU-accelerated inference, Strands Agents for orchestration, and Amazon Bedrock AgentCore for memory and observability. Designed for scalable enterprise AI systems, the framework supports parallel agent execution, persistent context, real-time monitoring, and serverless deployment, enabling organizations to build reliable, high-performance generative AI workflows at scale.

See you next time!

Welcome to DataPro #172 — this week, we explore the AI skills, agents, and production systems shaping 2026, alongside insights from the upcoming free AI Skills Conf happening tomorrow.

As AI rapidly moves beyond prompting into agentic workflows, context engineering, multimodal systems, and operational AI infrastructure, the real advantage now belongs to professionals who can build and deploy AI systems, not just use them. That’s why this edition features an expert-led deep dive from AI engineer Hari Prasad Renganathan on the 10 essential AI skills professionals need in 2026, covering RAG, LLMOps, AI evaluation, autonomous agents, and more.

The same shift is driving massive interest in the AI Skills Conf, where 6,000+ AI professionals are already registered to hear leaders from Google DeepMind, AWS, Meta, Spotify, SAP, DoorDash, and Scale AI discuss topics like “How to Become Irreplaceable with AI,” “Building Your AI Chief of Staff,” “Vibe Coding,” and “The Context Engineering and Agentic Memory.”

This is part of Packt DataPro’s Knowledge Partner initiative with global AI communities, complementing the hands-on workshops and technical deep dives delivered by the Packt Virtual Conference team. I’ll also be joining the “AI ROI Reality Check” panel alongside leaders from Spotify and SAP to discuss where enterprises are seeing measurable business value from AI today.

🚀 Registration is free, and even if you miss the live sessions, all recordings, AI workflows, templates, guides, and bonus resources will be delivered directly to your inbox afterward.

Register free and secure your spot

In this week’s highlights:

• OpenAI launched Daybreak for AI-driven cyber defense workflows
• BASF is using evolutionary AI to optimize global supply chains
• Sakana AI and NVIDIA introduced sparse transformer architectures for faster LLMs
• Google Cloud unveiled Cloud Storage Rapid for AI-scale infrastructure
• AWS and Exa demonstrated autonomous web research agents powered by real-time retrieval

The AI era is no longer just about using tools. It is about understanding the systems behind them.

Cheers,

Merlyn Shelley,

Growth Lead, Packt.

“Most Developers Are Learning AI the Wrong Way”: Hari Prasad Renganathan’s 10 Essential AI Skills for 2026

From prompt engineering and AI agents to LLMOps and multimodal systems, here’s the practical roadmap developers and AI professionals need to stay ahead in 2026.

Artificial Intelligence is no longer a futuristic concept reserved for research labs or billion-dollar tech companies. It is now woven into the everyday workflows of developers, startups, enterprises, product teams, and even non-technical professionals. In 2026, the biggest challenge is not whether AI will transform industries. It already has. The real question is this:

What should developers and AI professionals learn next to stay relevant?

The pace of AI innovation is moving faster than most people can keep up with. New frameworks emerge every month. AI models become more capable every quarter. Entire workflows are being automated in ways that seemed impossible just a few years ago. Yet amid all the noise, there are a handful of foundational topics that truly matter.

In a recent webinar hosted by Packt Talks, AI engineer and founder Hari Prasad Renganathan shared a practical roadmap for developers looking to thrive in the AI era. With experience spanning Columbia University, AI leadership roles at YC-backed startups, and building real-world AI products, Hari distilled the overwhelming AI landscape into ten critical topics every professional should understand.

Hosted by our Growth Lead, Abhishek Kaushik, the session focused not on hype, but on practical, production-ready AI skills that companies are already adopting today.

This article expands on those ideas into a complete guide for developers, engineers, and AI professionals preparing for the next wave of intelligent systems.

1. Advanced Prompt Engineering

Every AI workflow starts with prompts.

While basic prompting is now common knowledge, advanced prompt engineering has become a serious professional skill. Developers who understand how to structure prompts effectively can dramatically improve the quality, reliability, and relevance of AI outputs.

Hari emphasized an important point during the session:

“The best prompt you can give is better context.”

That single idea captures the essence of modern prompt engineering.

Today’s professionals must go beyond simple instructions and learn how to provide context, examples, reasoning patterns, and clear output expectations. Techniques such as few-shot prompting, chain-of-thought prompting, and structured outputs have become critical in enterprise AI systems.

Structured outputs are especially important because they allow AI systems to return clean JSON, XML, or CSV responses that integrate directly into applications and workflows. Prompt engineering may look simple on the surface, but in production systems, it directly impacts performance, cost, and user trust.

2. Retrieval-Augmented Generation (RAG)

Large Language Models are powerful, but they have limitations.

One major limitation is that they cannot naturally access an organization’s private knowledge base. That is where Retrieval-Augmented Generation, or RAG, becomes essential.

RAG allows AI systems to retrieve relevant information from documents, databases, PDFs, internal systems, or even web sources before generating a response. Instead of relying purely on pretrained knowledge, the model becomes context-aware and personalized.

Hari described RAG as one of the most practical skills AI engineers should learn because it bridges the gap between generic AI and real-world enterprise applications.

At its core, RAG works by converting information into vector embeddings, storing them in databases, retrieving the most relevant chunks during a query, and feeding those results back into the model. This creates AI systems that feel dramatically smarter and more useful.

Most enterprise AI tools today rely heavily on some form of retrieval system because businesses care less about generic intelligence and more about contextual accuracy.

3. Agentic AI Systems

If prompt engineering is the first level of AI maturity, agentic AI is the next major leap.

Traditional AI systems behave like assistants that answer questions. Agentic systems behave more like autonomous workers capable of completing tasks on behalf of users.

Hari described this transformation perfectly during the webinar:

“What if we can give hands and legs to that AI?”

That is the essence of AI agents.

Instead of simply generating text, agents can send emails, interact with APIs, manage workflows, run code, query databases, and use external tools. Developers can now create systems where AI decides what steps should happen next rather than following rigid workflows.

Modern frameworks such as LangGraph, CrewAI, and AutoGen are enabling developers to build increasingly sophisticated agentic systems. Hari specifically highlighted LangGraph as his preferred framework for production-grade AI workflows.

However, he also offered a realistic perspective. Despite the hype surrounding AI agents, most businesses are still primarily using prompt engineering and RAG because those systems are easier to deploy and maintain. Agentic AI is powerful, but the industry is still discovering its most commercially viable use cases.

4. Fine-Tuning Large Language Models

A few years ago, fine-tuning was considered one of the most important areas in AI development. Today, it has become more specialized.

Modern foundation models are already extremely capable. Many business problems can now be solved using better prompts, retrieval systems, and workflow orchestration without retraining models.

Still, fine-tuning remains valuable for highly customized applications.

Fine-tuning involves taking an existing model and retraining it on domain-specific data so it behaves in a more tailored way. Industries such as finance, healthcare, and legal services often benefit from fine-tuned systems because they require strict consistency and domain-specific behavior.

Hari pointed out that fine-tuning introduces operational complexity. Beyond training costs, organizations must also handle hosting, scaling, monitoring, and infrastructure management. That is why many companies prefer retrieval-based systems before committing to full fine-tuning pipelines.

Even so, developers should still understand fine-tuning because it remains an important tool for advanced AI applications. Platforms like Hugging Face have made experimentation with open-source models significantly more accessible.

5. LLMOps and AI Production Systems

Building an AI demo is easy. Deploying AI reliably at scale is the real challenge.

That challenge is where LLMOps comes in.

LLMOps focuses on operationalizing AI systems in production environments. Developers must think beyond experimentation and address real-world engineering concerns such as scalability, latency, reliability, monitoring, and governance.

Hari highlighted a key difference between traditional software and AI systems. Traditional systems are deterministic, meaning the same input produces the same output every time. AI systems behave differently. Even the same prompt can produce slightly different responses at different times.

This creates entirely new operational challenges.

Production-grade AI systems therefore require observability tools, prompt versioning systems, human review workflows, and robust monitoring pipelines. Tools like LangSmith and LangFuse are increasingly becoming essential for debugging and tracking AI behavior in production.

Without proper LLMOps practices, even impressive AI products quickly become unstable, expensive, and difficult to maintain.

6. AI Evaluation Frameworks

One of the hardest problems in AI engineering is evaluation.

How do you determine whether an AI system is actually performing well?

Traditional software testing relies on predictable outputs, but AI systems are probabilistic by nature. This makes evaluation significantly more difficult.

Hari explained that a major portion of his professional work involved building evaluation frameworks for AI systems. These frameworks help teams measure output quality, detect hallucinations, monitor performance degradation, and maintain alignment with business goals.

One popular approach is “LLM-as-a-Judge,” where another model evaluates the responses generated by the primary system. Developers also use benchmark testing, human review loops, and ranking systems to assess quality.

Evaluation remains one of the least solved yet most critical areas in AI engineering. As companies deploy more AI systems into production, professionals who understand evaluation methodologies will become increasingly valuable.

7. Multimodal AI

AI is no longer limited to text.

Modern AI systems can now process images, videos, documents, screenshots, diagrams, and audio inputs alongside traditional text interactions. This shift toward multimodal AI is reshaping how humans interact with machines.

Today’s models can analyze visual data, extract information from documents, interpret diagrams, and even generate media content. Industries such as healthcare, education, design, and autonomous systems are already benefiting from these capabilities.

Hari noted that while most current commercial applications remain text-focused, multimodal systems represent the next major evolution in AI interfaces. Continue reading on the Packt Medium page.

Data Science & ML Research Roundup

◾How Amazon Finance streamlines regulatory inquiries by using generative AI on AWS: Amazon FinTech teams built a scalable AI-powered regulatory response system using Amazon Bedrock, Claude Sonnet 4.5, OpenSearch, and AWS Lambda. The platform automates document retrieval, multi-turn conversations, and compliance workflows across thousands of regulatory inquiries. With RAG pipelines, vector search, and observability through Langfuse and OpenTelemetry, the system improves accuracy, scalability, security, and response speed for complex compliance operations.

◾Build a Hybrid-Memory Autonomous Agent with Modular Architecture and Tool Dispatch Using OpenAI: This tutorial walks through building a hybrid-memory autonomous AI agent using OpenAI, BM25, vector search, and modular tool orchestration. The architecture combines long-term memory, semantic retrieval, tool calling, and autonomous reasoning into a production-ready framework. With runtime tool swapping, multi-turn memory, and RAG workflows, the system demonstrates how AI agents can reason, remember, and act with scalable autonomy.

◾Automate schema generation for intelligent document processing: AWS has introduced multi-document discovery for the IDP Accelerator, helping teams classify unknown document collections and generate extraction schemas automatically. Using Amazon Bedrock, Cohere Embed v4, k-means clustering, and Strands Agents, the feature groups documents by visual structure, creates JSON schemas, and flags overlaps through quality reports, reducing manual setup for large-scale intelligent document processing.

◾MedAIBase/AntAngelMed: AntAngelMed is a 100B open-source medical LLM built on a highly efficient MoE architecture, activating just 6.1B parameters while matching ~40B dense model performance. Developed by MedAIBase and healthcare partners, it leads benchmarks like HealthBench and MedBench, offering advanced diagnostic reasoning, safety-focused medical intelligence, 128K context support, and high-speed inference exceeding 200 tokens per second.

◾Daybreak | OpenAI for cybersecurity: OpenAI Daybreak is a cybersecurity initiative designed to embed AI-driven defense directly into software development workflows. Combining frontier OpenAI models with Codex Security, the platform helps teams identify vulnerabilities, validate patches, automate remediation, and improve threat modeling at scale. With tiered cyber-focused GPT-5.5 access levels and partnerships across major security firms, Daybreak aims to make software resilient by design through continuous AI-assisted defense.

◾Sparser, Faster, Lighter Transformer Language Models: Sakana AI and NVIDIA introduced TwELL, a sparse data format and custom CUDA kernels that make transformer LLMs faster, lighter, and more energy efficient. By leveraging activation sparsity in feedforward layers, the system delivers over 20% speedups in inference and training while reducing memory usage and power consumption. The research highlights how sparse architectures could become a key path for scaling future LLM performance efficiently.

◾How BASF manages thousands of supply chain decisions with AlphaEvolve: BASF Agricultural Solutions is using Google Cloud’s AlphaEvolve to build a digital twin of its global supply chain, helping planners manage over 5,000 value chains and complex production dependencies. By evolving algorithms from historical operational data, the system improved planning accuracy by over 80%, enabling smarter inventory management, dynamic safety stocks, and network-wide optimization across BASF’s global manufacturing operations.

◾Cloud Storage Rapid turbocharges object storage for AI, analytics: Google Cloud introduced Cloud Storage Rapid, a high-performance object storage family built for AI and analytics workloads. Featuring Rapid Bucket and Rapid Cache, the platform delivers ultra-low latency, multi-terabyte throughput, faster checkpointing, and improved GPU utilization for large-scale AI training and inference. Designed for data-intensive workloads, it helps organizations reduce bottlenecks, lower infrastructure costs, and scale AI systems more efficiently.

◾Building web search-enabled agents with Strands and Exa: AWS and Exa demonstrated how to build web search-enabled AI agents using the Strands Agents SDK and Exa’s AI-native search tools. The integration enables agents to perform semantic web search, retrieve structured page content, and autonomously conduct multi-step research workflows. By combining Exa’s search capabilities with Strands’ model-driven orchestration, developers can build research, fact-checking, and competitive intelligence agents grounded in real-time web data.

See you next time!

Welcome to DataPro #171 — where we unpack the systems, workflows, and AI skills powering production AI today, featuring AI Skills Conf by Community Sprints.

This week, we highlight insights from Richa Awasthi, Vice President at JPMorgan Chase & Co., on how predictive risk analytics is transforming financial stability, helping institutions shift from reactive risk management to real-time, AI-driven decision-making.

Meanwhile, AI is moving far beyond basic prompting, with companies prioritizing skills in agentic workflows, automation, governance, and production deployment. That’s why Packt DataPro is joining the AI Skills Conf on May 14 as a Knowledge Partner, helping bring more practitioner-focused AI learning to our readers alongside the hands-on technical deep dives from the Packt Virtual Conference team.

🚀 Join the Free Virtual AI Skills Conf — May 14

5,000+ professionals | 20+ Speakers from Google DeepMind, AWS, Meta, Spotify, DoorDash, SAP, Scale AI & more

Learn the practical AI skills shaping 2026:

• Agentic workflows and AI-native operations
• AI memory systems and context engineering
• Enterprise AI adoption and governance
• AI ROI and real-world implementation strategies
• Production workflows teams are deploying today

I’ll also be joining the “AI ROI Reality Check” panel alongside leaders from Spotify and SAP.

Register free and secure your spot

Also in this edition:

• Google launches 50+ managed MCP servers for enterprise AI agents
• AWS introduces autonomous frontier agents for DevOps and security operations
• Hapag-Lloyd automates customer intelligence workflows using Amazon Bedrock
• Zyphra releases ZAYA1-8B, a lightweight reasoning MoE rivaling frontier models
• Mend introduces an enterprise AI governance framework for secure AI adoption
• CopilotKit brings persistent memory infrastructure to agentic applications

The next generation of AI work is already taking shape. The advantage now belongs to those learning how these systems actually operate in production.

Cheers,

Merlyn Shelley,

Growth Lead, Packt.

Reimagining Financial Stability Through Predictive Risk Analytics

By Richa Awasthi, Vice President at JPMorgan Chase & Co.

The real imperative now is not simply to react to risks as they arise, but to anticipate and address them before they can impact the system, leveraging analytical foresight at every step. In this article, I explore how predictive risk analytics can transform not only institutional decision-making, but also the resilience of the broader US financial system.

A turning point is underway in the world of financial stability; institutions are not only shifting their tools but also their mindsets. The introduction and refinement of predictive risk analytics is reshaping practices far beyond mere credit scoring or regulatory compliance. Instead, it is promising a systemic transformation; one that’s anchored in real-time data, advanced machine learning, and an unrelenting drive to foresee threats before they trigger a crisis.

‘By integrating predictive analytics into every layer of financial oversight, these innovations are reimagining how systemic risk is managed in the US, ensuring a more resilient and inclusive economy.’

The Evolving Landscape of Financial Stability

Over the past decade, US financial institutions have navigated significant economic turbulence, including volatile interest rates, unexpected inflationary pressures, and the emergence of new asset classes — factors that have exposed the limitations of traditional frameworks for managing systemic risk.

According to the Federal Reserve and the United States Economic Forecast by Deloitte, the average CPI growth in 2025 is projected at 2.9%, moderating in 2026 before falling closer to 2.3% by the end of the decade. Meanwhile, the baseline forecast for the 10-year Treasury yield suggests sustained pressure, remaining above 4.1% through 2030, even as short-term rates fluctuate in response to shifting monetary policies.

This emphasises, ‘Strengthening US financial stability requires embedding predictive insights across all layers of risk oversight, from credit allocation to liquidity management, so that institutions can anticipate emerging threats, respond proactively, and maintain systemic resilience.’

The Rise of Predictive Analytics

The predictive analytics market is currently experiencing explosive growth. A 2024 report by MarketsandMarkets projects the global market for predictive analytics will climb from $10.5 billion in 2023 to $14.5 billion by 2024, registering an impressive 13.5% CAGR. Fueled by AI and machine learning, these systems now comb through vast, multidimensional datasets, ranging from transaction histories to alternative data sources, identifying patterns that are invisible to conventional models.

Risk management is the primary beneficiary. Financial institutions are leveraging predictive models that assess real-time exposures, estimate macroeconomic sensitivities, and recalibrate risk profiles in response to destabilising events. According to PwC, by 2030, 95% of financial models will incorporate Environmental, Social, and Governance (ESG) factors, reflecting a broader commitment to holistic oversight.

‘Preventing banking crises is no longer a matter of luck or last-minute intervention. Predictive analytics offers us a toolkit for ongoing surveillance, early warning, and targeted mitigation, fundamentally rewiring the calculus of risk.’

Transforming Small Business Access to Credit

While the macroeconomic benefits of predictive risk analytics are clear, its implications for small business lending are particularly consequential for US economic growth and resilience.

Traditionally, access to credit relied on rigid, historical measures, often excluding startups, minority entrepreneurs, and those with unconventional financial profiles. Predictive analytics, by contrast, enables lenders to assess future potential, not merely past performance, allowing capital to flow where it can generate sustainable economic impact.

Recent analyses indicate that global financial institutions leveraging predictive models have reduced business loan default rates by up to 30%, while approval rates for previously underserved applicants increased by over 25%.

Predictive Models as Crisis Prevention Tools

Financial crises have frequently exposed the limitations of traditional risk management. The 2008 banking crisis, for instance, exposed the fragility of models overly reliant on lagging indicators and static assumptions.

Contemporary predictive models utilise statistical, machine learning, and data mining techniques to identify risks well in advance of their materialisation. Early interventions allow for targeted responses rather than blanket interventions.

‘Banking crises don’t happen overnight; they simmer. What predictive analytics allows us to do is notice when the system is beginning to heat up and take action while there’s still time to avert damage.’

A Note of Caution: The Human Factor

While predictive analytics can help illuminate systemic weak spots, there remains a real danger in over-reliance on algorithmic models. Human judgment, ethical reasoning, and holistic context are still critical.

‘Predictive analytics must complement, not supplant, prudent oversight and ethical grounding.’

Technology, Policy, and the Path Ahead

Policy initiatives across the US are increasingly focused on harnessing such tools for broader stability. The Federal Reserve plans to continue integrating advanced analytics into its supervisory frameworks.

Reflecting Forward: A System Reimagined

As the decade unfolds, the convergence of predictive analytics, policy innovation, and professional expertise points toward a financial system both more resilient and more inclusive.

‘Predictive analytics is a critical tool, yet the real transformation occurs when we rebuild processes and organisational culture around its insights.’

Data Science & ML Research Roundup

◾ Google-managed MCP servers are available for everyone: Google Cloud announced 50+ managed MCP servers for AI agents, enabling secure, enterprise-grade connectivity across Google Cloud services. These servers support interoperability with tools like ChatGPT, Claude, LangChain, and Gemini CLI while offering centralized discovery, governance, observability, and security. Use cases span infrastructure automation, analytics, developer assistance, productivity workflows, and customer experiences, helping agents move from prototypes to production-ready autonomous systems.

◾ AWS launches frontier agents for security testing and cloud operations: AWS announced two new frontier AI agents: AWS Security Agent and AWS DevOps Agent, designed for autonomous security testing and cloud operations. AWS Security Agent reduces penetration testing from weeks to hours by autonomously identifying and validating vulnerabilities. AWS DevOps Agent accelerates incident resolution across multicloud environments with up to 75% lower MTTR. Together, they represent AI systems that independently manage complex, persistent enterprise workflows.

◾ How Hapag-Lloyd uses Amazon Bedrock to transform customer feedback into actionable insights: Hapag-Lloyd built an AI-powered customer feedback analysis platform using Amazon Bedrock, OpenSearch, LangChain, and LangGraph to automate sentiment analysis, insight generation, and reporting. The solution replaces manual feedback reviews with scalable AI workflows, enabling faster product decisions and real-time insights. Features include AI chatbots, biweekly reports, semantic search, and Bedrock Guardrails for responsible AI, helping teams focus more on innovation and customer experience.

◾ Zyphra Releases ZAYA1-8B: A Reasoning MoE Trained on AMD Hardware That Punches Far Above Its Weight Class: Zyphra released ZAYA1-8B, an Apache 2.0 open-weight Mixture-of-Experts model with 760M active and 8.4B total parameters, trained entirely on AMD MI300 hardware. Despite its small active footprint, it rivals frontier reasoning models in math and coding benchmarks using innovations like MoE++, 8× KV-cache compression, and Markovian RSA test-time compute, enabling efficient, high-performance reasoning with lower inference costs and latency.

◾ Mend Releases AI Security Governance Framework: Covering Asset Inventory, Risk Tiering, AI Supply Chain Security, and Maturity Model: Mend released an AI Security Governance Framework to help organizations manage rapidly growing AI adoption through structured governance, risk tiering, and supply chain security. The framework covers AI asset inventory, least-privilege access, AI-BOMs, monitoring for AI-specific threats, and maturity modeling aligned with NIST, OWASP, ISO, and EU AI Act standards. It provides practical guidance for scaling AI securely without slowing engineering velocity.

◾ CopilotKit Introduces Enterprise Intelligence Platform That Gives Agentic Applications Persistent Memory Across Sessions and Devices: CopilotKit introduced its Enterprise Intelligence Platform, adding persistent memory infrastructure for agentic applications across sessions and devices. Built on top of the CopilotKit stack, the platform enables durable “Threads” that preserve UI state, workflows, voice, files, and multimodal interactions. It supports framework-agnostic agents, enterprise security features, and resumable long-running workflows, helping teams move agentic applications from stateless demos to production-ready systems.

See you next time!