Skip to content

feat(boto3): Instrument Bedrock Runtime Converse calls as gen_ai spans - #8186

Draft
sentry-junior[bot] wants to merge 7 commits into
masterfrom
junior/boto3-bedrock-converse
Draft

sentry-junior[bot] wants to merge 7 commits into
masterfrom
junior/boto3-bedrock-converse

Conversation

@sentry-junior

@sentry-junior sentry-junior Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Description

This is the first of 3 stacked PRs that add AWS Bedrock Runtime gen_ai instrumentation to Boto3Integration, so Bedrock calls appear in AI Agents monitoring. Spans, ops, and attributes match the JS SDK's aws-sdk Bedrock Runtime extension (packages/server-utils/src/integrations/aws-sdk/services/bedrock-runtime.ts).

This PR covers Converse, using the existing _services extension registry:

  • span chat {modelId}, op gen_ai.chat, gen_ai.provider.name = aws.bedrock
  • request model, max tokens, temperature, top_p, stop sequences (new SPANDATA.GEN_AI_REQUEST_STOP_SEQUENCES), finish reasons, and token usage
  • adds a get_span_name() hook to _ServiceExtension

Decisions the diff doesn't show:

  • Origin auto.ai.boto3, not the generic auto.http.boto3. It follows the trace-origin spec (auto.<category>.<integration>). JS uses auto.ai.aws_sdk for the same spans. Other bedrock-runtime operations keep the generic HTTP span.
  • Prompt/response capture honors the data_collection option. When data_collection is set, data_collection.gen_ai.inputs gates system instructions, input messages, and tool definitions, and data_collection.gen_ai.outputs gates output messages (text, reasoning, tool calls). Each is gated independently. Without data_collection, both require send_default_pii and the new Boto3Integration(include_prompts=True) option, which defaults to True. JS capture is in progress in feat(server-utils): Record Bedrock prompts, tools, and responses sentry-javascript#26118.
  • Tool definitions are gated on inputs in both modes. This differs from Anthropic and OpenAI, which always record tool definitions when data_collection isn't set (a legacy branch kept until send_default_pii is deprecated).
  • Input messages are truncated like Anthropic and OpenAI. When should_truncate_gen_ai_input() is true (gen_ai spans sent in transactions), truncate_and_annotate_messages reduces the history to the last message. The text, tool call arguments, and tool results of that message's parts are then capped at MAX_SINGLE_MESSAGE_CONTENT_CHARS in total, with the same ... suffix, in the Bedrock code: the shared helper only caps a top-level content, not parts. The original-length _meta annotation doesn't apply to gen_ai.input.messages yet, and teaching the shared helper about parts is a follow-up.
  • Only the input messages are added after the span starts, by a new _ServiceExtension.get_span_start_attributes() hook, because truncation needs the started span. ignore_spans doesn't see them, and spans it drops skip that work. Max tokens, temperature, top_p, stop sequences, system instructions, and tool definitions stay request attributes.
  • gen_ai.usage.input_tokens includes cache read/write tokens. Bedrock's inputTokens excludes them, so this differs from JS only for prompt-cached requests.
  • Media blocks (image, document, video) aren't recorded.
  • Like the rest of boto3, this never starts a transaction. Calls without a parent span aren't recorded.

Stack: this PR → InvokeModel → streaming (ConverseStream + InvokeModelWithResponseStream).

Heads-up: major/3.0 has diverged boto3 service-extension work (_services/*, _utils.py), so porting this there will need conflict resolution.

Changelog note

Converse spans change from http.client / auto.http.boto3 to gen_ai.chat / auto.ai.boto3, and their name changes from aws.bedrock-runtime.Converse to chat <modelId>, which affects ignore_spans rules that match on the name. Boto3Integration is auto-enabled, so users with send_default_pii=True start sending Converse prompts and responses unless they set Boto3Integration(include_prompts=False) or configure data_collection.

Issues

via @dcramer.

--

View Session [Sentry]

Record Bedrock Runtime `Converse` calls as `gen_ai.chat` spans with the
`auto.ai.boto3` origin, the request model and inference parameters, token
usage, and finish reasons. Input and output messages, system instructions,
and tool definitions are recorded when `data_collection` allows gen_ai
inputs/outputs, or with `send_default_pii` when `data_collection` isn't set.

Adds a `get_span_name()` hook to `_ServiceExtension`.

Refs #5470

Co-Authored-By: David Cramer <23610+dcramer@users.noreply.github.com>
Record Bedrock prompts and responses like the other gen_ai integrations:
`data_collection.gen_ai.inputs`/`outputs` decide when `data_collection` is
set, otherwise `send_default_pii` and the new `include_prompts` option of
Boto3Integration do.

Refs #5470

Co-Authored-By: Sergiy Dybskiy <3760543+sergical@users.noreply.github.com>
@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Coverage Results 📊

✅ 138456 passed | ⏭️ 7497 skipped | Total: 145953 | Pass Rate: 94.86% | Execution Time: 451m 35s

📊 Comparison with Base Branch

Metric Change
Total Tests 📈 +1283
Passed Tests 📈 +1242
Failed Tests —
Skipped Tests 📈 +41

All tests are passing successfully.

✅ Patch coverage is 96.37% (239 of 248 changed executable lines covered; target 80%).
Project statement coverage is 90.38% (up 0.06 percentage points from base (dbc50bd) to head (e8f6930)).

Changed files with executable lines (7)
File Patch coverage Changed executable lines
sentry_sdk/integrations/boto3/_services/bedrock_runtime.py 96.00% 216/225 covered; missed: 230, 241, 321, 326, 330, 355, 360, 410, 416; partial branches: 90, 149, 153, 157, 159, 178, 204, 229, 240, 257, 261, 290, 296, 298, 320, 325, 329, 354, 359, 369, 378, 380, 382, 390, 398, 409, 415, 419, 421, 425
sentry_sdk/consts.py 100.00% 2/2 covered
sentry_sdk/integrations/boto3/init.py 100.00% 2/2 covered
sentry_sdk/integrations/boto3/_instrumentation.py 100.00% 12/12 covered
sentry_sdk/integrations/boto3/_services/base.py 100.00% 4/4 covered
sentry_sdk/integrations/boto3/_services/registry.py 100.00% 2/2 covered
sentry_sdk/integrations/boto3/consts.py 100.00% 1/1 covered
Coverage diff
@@            Coverage Diff             @@
##        master     #8186       +/-##
==========================================
+ Coverage    90.32%    90.38%    +0.06%
==========================================
  Files          203       204        +1
  Tracked lines     26905     27151      +246
  Branches     10048     10186      +138
==========================================
+ Hits         24302     24540      +238
- Misses        2603      2611        +8
- Partials      1516      1544       +28

Generated by Coverage Action

…tegrations

When gen_ai spans are sent in transactions (`should_truncate_gen_ai_input()`),
keep only the last input message with `truncate_and_annotate_messages()`, like
the Anthropic and OpenAI integrations, so long histories don't exceed the
transaction size limit.

The helper needs the started span, so request data is now added through a
new `_ServiceExtension.get_span_start_attributes()` hook.

Refs #5470

Co-Authored-By: Sergiy Dybskiy <3760543+sergical@users.noreply.github.com>
`truncate_and_annotate_messages()` only truncates a message's `content`, so
when only the last input message is kept, its text parts were kept in full.
Truncate them to `MAX_SINGLE_MESSAGE_CONTENT_CHARS` with the same `...`
suffix.

Only the input messages need the started span, so the request parameters,
system instructions, and tool definitions are request attributes again.
Messages aren't transformed for spans that won't be sent.

Co-Authored-By: Sergiy Dybskiy <3760543+sergical@users.noreply.github.com>
@sentry-junior
sentry-junior Bot force-pushed the junior/boto3-bedrock-converse branch from 886ef2a to 1057e9c Compare November 5, 2026 00:29
sentry-junior Bot and others added 3 commits November 5, 2026 00:55
The last input message's text was capped, but its tool call arguments and
tool results weren't, so a huge tool result, which is usually the last
message in agent loops, was kept in full. They now share the message's
limit with its text.

Co-Authored-By: Sergiy Dybskiy <3760543+sergical@users.noreply.github.com>
Co-Authored-By: Sergiy Dybskiy <3760543+sergical@users.noreply.github.com>
…esults

Only string tool call arguments and tool results were capped, so a large JSON
tool input or result was kept in full. JSON values now count with their
serialized length, and are replaced by their truncated serialization when they
don't fit.

Co-Authored-By: Sergiy Dybskiy <3760543+sergical@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant