Skip to content

Instrumented instruction#7212

Merged
youknowone merged 9 commits intoRustPython:mainfrom
youknowone:instrumented-instruction
Feb 27, 2026
Merged

Instrumented instruction#7212
youknowone merged 9 commits intoRustPython:mainfrom
youknowone:instrumented-instruction

Conversation

@youknowone
Copy link
Member

@youknowone youknowone commented Feb 23, 2026

close #7197 #7206

Summary by CodeRabbit

  • New Features

    • Full runtime monitoring subsystem exposed as sys.monitoring with tool registration, callbacks, per-code instrumentation and branch inspection.
    • New APIs: inspect code branches (PyCode.co_branches) and set a frame's current line from Python (frame f_lineno / set_lasti support).
  • Improvements

    • More accurate source-line tracking and LINE events across try/for/compare/annotations.
    • Safer, versioned lazy re-instrumentation and richer instruction instrumentation controls.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 23, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds a runtime monitoring subsystem (sys.monitoring), per-code instrumentation metadata and versioning, interpreter- and frame-level event emission with instrumented-opcode paths, codegen/IR changes to preserve line markers, bytecode/instruction helpers, and new VM/global state fields for coordination.

Changes

Cohort / File(s) Summary
Monitoring subsystem & sys module
crates/vm/src/stdlib/sys/monitoring.rs, crates/vm/src/stdlib/sys.rs
Add MonitoringState, event bitflags, tool APIs (use_tool_id, register_callback, get/set events, instrument_code, etc.), dispatch helpers (fire_*), and expose sys.monitoring Python submodule and sentinels.
VM global state
crates/vm/src/vm/mod.rs, crates/vm/src/vm/interpreter.rs
Add monitoring: PyMutex<MonitoringState>, monitoring_events: MonitoringEventsMask, and instrumentation_version: AtomicU64 to PyGlobalState and initialize them; import AtomicU64.
Interpreter & Frame instrumentation
crates/vm/src/frame.rs, crates/vm/src/builtins/frame.rs, crates/vm/src/builtins/code.rs
Introduce instrumented-opcode execution paths, event firing (CALL/RETURN/LINE/INSTRUCTION/BRANCH/etc.), per-code CoMonitoringData, PyCode monitoring fields and co_branches, frame setters (set_lasti, set_f_lineno), pending unwind/pops and instrumentation/version checks.
Codegen & IR line handling
crates/codegen/src/compile.rs, crates/codegen/src/ir.rs
Preserve and restore source ranges around constructs (try/for/MAKE_FUNCTION/compare/etc.) and retain line-marker NOPs during finalize/constant-folding to align LINE events with high-level constructs.
Bytecode & instruction helpers
crates/compiler-core/src/bytecode.rs, crates/compiler-core/src/bytecode/instruction.rs
Add decode_exception_table; change CodeUnits inner storage to UnsafeCell<Box<[CodeUnit]>> with Sync/Clone/Debug and unsafe replace_op(); add Instruction::is_instrumented, to_instrumented, and to_base.
Interpreter wiring / execution loop
crates/vm/src/vm/interpreter.rs, crates/vm/src/vm/mod.rs
Initialize monitoring-related VM fields, expose instrumentation_version, and adjust imports for AtomicU64.
Stdlib sys wiring
crates/vm/src/stdlib/sys.rs
Bind and expose sys.monitoring Python submodule in the stdlib sys module.
Tests / deps
scripts/update_lib/deps.py
Add test_monitoring test dependency mapping (hard_dep: profilee).
Dictionaries & minor docs
.cspell.dict/cpython.txt, crates/vm/src/types/slot.rs
Add firsttraceable to cspell dictionary; add inline comment in slot.rs (no behavior change).

Sequence Diagram(s)

sequenceDiagram
    participant Tool as Tool/Debugger
    participant SysMon as SysMonitoring
    participant VM as VM (PyGlobalState)
    participant Code as PyCode
    participant Frame as FrameExecutor

    Tool->>SysMon: use_tool_id(tool_id, name)
    Tool->>SysMon: register_callback(event, func)
    SysMon->>VM: update_events_mask()

    VM->>Code: check instrumentation_version
    alt version mismatch
        VM->>Code: instrument_code(events)
        Code->>Code: rewrite bytecode (insert instrumented ops)
        Code->>VM: bump instrumentation_version
    end

    VM->>Frame: execute bytecode
    Frame->>SysMon: fire_py_start()

    loop per instruction
        Frame->>Code: consult monitoring_data
        alt instrumented opcode
            Frame->>SysMon: fire_instruction()/fire_line()/fire_call()/fire_branch...
            SysMon->>Tool: invoke callback(event, payload)
            Frame->>Frame: remap to base opcode and resume
        else normal opcode
            Frame->>Frame: execute opcode normally
        end
    end

    Frame->>SysMon: fire_py_return()
    SysMon->>Tool: invoke callback(PY_RETURN)
Loading

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120 minutes

Possibly related PRs

Suggested reviewers

  • ShaharNaveh
  • fanninpm

Poem

🐰 I hopped through bytecode, light and quick,

planting markers where the lines would stick.
Tools listen in as branches sing,
callbacks chirp at every ping.
Hop—trace—replay; a rabbit's tracing trick.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Out of Scope Changes check ⚠️ Warning The PR includes one out-of-scope addition: a test dependency entry for 'test_monitoring' in scripts/update_lib/deps.py, which is infrastructure-related and not part of the core monitoring implementation objectives. Remove the test dependency entry from scripts/update_lib/deps.py as it is not directly related to implementing the monitoring subsystem functionality specified in issue #7197.
Title check ❓ Inconclusive The title 'Instrumented instruction' is vague and generic, using a broad term that doesn't clearly convey the scope of changes including sys.monitoring, bytecode instrumentation, event emission, and line attribution improvements. Consider using a more descriptive title that captures the main focus, such as 'Add sys.monitoring subsystem with instrumented instructions' or 'Implement monitoring and bytecode instrumentation support'.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check ✅ Passed The PR implements all major requirements from issue #7197: adds sys.monitoring subsystem with per-tool event controls and callbacks [#7197], improves source-line attribution and preserves line markers in loops/exceptions [#7197], expands runtime event emission for line/instruction/call/return/branch/jump/raise/yield [#7197], and introduces bytecode instrumentation with instrumented instruction methods [#7197].
Docstring Coverage ✅ Passed Docstring coverage is 96.09% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Contributor

github-actions bot commented Feb 23, 2026

📦 Library Dependencies

The following Lib/ modules were modified. Here are their dependencies:

[ ] lib: cpython/Lib/opcode.py
[ ] lib: cpython/Lib/_opcode_metadata.py
[x] test: cpython/Lib/test/test__opcode.py (TODO: 2)
[x] test: cpython/Lib/test/test_opcodes.py

dependencies:

  • opcode (native: _opcode, builtins)
    • _opcode_metadata
    • _opcode_metadata

dependent tests: (39 tests)

  • opcode: test__opcode test_code test_compile test_dis test_peepholer
    • dis: test_ast test_compiler_assemble test_dtrace test_fstring test_inspect test_monitoring test_opcache test_patma test_positional_only_arg
      • inspect: test_abc test_argparse test_asyncgen test_buffer test_builtin test_collections test_coroutines test_decimal test_enum test_functools test_generators test_grammar test_ntpath test_operator test_posixpath test_pydoc test_signal test_traceback test_type_annotations test_types test_typing test_unittest test_yield_from test_zipimport
      • trace: test_trace

[ ] lib: cpython/Lib/bdb.py
[ ] test: cpython/Lib/test/test_bdb.py (TODO: 9)

dependencies:

  • bdb (native: sys)
    • contextlib, fnmatch, inspect, os, threading, weakref

dependent tests: (1 tests)

  • bdb: test_bdb

[x] test: cpython/Lib/test/test_compile.py (TODO: 31)
[x] test: cpython/Lib/test/test_compiler_assemble.py
[x] test: cpython/Lib/test/test_compiler_codegen.py
[x] test: cpython/Lib/test/test_peepholer.py (TODO: 31)

dependencies:

dependent tests: (no tests depend on compile)

[x] lib: cpython/Lib/dis.py
[ ] test: cpython/Lib/test/test_dis.py (TODO: 42)

dependencies:

  • dis

dependent tests: (56 tests)

  • dis: test__opcode test_ast test_code test_compile test_compiler_assemble test_dis test_dtrace test_fstring test_inspect test_monitoring test_opcache test_patma test_peepholer test_positional_only_arg
    • inspect: test_abc test_argparse test_asyncgen test_buffer test_builtin test_collections test_coroutines test_decimal test_enum test_functools test_generators test_grammar test_ntpath test_operator test_posixpath test_pydoc test_signal test_traceback test_type_annotations test_types test_typing test_unittest test_yield_from test_zipimport
      • asyncio: test_asyncio test_contextlib_async test_logging test_os test_sys_settrace test_unittest
      • bdb: test_bdb
      • dataclasses: test__colorize test_ctypes test_genericalias test_pprint test_regrtest test_zoneinfo
      • importlib.metadata: test_importlib
      • rlcompleter: test_rlcompleter
      • trace: test_trace
      • xmlrpc.server: test_docxmlrpc test_xmlrpc

[ ] test: cpython/Lib/test/test_monitoring.py (TODO: 13)

dependencies:

dependent tests: (no tests depend on monitoring)

[x] test: cpython/Lib/test/test_sys.py (TODO: 8)
[ ] test: cpython/Lib/test/test_syslog.py (TODO: 2)
[ ] test: cpython/Lib/test/test_sys_setprofile.py (TODO: 14)
[ ] test: cpython/Lib/test/test_sys_settrace.py (TODO: 52)

dependencies:

dependent tests: (215 tests)

  • sys: regrtestdata test___all__ test__colorize test__locale test__osx_support test_android test_annotationlib test_argparse test_array test_ast test_asyncio test_audit test_bdb test_bigaddrspace test_bigmem test_bisect test_buffer test_builtin test_bytes test_bz2 test_c_locale_coercion test_calendar test_call test_cmath test_cmd test_cmd_line test_cmd_line_script test_code test_code_module test_codeccallbacks test_codecs test_collections test_compile test_compileall test_complex test_concurrent_futures test_context test_contextlib test_coroutines test_csv test_ctypes test_datetime test_dbm test_dbm_sqlite3 test_descr test_dict test_difflib test_dis test_doctest test_doctest2 test_docxmlrpc test_dtrace test_dynamic test_dynamicclassattribute test_email test_ensurepip test_enum test_enumerate test_eof test_except_star test_exceptions test_faulthandler test_fcntl test_file test_file_eintr test_fileinput test_fileio test_float test_fork1 test_format test_fractions test_frozen test_functools test_future_stmt test_gc test_generators test_genericpath test_getopt test_glob test_grammar test_gzip test_hash test_hashlib test_http_cookiejar test_httpservers test_importlib test_inspect test_int test_io test_iter test_itertools test_json test_largefile test_launcher test_list test_locale test_logging test_long test_lzma test_mailbox test_marshal test_math test_memoryio test_memoryview test_metaclass test_mimetypes test_mmap test_monitoring test_msvcrt test_multiprocessing_fork test_multiprocessing_main_handling test_ntpath test_numeric_tower test_operator test_optparse test_ordered_dict test_os test_osx_env test_pathlib test_patma test_peepholer test_pickle test_pkg test_pkgutil test_platform test_plistlib test_posix test_posixpath test_print test_property test_pty test_pwd test_py_compile test_pyclbr test_pydoc test_pyexpat test_quopri test_raise test_range test_re test_regrtest test_repl test_reprlib test_resource test_runpy test_script_helper test_select test_selectors test_shutil test_signal test_site test_slice test_smtplib test_socket test_sqlite3 test_ssl test_stable_abi_ctypes test_stat test_statistics test_str test_strftime test_string_literals test_strtod test_struct test_subprocess test_support test_symtable test_sys test_sys_setprofile test_sys_settrace test_sysconfig test_syslog test_tarfile test_tempfile test_termios test_threadedtempfile test_threading test_threading_local test_threadsignals test_time test_timeit test_tomllib test_tools test_trace test_traceback test_type_comments test_types test_typing test_unicode_file test_unicode_file_functions test_unicodedata test_unittest test_univnewlines test_urllib test_urllib2 test_urllib2net test_urlparse test_utf8_mode test_uuid test_venv test_wait3 test_wait4 test_wave test_weakref test_webbrowser test_winconsoleio test_with test_wsgiref test_xml_etree test_xmlrpc test_zipapp test_zipfile test_zipfile64 test_zipimport test_zlib

Legend:

  • [+] path exists in CPython
  • [x] up-to-date, [ ] outdated

@youknowone youknowone force-pushed the instrumented-instruction branch 4 times, most recently from cff3dfc to c975125 Compare February 24, 2026 11:37
@youknowone youknowone marked this pull request as ready for review February 24, 2026 14:09
///
/// # Panics (debug)
/// Panics if called on an already-instrumented opcode.
pub fn to_instrumented(self) -> Option<Self> {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to_instrumented and to_based transform into each others in the same type.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🧹 Nitpick comments (1)
crates/vm/src/frame.rs (1)

2568-2584: Consider consolidating identical jump instrumentation arms.

InstrumentedJumpForward and InstrumentedJumpBackward are identical aside from the label variable. You can merge them to keep the match smaller and avoid duplication.

Refactor sketch
-            Instruction::InstrumentedJumpForward => {
-                let src_offset = (self.lasti() - 1) * 2;
-                let target = bytecode::Label::from(u32::from(arg));
-                self.jump(target);
-                if self.monitoring_mask & monitoring::EVENT_JUMP != 0 {
-                    monitoring::fire_jump(vm, self.code, src_offset, target.0 * 2)?;
-                }
-                Ok(None)
-            }
-            Instruction::InstrumentedJumpBackward => {
-                let src_offset = (self.lasti() - 1) * 2;
-                let dest = bytecode::Label::from(u32::from(arg));
-                self.jump(dest);
-                if self.monitoring_mask & monitoring::EVENT_JUMP != 0 {
-                    monitoring::fire_jump(vm, self.code, src_offset, dest.0 * 2)?;
-                }
-                Ok(None)
-            }
+            Instruction::InstrumentedJumpForward | Instruction::InstrumentedJumpBackward => {
+                let src_offset = (self.lasti() - 1) * 2;
+                let target = bytecode::Label::from(u32::from(arg));
+                self.jump(target);
+                if self.monitoring_mask & monitoring::EVENT_JUMP != 0 {
+                    monitoring::fire_jump(vm, self.code, src_offset, target.0 * 2)?;
+                }
+                Ok(None)
+            }

As per coding guidelines, "When branches differ only in a value but share common logic, extract the differing value first, then call the common logic once to avoid duplicate code."

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/frame.rs` around lines 2568 - 2584, The two match arms for
Instruction::InstrumentedJumpForward and Instruction::InstrumentedJumpBackward
are identical except for the label variable; consolidate them by matching both
variants together (Instruction::InstrumentedJumpForward |
Instruction::InstrumentedJumpBackward) and compute the label once from arg
(e.g., let label = bytecode::Label::from(u32::from(arg))), then reuse the shared
logic: compute src_offset via self.lasti(), call self.jump(label), and
conditionally call monitoring::fire_jump(vm, self.code, src_offset, label.0 * 2)
using the existing monitoring_mask check to avoid duplication.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@crates/compiler-core/src/bytecode.rs`:
- Around line 102-128: The computation end: start + size in
decode_exception_table can overflow on malformed input; replace the unchecked
addition with a checked add (e.g., start.checked_add(size)) and abort decoding
when it returns None (break/return) so you never push an entry where end <
start; update the ExceptionTableEntry creation to use the checked result and
keep the rest of the loop logic (read_varint_with_start/read_varint and
depth_lasti handling) unchanged.
- Around line 401-415: The safety docs for CodeUnits::replace_op must state that
mutation occurs through a shared reference and therefore the caller must ensure
exclusive access (no other threads or aliases access the same CodeUnits) or that
interior mutability/synchronization is used (e.g. stored in an
UnsafeCell/Mutex/RwLock). Update the unsafe fn replace_op documentation to
explicitly require exclusive/unique ownership or external synchronization to
prevent data races, and mention that the caller must guarantee the CodeUnit
memory is not concurrently accessed while calling replace_op; reference the
CodeUnits type and the replace_op method in the wording.

In `@crates/compiler-core/src/bytecode/instruction.rs`:
- Around line 422-511: The debug assertion in to_instrumented currently checks
self.to_base().is_none() which misses InstrumentedLine and
InstrumentedInstruction; change the guard to
debug_assert!(!self.is_instrumented(), "to_instrumented called on
already-instrumented opcode {self:?}") so it uses the canonical
is_instrumented() check; update the assertion message if desired, then run cargo
fmt, cargo clippy, and perform a clean bytecode build (remove relevant target
build dirs and any .pyc files) to ensure the bytecode changes compile and lint
cleanly.

In `@crates/vm/src/builtins/code.rs`:
- Around line 916-979: co_branches assumes a fixed fall-through of (i + 2) * 2
for POP_JUMP_IF_* but NOT_TAKEN may be absent; update the POP_JUMP_IF_* arm in
co_branches to compute the fall-through from the actual next opcode instead of
using (i + 2): inspect instructions.get(i + 1).map(|u| u.op) and if that opcode
is the NOT_TAKEN sentinel (or its instrumented variant) use (i + 2) as the
fall-through index, otherwise use (i + 1); keep the jump target as oparg * 2 and
return the tuple creation logic unchanged for co_branches and the
Instruction::PopJumpIfFalse/PopJumpIfTrue/PopJumpIfNone/PopJumpIfNotNone match.

In `@crates/vm/src/builtins/frame.rs`:
- Around line 74-104: set_f_lineno currently only ensures the requested line
exists in self.code.locations and then calls self.set_lasti(idx as u32) without
validating the resulting instruction index; add explicit bounds checks and
execution-safety checks before calling set_lasti: verify the computed idx is
within the valid instruction range (compare against the code's
instruction/bytecode length or an explicit instruction count on self.code),
return a ValueError if out of range, and additionally refuse or validate targets
that would jump into the middle of an exception-handler setup or otherwise
unsafe region (use the code's exception table/handler metadata on self.code to
detect unsafe indices) so that set_lasti is only called for safe, in-bounds
instruction indices.

In `@crates/vm/src/frame.rs`:
- Around line 261-270: set_lasti currently only updates the atomic lasti when
threading is enabled, leaving the mutex-protected FrameState.lasti out of sync
with external f_lasti and ExecutingFrame::lasti; modify set_lasti to also
acquire the state mutex and update FrameState.lasti (the same field read by
ExecutingFrame::lasti) when #[cfg(feature = "threading")] so both the atomic
(self.lasti.store(...)) and the mutex-protected value are kept in sync, ensuring
reads via ExecutingFrame::lasti observe external updates.

---

Nitpick comments:
In `@crates/vm/src/frame.rs`:
- Around line 2568-2584: The two match arms for
Instruction::InstrumentedJumpForward and Instruction::InstrumentedJumpBackward
are identical except for the label variable; consolidate them by matching both
variants together (Instruction::InstrumentedJumpForward |
Instruction::InstrumentedJumpBackward) and compute the label once from arg
(e.g., let label = bytecode::Label::from(u32::from(arg))), then reuse the shared
logic: compute src_offset via self.lasti(), call self.jump(label), and
conditionally call monitoring::fire_jump(vm, self.code, src_offset, label.0 * 2)
using the existing monitoring_mask check to avoid duplication.

ℹ️ Review info

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9f250a0 and 0ffd745.

⛔ Files ignored due to path filters (8)
  • Lib/opcode.py is excluded by !Lib/**
  • Lib/test/dis_module.py is excluded by !Lib/**
  • Lib/test/profilee.py is excluded by !Lib/**
  • Lib/test/test_compile.py is excluded by !Lib/**
  • Lib/test/test_dis.py is excluded by !Lib/**
  • Lib/test/test_monitoring.py is excluded by !Lib/**
  • Lib/test/test_peepholer.py is excluded by !Lib/**
  • crates/codegen/src/snapshots/rustpython_codegen__compile__tests__nested_double_async_with.snap is excluded by !**/*.snap
📒 Files selected for processing (14)
  • .cspell.dict/cpython.txt
  • crates/codegen/src/compile.rs
  • crates/codegen/src/ir.rs
  • crates/compiler-core/src/bytecode.rs
  • crates/compiler-core/src/bytecode/instruction.rs
  • crates/vm/src/builtins/code.rs
  • crates/vm/src/builtins/frame.rs
  • crates/vm/src/frame.rs
  • crates/vm/src/stdlib/sys.rs
  • crates/vm/src/stdlib/sys/monitoring.rs
  • crates/vm/src/types/slot.rs
  • crates/vm/src/vm/interpreter.rs
  • crates/vm/src/vm/mod.rs
  • scripts/update_lib/deps.py

Comment on lines +261 to +270
pub fn set_lasti(&self, val: u32) {
#[cfg(feature = "threading")]
{
self.lasti.store(val, atomic::Ordering::Relaxed);
}
#[cfg(not(feature = "threading"))]
{
self.lasti.set(val);
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Keep FrameState.lasti in sync when threading is enabled.

In threading builds, ExecutingFrame::lasti() reads state.lasti, but set_lasti only updates the atomic field. That means external f_lasti updates won’t affect the executing instruction pointer. Consider syncing the mutex-protected value too.

Suggested fix
 pub fn set_lasti(&self, val: u32) {
     #[cfg(feature = "threading")]
     {
         self.lasti.store(val, atomic::Ordering::Relaxed);
+        self.state.lock().lasti = val;
     }
     #[cfg(not(feature = "threading"))]
     {
         self.lasti.set(val);
     }
 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/frame.rs` around lines 261 - 270, set_lasti currently only
updates the atomic lasti when threading is enabled, leaving the mutex-protected
FrameState.lasti out of sync with external f_lasti and ExecutingFrame::lasti;
modify set_lasti to also acquire the state mutex and update FrameState.lasti
(the same field read by ExecutingFrame::lasti) when #[cfg(feature =
"threading")] so both the atomic (self.lasti.store(...)) and the mutex-protected
value are kept in sync, ensuring reads via ExecutingFrame::lasti observe
external updates.

Copy link
Contributor

@ShaharNaveh ShaharNaveh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! we now have instrumented opcodes 🥳

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
crates/compiler-core/src/bytecode/instruction.rs (1)

513-527: ⚠️ Potential issue | 🟠 Major

label_arg() does not handle instrumented jump variants, causing label_targets() to miss jump targets.

When monitoring is enabled, instructions are replaced with Instrumented* variants in the bytecode (via replace_op() in monitoring.rs). However, label_arg() only matches base variants (JumpForward, JumpBackward, PopJumpIf*, ForIter, Send), so it returns None for InstrumentedJumpForward, InstrumentedJumpBackward, InstrumentedForIter, InstrumentedPopJumpIf*, and InstrumentedSend. Since label_targets() iterates self.instructions and calls label_arg() on each instruction, it will silently miss jump targets when instrumented bytecode is present. This affects bytecode display and JIT compilation.

Delegate to to_base() (which already handles all these variants correctly) to ensure instrumented instructions are properly analyzed:

Suggested implementation
fn label_arg(&self) -> Option<Arg<Label>> {
    match self {
        Self::JumpBackward { target: l }
        | Self::JumpBackwardNoInterrupt { target: l }
        | Self::JumpForward { target: l }
        | Self::PopJumpIfTrue { target: l }
        | Self::PopJumpIfFalse { target: l }
        | Self::PopJumpIfNone { target: l }
        | Self::PopJumpIfNotNone { target: l }
        | Self::ForIter { target: l }
        | Self::Send { target: l } => Some(*l),
        _ => {
            if let Some(base) = self.to_base() {
                base.label_arg()
            } else {
                None
            }
        }
    }
}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/compiler-core/src/bytecode/instruction.rs` around lines 513 - 527,
label_arg() on Instruction (impl InstructionMetadata) only matches base
jump/send variants and thus skips Instrumented* variants; update label_arg() to
delegate to self.to_base() for non-matching cases so instrumented instructions
return the same Arg<Label> as their base. Specifically, inside
Instruction::label_arg(), keep the existing match for
JumpBackward/JumpForward/PopJumpIf*/ForIter/Send, and in the _ arm call
self.to_base() and, if Some(base), return base.label_arg(), otherwise None; this
ensures label_targets() (which iterates self.instructions) and
monitoring.replace_op() handled instrumented variants are analyzed correctly.
♻️ Duplicate comments (2)
crates/compiler-core/src/bytecode.rs (1)

119-125: ⚠️ Potential issue | 🟡 Minor

Unchecked start + size can wrap on malformed input.

Line 121 computes end: start + size without overflow protection. On a crafted/corrupted varint table, this can produce end < start, yielding a nonsensical entry. The sibling function find_exception_handler (line 85) has the same issue, but since decode_exception_table returns entries for external consumption, a bogus end is more dangerous here.

Suggested fix
+        let end = match start.checked_add(size) {
+            Some(end) => end,
+            None => break,
+        };
         entries.push(ExceptionTableEntry {
             start,
-            end: start + size,
+            end,
             target,
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/compiler-core/src/bytecode.rs` around lines 119 - 125, The code
unconditionally computes end = start + size which can overflow on malformed
input; in decode_exception_table (and similarly in find_exception_handler)
replace the unchecked addition with a checked addition (use
start.checked_add(size)) and handle the None case by returning an error (or
skipping the malformed entry) instead of producing an end < start; ensure the
handling is consistent for ExceptionTableEntry creation (and its
depth/push_lasti logic) so no bogus entries are emitted to external consumers.
crates/vm/src/frame.rs (1)

261-270: FrameState.lasti still not synced in threading builds.

set_lasti only stores to the atomic self.lasti, leaving self.state.lasti (the value read by ExecutingFrame::lasti()) stale in threading builds.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/frame.rs` around lines 261 - 270, In threading builds set_lasti
currently only writes the atomic self.lasti, leaving self.state.lasti stale;
update set_lasti to write both places so ExecutingFrame::lasti() (which reads
self.state.lasti) sees the new value — i.e., in the #[cfg(feature =
"threading")] branch keep the atomic store(self.lasti.store(val, ...)) and also
update the backing FrameState.lasti (self.state.lasti.set(val) or assign
accordingly) so both the atomic and self.state.lasti are kept in sync.
🧹 Nitpick comments (3)
crates/vm/src/frame.rs (3)

395-397: monitoring_mask and prev_line set but never read in yield_from_target.

The ExecutingFrame built here only calls exec.yield_from_target() which neither reads monitoring_mask nor prev_line. These initializations are dead code in this context.

♻️ Proposed cleanup
-            monitoring_mask: 0,
-            prev_line: 0,

Remove the two dead fields, or if ExecutingFrame requires them to be set (struct exhaustiveness), document why.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/frame.rs` around lines 395 - 397, The ExecutingFrame
initialization sets monitoring_mask and prev_line but yield_from_target never
reads them, so drop these dead initializations: remove monitoring_mask and
prev_line from the ExecutingFrame literal (or from the struct if they truly are
unused) and rebuild; if ExecutingFrame must maintain those fields for
exhaustiveness, either derive/implement Default and use ..Default::default() or
add a clear comment in the ExecutingFrame/ yield_from_target caller explaining
why the fields are intentionally set despite not being read. Ensure references
to monitoring_mask, prev_line, ExecutingFrame, and yield_from_target are updated
accordingly.

503-511: Redundant prev_line update when tracing already set it.

When the tracing block (lines 490-497) fires and updates self.prev_line, the block at lines 503-511 unconditionally writes the same value again. The second write is always a no-op but adds unnecessary noise.

♻️ Proposed cleanup
-        if !matches!(
-            op,
-            Instruction::Resume { .. }
-                | Instruction::ExtendedArg
-                | Instruction::InstrumentedLine
-        ) && let Some((loc, _)) = self.code.locations.get(idx)
-        {
-            self.prev_line = loc.line.get() as u32;
-        }
+        if !vm.use_tracing.get()
+            && !matches!(
+                op,
+                Instruction::Resume { .. }
+                    | Instruction::ExtendedArg
+                    | Instruction::InstrumentedLine
+            )
+        {
+            if let Some((loc, _)) = self.code.locations.get(idx) {
+                self.prev_line = loc.line.get() as u32;
+            }
+        }

Or simply remove the else-redundant guard; both writes are idempotent. Consider adding a short comment explaining the write's purpose (LINE event deduplication in InstrumentedLine).

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/frame.rs` around lines 503 - 511, The block that
unconditionally sets self.prev_line using self.code.locations.get(idx) after the
tracing branch is redundant because the tracing branch (which also updates
self.prev_line) already wrote the same value; remove the second update in the
matching block (the code that checks !matches!(op, Instruction::Resume |
Instruction::ExtendedArg | Instruction::InstrumentedLine) && let Some((loc, _))
= self.code.locations.get(idx)) so self.prev_line is only set in the tracing
branch, or convert that assignment into an else-branch of the tracing logic;
also add a short comment by the remaining write (referencing self.prev_line and
InstrumentedLine) explaining it's for LINE event deduplication.

2673-2684: offset.saturating_sub(2) in InstrumentedNotTaken assumes preceding instruction is always the conditional jump.

The src_offset is computed as (lasti - 1) * 2 (this instruction's position), then reduced by 2 to point at the preceding instruction. This implicitly requires the compiler to always emit NOT_TAKEN immediately after its paired conditional jump with no intervening opcodes, and that the instruction width is always 2 bytes.

This is currently guaranteed by the codegen, but it's worth a brief comment explaining the invariant to prevent future breakage.

📝 Suggested comment
 Instruction::InstrumentedNotTaken => {
     if self.monitoring_mask & monitoring::EVENT_BRANCH_LEFT != 0 {
         let offset = (self.lasti() - 1) * 2;
         let dest_offset = self.lasti() * 2;
+        // NOT_TAKEN is always emitted immediately after its paired POP_JUMP_IF_*,
+        // so src_offset points at that conditional jump (offset - 2).
         monitoring::fire_branch_left(
             vm,
             self.code,
             offset.saturating_sub(2),
             dest_offset,
         )?;
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/frame.rs` around lines 2673 - 2684, Add a short explanatory
comment immediately above the InstrumentedNotTaken handling block
(Instruction::InstrumentedNotTaken) that documents the invariant: the NOT_TAKEN
marker is emitted directly after its paired conditional jump with no intervening
opcodes and all instruction slots are 2 bytes, which is why the src_offset is
computed as (lasti - 1) * 2 and then adjusted with .saturating_sub(2) before
calling monitoring::fire_branch_left; additionally consider a debug-time check
(e.g., using debug_assert!) that self.lasti() >= 1 to catch violations early.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@crates/codegen/src/ir.rs`:
- Around line 260-267: The code uses std::collections::HashSet for non_nop_lines
and kept_nop_lines which breaks no_std builds; replace those with the
already-imported IndexSet so the crate remains no_std-compatible: change the
declarations for non_nop_lines and kept_nop_lines to use IndexSet (e.g.
IndexSet<_> and IndexSet<OneIndexed>) and adjust any type annotations/usages
accordingly (membership checks, .new(), .collect()) to use IndexSet APIs.

In `@crates/compiler-core/src/bytecode.rs`:
- Around line 401-418: The current unsafe mutation in CodeUnits::replace_op
casts self.0.as_ptr() to *mut CodeUnit and writes through a shared &self, which
is unsound; wrap the inner buffer in UnsafeCell (e.g., make CodeUnits hold
UnsafeCell<Box<[CodeUnit]>> or UnsafeCell<Vec<CodeUnit>>) so interior mutation
through &self is legal, update uses of self.0 to access the UnsafeCell (e.g.,
get a raw pointer via UnsafeCell::get) inside replace_op, and replace
write_volatile with core::ptr::write since your safety contract requires
exclusive access; ensure the function docs and safety comment reflect the new
UnsafeCell-based design and exclusive-access requirement.

In `@crates/vm/src/frame.rs`:
- Around line 3266-3307: The is_python_call detection is too strict (it only
checks callable.downcast_ref::<PyFunction>()) and mislabels wrapped Python
callables (PyBoundMethod, PyStaticMethod, PyPartial, etc.), causing duplicate
monitoring events; update execute_call_instrumented to replace the single-line
is_python_call check with a small helper (e.g., is_underlying_py_function or
is_python_callable) that recursively unwraps known wrapper types (PyBoundMethod,
PyStaticMethod, PyPartial, and any other delegating types in your codebase) to
inspect the wrapped callable and return true if the ultimate target is a
PyFunction, then use that helper where is_python_call is used (for
monitoring::fire_c_return and monitoring::fire_c_raise) so only true C calls
trigger C_RETURN/C_RAISE.

---

Outside diff comments:
In `@crates/compiler-core/src/bytecode/instruction.rs`:
- Around line 513-527: label_arg() on Instruction (impl InstructionMetadata)
only matches base jump/send variants and thus skips Instrumented* variants;
update label_arg() to delegate to self.to_base() for non-matching cases so
instrumented instructions return the same Arg<Label> as their base.
Specifically, inside Instruction::label_arg(), keep the existing match for
JumpBackward/JumpForward/PopJumpIf*/ForIter/Send, and in the _ arm call
self.to_base() and, if Some(base), return base.label_arg(), otherwise None; this
ensures label_targets() (which iterates self.instructions) and
monitoring.replace_op() handled instrumented variants are analyzed correctly.

---

Duplicate comments:
In `@crates/compiler-core/src/bytecode.rs`:
- Around line 119-125: The code unconditionally computes end = start + size
which can overflow on malformed input; in decode_exception_table (and similarly
in find_exception_handler) replace the unchecked addition with a checked
addition (use start.checked_add(size)) and handle the None case by returning an
error (or skipping the malformed entry) instead of producing an end < start;
ensure the handling is consistent for ExceptionTableEntry creation (and its
depth/push_lasti logic) so no bogus entries are emitted to external consumers.

In `@crates/vm/src/frame.rs`:
- Around line 261-270: In threading builds set_lasti currently only writes the
atomic self.lasti, leaving self.state.lasti stale; update set_lasti to write
both places so ExecutingFrame::lasti() (which reads self.state.lasti) sees the
new value — i.e., in the #[cfg(feature = "threading")] branch keep the atomic
store(self.lasti.store(val, ...)) and also update the backing FrameState.lasti
(self.state.lasti.set(val) or assign accordingly) so both the atomic and
self.state.lasti are kept in sync.

---

Nitpick comments:
In `@crates/vm/src/frame.rs`:
- Around line 395-397: The ExecutingFrame initialization sets monitoring_mask
and prev_line but yield_from_target never reads them, so drop these dead
initializations: remove monitoring_mask and prev_line from the ExecutingFrame
literal (or from the struct if they truly are unused) and rebuild; if
ExecutingFrame must maintain those fields for exhaustiveness, either
derive/implement Default and use ..Default::default() or add a clear comment in
the ExecutingFrame/ yield_from_target caller explaining why the fields are
intentionally set despite not being read. Ensure references to monitoring_mask,
prev_line, ExecutingFrame, and yield_from_target are updated accordingly.
- Around line 503-511: The block that unconditionally sets self.prev_line using
self.code.locations.get(idx) after the tracing branch is redundant because the
tracing branch (which also updates self.prev_line) already wrote the same value;
remove the second update in the matching block (the code that checks
!matches!(op, Instruction::Resume | Instruction::ExtendedArg |
Instruction::InstrumentedLine) && let Some((loc, _)) =
self.code.locations.get(idx)) so self.prev_line is only set in the tracing
branch, or convert that assignment into an else-branch of the tracing logic;
also add a short comment by the remaining write (referencing self.prev_line and
InstrumentedLine) explaining it's for LINE event deduplication.
- Around line 2673-2684: Add a short explanatory comment immediately above the
InstrumentedNotTaken handling block (Instruction::InstrumentedNotTaken) that
documents the invariant: the NOT_TAKEN marker is emitted directly after its
paired conditional jump with no intervening opcodes and all instruction slots
are 2 bytes, which is why the src_offset is computed as (lasti - 1) * 2 and then
adjusted with .saturating_sub(2) before calling monitoring::fire_branch_left;
additionally consider a debug-time check (e.g., using debug_assert!) that
self.lasti() >= 1 to catch violations early.

ℹ️ Review info

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 0ffd745 and 97bd821.

⛔ Files ignored due to path filters (1)
  • Lib/test/test_peepholer.py is excluded by !Lib/**
📒 Files selected for processing (4)
  • crates/codegen/src/ir.rs
  • crates/compiler-core/src/bytecode.rs
  • crates/compiler-core/src/bytecode/instruction.rs
  • crates/vm/src/frame.rs

Comment on lines +3266 to +3307
let is_python_call = callable.downcast_ref::<PyFunction>().is_some();

// Fire CALL event
let call_arg0 = if self.monitoring_mask & monitoring::EVENT_CALL != 0 {
let arg0 = final_args
.args
.first()
.cloned()
.unwrap_or_else(|| monitoring::get_missing(vm));
let offset = (self.lasti() - 1) * 2;
monitoring::fire_call(vm, self.code, offset, &callable, arg0.clone())?;
Some(arg0)
} else {
None
};

match callable.call(final_args, vm) {
Ok(value) => {
if let Some(arg0) = call_arg0
&& !is_python_call
{
let offset = (self.lasti() - 1) * 2;
monitoring::fire_c_return(vm, self.code, offset, &callable, arg0)?;
}
self.push_value(value);
Ok(None)
}
Err(exc) => {
let exc = if let Some(arg0) = call_arg0
&& !is_python_call
{
let offset = (self.lasti() - 1) * 2;
match monitoring::fire_c_raise(vm, self.code, offset, &callable, arg0) {
Ok(()) => exc,
Err(monitor_exc) => monitor_exc,
}
} else {
exc
};
Err(exc)
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

# First, let's find the file and examine the context around the disputed code
fd -t f "frame.rs" | grep -E "crates/vm/src"

Repository: RustPython/RustPython

Length of output: 119


🏁 Script executed:

# Check the structure of is_python_call and callable types
rg -n "PyBoundMethod|is_python_call|PyFunction" crates/vm/src/frame.rs -A 3 -B 3

Repository: RustPython/RustPython

Length of output: 2848


🏁 Script executed:

# Search for PyBoundMethod definition across the codebase
fd -t f "*.rs" | xargs rg -l "struct PyBoundMethod|PyBoundMethod {" | head -5

Repository: RustPython/RustPython

Length of output: 484


🏁 Script executed:

# Check how PyBoundMethod is called/invoked
rg -n "PyBoundMethod" --type rust | head -20

Repository: RustPython/RustPython

Length of output: 1813


🏁 Script executed:

# Look for monitoring tests related to bound methods or C_RETURN/C_RAISE
fd -t f "*.py" -path "*/tests/*" | xargs rg -l "C_RETURN\|C_RAISE\|bound" 2>/dev/null | head -10

Repository: RustPython/RustPython

Length of output: 233


🏁 Script executed:

cat -n crates/vm/src/builtins/function.rs | sed -n '968,1050p'

Repository: RustPython/RustPython

Length of output: 2884


🏁 Script executed:

# Look at PyBoundMethod::call implementation
rg -n "impl Callable for PyBoundMethod" crates/vm/src/builtins/function.rs -A 10

Repository: RustPython/RustPython

Length of output: 407


🏁 Script executed:

# Check what PyBoundMethod.call does
rg -n "fn call" crates/vm/src/builtins/function.rs | grep -A 2 "PyBoundMethod"

Repository: RustPython/RustPython

Length of output: 47


🏁 Script executed:

# Let's look at the Callable trait to understand what call returns
rg -n "pub trait Callable" crates/vm/src/protocol/callable.rs -A 5

Repository: RustPython/RustPython

Length of output: 47


🏁 Script executed:

# Search for how PyBoundMethod actually calls the wrapped function
cat -n crates/vm/src/builtins/function.rs | sed -n '973,1000p'

Repository: RustPython/RustPython

Length of output: 1107


🏁 Script executed:

# Check if there are any tests related to monitoring bound methods
find . -name "*.rs" -type f | xargs rg -l "monitoring.*bound\|bound.*monitoring" 2>/dev/null

Repository: RustPython/RustPython

Length of output: 49


🏁 Script executed:

# Check monitoring module for C_RETURN and C_RAISE semantics
rg -n "fire_c_return\|fire_c_raise" crates/vm/src/ -B 2 -A 5 | head -60

Repository: RustPython/RustPython

Length of output: 47


🏁 Script executed:

# Look at the monitoring module to understand C_RETURN vs PY_RETURN
fd -type f "monitoring.rs" crates/vm/src/

Repository: RustPython/RustPython

Length of output: 235


🏁 Script executed:

# Check if there's any existing logic for detecting Python callables
rg -n "PyFunction\|is_python" crates/vm/src/frame.rs | grep -i "call\|python" | head -20

Repository: RustPython/RustPython

Length of output: 47


🏁 Script executed:

# Look for how CPython's INSTRUMENTED_CALL is implemented
web_search
CPython INSTRUMENTED_CALL implementation PyFunction_Check bound methods

Repository: RustPython/RustPython

Length of output: 161


🏁 Script executed:

# Check the actual implementation of fire_c_return and fire_c_raise
rg -n "pub fn fire_c_return\|pub fn fire_c_raise" crates/vm/src/ -A 10

Repository: RustPython/RustPython

Length of output: 47


🏁 Script executed:

# Find monitoring files
find crates/vm/src -name "*monitoring*" -type f

Repository: RustPython/RustPython

Length of output: 103


🏁 Script executed:

# Search for fire_c_return definition more carefully
rg "fire_c_return" crates/vm/src/ -A 8

Repository: RustPython/RustPython

Length of output: 1780


🏁 Script executed:

# Look at the frame.rs file to see the full context of the monitoring calls
sed -n '3268,3310p' crates/vm/src/frame.rs

Repository: RustPython/RustPython

Length of output: 1591


🏁 Script executed:

# Check what happens after PyBoundMethod.call - does it unwrap to PyFunction internally?
rg -n "zelf.function.call" crates/vm/src/builtins/function.rs -B 5 -A 3

Repository: RustPython/RustPython

Length of output: 351


🏁 Script executed:

# Try to understand if PyBoundMethod ever results in a C call
rg -n "PyBoundMethod" crates/vm/src/builtins/ | head -20

Repository: RustPython/RustPython

Length of output: 1393


🏁 Script executed:

# Check if there's test coverage for monitoring events with bound methods
find . -name "*.py" -path "*/tests/*" 2>/dev/null | xargs grep -l "C_RETURN\|C_RAISE" 2>/dev/null | head -3

Repository: RustPython/RustPython

Length of output: 47


🏁 Script executed:

# Let's check the monitoring module to understand C_RETURN vs PY_RETURN semantics
cat -n crates/vm/src/stdlib/sys/monitoring.rs | head -100

Repository: RustPython/RustPython

Length of output: 4590


🏁 Script executed:

# Look for the fire_c_return implementation and documentation
rg -n "pub fn fire_c_return" crates/vm/src/stdlib/sys/monitoring.rs -A 15

Repository: RustPython/RustPython

Length of output: 440


🏁 Script executed:

# Check if there's any documentation about when C_RETURN should fire
rg -n "C_RETURN\|PY_RETURN" crates/vm/src/stdlib/sys/monitoring.rs | head -20

Repository: RustPython/RustPython

Length of output: 47


🏁 Script executed:

# Look at how fire_call and fire_c_return are paired in the codebase
rg -n "fire_call" crates/vm/src/frame.rs -B 3 -A 3 | head -40

Repository: RustPython/RustPython

Length of output: 715


🏁 Script executed:

# Check if PyFunction is handled specially anywhere else in monitoring
rg -n "PyFunction" crates/vm/src/stdlib/sys/monitoring.rs

Repository: RustPython/RustPython

Length of output: 47


🏁 Script executed:

# Look for how Python calls fire PY_RETURN to compare with C_RETURN
rg -n "fire_py_return\|fire_py_yield\|PY_RETURN" crates/vm/src/ -l

Repository: RustPython/RustPython

Length of output: 47


🏁 Script executed:

# Let's trace through what happens when PyBoundMethod wraps a PyFunction
# Check if there are any comments about this in the code
rg -n "bound\|wrapper\|wrapped" crates/vm/src/frame.rs | grep -i "call\|monitor" | head -10

Repository: RustPython/RustPython

Length of output: 47


🏁 Script executed:

# Let's look for tests that exercise monitoring with bound methods
find . -name "*.py" -path "*/tests/*" 2>/dev/null | xargs grep -l "method" 2>/dev/null | xargs grep -l "monitoring\|C_RETURN\|C_RAISE" 2>/dev/null

Repository: RustPython/RustPython

Length of output: 49


🏁 Script executed:

# Check if there's any special handling for bound methods elsewhere in monitoring
rg -n "bound" crates/vm/src/stdlib/sys/monitoring.rs

Repository: RustPython/RustPython

Length of output: 47


🏁 Script executed:

# Let's understand the semantics: does PyBoundMethod creating a frame?
# Look at how Python frames are created
rg -n "new_frame\|create_frame" crates/vm/src/frame.rs | head -10

Repository: RustPython/RustPython

Length of output: 47


🏁 Script executed:

# The key question: when PyBoundMethod.call delegates to zelf.function.call,
# does that function itself create a frame? Let's check PyFunction.call
rg -n "impl Callable for PyFunction" crates/vm/src/builtins/function.rs -A 20

Repository: RustPython/RustPython

Length of output: 653


🏁 Script executed:

# Now let's understand the problem better. When PyBoundMethod wraps a PyFunction,
# what happens? Let's trace: PyBoundMethod.call -> zelf.function.call
# If zelf.function is PyFunction, then PyFunction.call -> PyFunction.invoke
# Let's check PyFunction.invoke to see if it creates a frame
rg -n "fn invoke" crates/vm/src/builtins/function.rs -A 5 | head -30

Repository: RustPython/RustPython

Length of output: 450


🏁 Script executed:

# Check if there are other callable types that create frames
rg -n "impl Callable for" crates/vm/src/ --type rust | head -20

Repository: RustPython/RustPython

Length of output: 1243


🏁 Script executed:

# The critical question: does PyFunction.invoke create a frame that fires PY_RETURN?
# Let's look at the invoke method
cat -n crates/vm/src/builtins/function.rs | sed -n '750,850p'

Repository: RustPython/RustPython

Length of output: 4031


🏁 Script executed:

# Also check: are there any existing issues or comments about this exact problem?
rg -n "is_python_call\|PyBoundMethod.*monitoring\|bound.*method.*return" crates/vm/src/frame.rs

Repository: RustPython/RustPython

Length of output: 47


🏁 Script executed:

# Let's verify: when is_python_call is false but the callable actually creates a Python frame,
# do we get both C_RETURN and PY_RETURN?
rg -n "fire_py_return\|fire_py_yield" crates/vm/src/frame.rs | head -10

Repository: RustPython/RustPython

Length of output: 47


🏁 Script executed:

# Let's see the invoke_with_locals implementation
cat -n crates/vm/src/builtins/function.rs | sed -n '519,592p'

Repository: RustPython/RustPython

Length of output: 3287


🏁 Script executed:

# Let's check if invoke creates a frame by looking for frame creation
rg -n "push_frame\|new_frame\|Frame {" crates/vm/src/builtins/function.rs | head -10

Repository: RustPython/RustPython

Length of output: 188


🏁 Script executed:

# Let's check the invoke_with_locals to see what it returns
sed -n '519,591p' crates/vm/src/builtins/function.rs

Repository: RustPython/RustPython

Length of output: 2716


🏁 Script executed:

# Check the actual fire_py_return to understand when it fires
rg -n "fire_py_return" crates/vm/src/ -A 5 | head -30

Repository: RustPython/RustPython

Length of output: 915


🏁 Script executed:

# Now let's check: in the execute_call_instrumented, what would happen if we call
# a PyBoundMethod wrapping a PyFunction?
# The PyBoundMethod.call prepends the object and calls zelf.function.call
# So if zelf.function is a PyFunction, it goes to PyFunction.invoke
# which creates a frame. When that frame returns, PY_RETURN fires.
# But from execute_call_instrumented's perspective:
# is_python_call = callable.downcast_ref::<PyFunction>() -> false (it's PyBoundMethod)
# So C_RETURN fires after the call succeeds
# So we'd get both C_RETURN (wrong) and PY_RETURN (correct)
# Let's verify if PY_RETURN fires by checking where it's called

rg -n "fire_py_return\|PY_RETURN" crates/vm/src/frame.rs | head -20

Repository: RustPython/RustPython

Length of output: 47


🏁 Script executed:

# Let's understand the actual impact. When PyBoundMethod wraps PyFunction,
# does it create a duplicate event problem? Let's check if there's special handling

# First, what happens in vm.run_frame?
rg -n "pub fn run_frame" crates/vm/src/ -A 10 | head -30

Repository: RustPython/RustPython

Length of output: 720


🏁 Script executed:

# Check if there are tests that show the expected behavior for monitoring bound methods
find . -name "test_*.py" -o -name "*_test.py" 2>/dev/null | xargs grep -l "bound.*method" 2>/dev/null | head -3

Repository: RustPython/RustPython

Length of output: 142


🏁 Script executed:

# Let's also check: does the review comment's claim about CPython hold true?
# Check if there are any CPython comparisons or references in the codebase
rg -n "CPython\|PyFunction_Check" crates/vm/src/ --type rust | head -10

Repository: RustPython/RustPython

Length of output: 47


🏁 Script executed:

# The core question: when is_python_call is false but we actually call Python code,
# do BOTH C_RETURN and PY_RETURN fire? Let's trace the semantics
# PY_RETURN fires when InstrumentedReturnValue is executed (line 2498)
# This happens inside vm.run_frame() for the callee frame
# C_RETURN fires here in execute_call_instrumented after the call completes

# So the question is: is it correct to have both events?
# Let's check if there's any suppression or filtering of events

rg -n "EVENT_C_RETURN\|EVENT_PY_RETURN" crates/vm/src/stdlib/sys/monitoring.rs | head -20

Repository: RustPython/RustPython

Length of output: 47


🏁 Script executed:

# Let's check functools.partial to see if it has the same issue
rg -n "impl Callable for PyPartial" crates/vm/src/stdlib/functools.rs -A 10

Repository: RustPython/RustPython

Length of output: 590


🏁 Script executed:

# Let's look at PyStaticMethod
rg -n "impl Callable for PyStaticMethod" crates/vm/src/builtins/staticmethod.rs -A 10

Repository: RustPython/RustPython

Length of output: 473


🏁 Script executed:

# Let's look at classmethod which also wraps callables
rg -n "PyClassMethod" crates/vm/src/builtins/classmethod.rs -A 5 | head -30

Repository: RustPython/RustPython

Length of output: 912


🏁 Script executed:

# Now let's see if there are any tests that would catch this issue
# Search for test files that test monitoring behavior
find . -path "*/tests/*" -name "*.py" 2>/dev/null | xargs grep -l "fire\|C_RETURN\|C_RAISE" 2>/dev/null

Repository: RustPython/RustPython

Length of output: 89


🏁 Script executed:

# Let's check if there's a test suite for monitoring
find . -path "*/tests/*" -name "*monitor*.py" 2>/dev/null

Repository: RustPython/RustPython

Length of output: 47


🏁 Script executed:

# Let's look at the actual behavior expected - what does CPython do?
# Let's check the PR or issue that introduced this code
git log --oneline -n 30 -- crates/vm/src/frame.rs | head -20

Repository: RustPython/RustPython

Length of output: 75


🏁 Script executed:

# Let's trace what happens when we have a PyBoundMethod wrapping a PyFunction
# In execute_call_instrumented:
# 1. is_python_call = callable.downcast_ref::<PyFunction>().is_some() -> FALSE (it's PyBoundMethod)
# 2. callable.call() is called
# 3. PyBoundMethod.call calls zelf.function.call() 
# 4. If zelf.function is PyFunction, it creates a frame
# 5. The frame execution eventually hits InstrumentedReturnValue and fires PY_RETURN
# 6. Back in execute_call_instrumented, we fire C_RETURN (because is_python_call = false)

# So we get BOTH C_RETURN and PY_RETURN for the same logical call.
# Let's verify by checking the learning from the retrieved context

# The learning mentioned is about a different file (ir.rs), so let's search for any
# existing awareness of this issue

rg -n "bound.*method.*monitor\|C_RETURN.*bound\|duplicate.*event" crates/vm/src/ --type rust

Repository: RustPython/RustPython

Length of output: 47


🏁 Script executed:

# Let's also check: are there actually existing tests for this?
find . -path "*/tests/*" -name "*.py" 2>/dev/null | xargs grep -l "bound.*method" 2>/dev/null | xargs grep -l "C_RETURN\|monitoring" 2>/dev/null

Repository: RustPython/RustPython

Length of output: 49


🏁 Script executed:

# Let's check if PyStaticMethod and PyPartial have the same issue
# Since they also wrap callables and delegate to them

# Summary of wrapper types that delegate to wrapped callables:
# PyBoundMethod - wraps (object, function) and calls zelf.function.call()
# PyPartial - wraps func + args and calls func.call()
# PyStaticMethod - wraps callable and calls callable.call()
# PyClassMethod - uses GetDescriptor to create bound methods

# All of these would have the same issue: if wrapping a PyFunction,
# is_python_call = false, but PY_RETURN still fires from the wrapped PyFunction

# The question is: should we check if the wrapped callable is Python?
# Or is the current behavior correct?

# Let's check CPython's source to understand the intent
web_search
CPython sys.monitoring INSTRUMENTED_CALL C_RETURN bound methods implementation

Repository: RustPython/RustPython

Length of output: 163


🏁 Script executed:

# Check if there are any Python tests in the test suite that show expected behavior
find . -path "*/tests/*" -name "test_*.py" 2>/dev/null | head -10

Repository: RustPython/RustPython

Length of output: 455


🏁 Script executed:

# Look for any README or documentation about monitoring
find . -name "README*" -o -name "MONITORING*" 2>/dev/null

Repository: RustPython/RustPython

Length of output: 608


🏁 Script executed:

# Let's check the code around the fire_call to understand the intent
sed -n '3260,3310p' crates/vm/src/frame.rs

Repository: RustPython/RustPython

Length of output: 1767


🏁 Script executed:

# Let's see if there are any comments in the code explaining the design
rg -n "is_python_call\|PyFunction.*Check\|bound.*method" crates/vm/src/frame.rs -B 5 -A 5 | head -50

Repository: RustPython/RustPython

Length of output: 47


🏁 Script executed:

# Check if there are inline comments explaining the semantics
sed -n '3240,3270p' crates/vm/src/frame.rs

Repository: RustPython/RustPython

Length of output: 1046


🏁 Script executed:

# Let's verify one more thing: check if there's any existing handling or discussion of this issue
rg -n "TODO\|FIXME\|BUG\|XXX" crates/vm/src/frame.rs | grep -i "python\|call\|monitor" | head -10

Repository: RustPython/RustPython

Length of output: 47


🏁 Script executed:

# Also check for any related issues in comments around line 3266
sed -n '3240,3280p' crates/vm/src/frame.rs | grep -E "//"

Repository: RustPython/RustPython

Length of output: 179


is_python_call check misclassifies Python callable wrappers as C calls, causing duplicate monitoring events.

When PyBoundMethod, PyStaticMethod, PyPartial, or similar wrappers delegate to a PyFunction, they are incorrectly identified as C calls. The check callable.downcast_ref::<PyFunction>().is_some() only matches bare PyFunction objects. A PyBoundMethod wrapping a PyFunction prepends the object and calls zelf.function.call(), which creates a frame and fires PY_RETURN when it returns. However, because is_python_call is false, C_RETURN also fires in execute_call_instrumented, resulting in duplicate events for a single Python call.

The same issue affects PyStaticMethod, PyPartial, and any wrapper that delegates to a Python callable. To fix this, check whether the wrapped callable is ultimately Python, not just whether the immediate callable is a bare PyFunction.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/frame.rs` around lines 3266 - 3307, The is_python_call
detection is too strict (it only checks callable.downcast_ref::<PyFunction>())
and mislabels wrapped Python callables (PyBoundMethod, PyStaticMethod,
PyPartial, etc.), causing duplicate monitoring events; update
execute_call_instrumented to replace the single-line is_python_call check with a
small helper (e.g., is_underlying_py_function or is_python_callable) that
recursively unwraps known wrapper types (PyBoundMethod, PyStaticMethod,
PyPartial, and any other delegating types in your codebase) to inspect the
wrapped callable and return true if the ultimate target is a PyFunction, then
use that helper where is_python_call is used (for monitoring::fire_c_return and
monitoring::fire_c_raise) so only true C calls trigger C_RETURN/C_RAISE.

@youknowone youknowone force-pushed the instrumented-instruction branch from 97bd821 to 56bb7b8 Compare February 26, 2026 09:05
@github-actions
Copy link
Contributor

Code has been automatically formatted

The code in this PR has been formatted using:

  • cargo fmt --all
    Please pull the latest changes before pushing again:
git pull origin instrumented-instruction

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

♻️ Duplicate comments (2)
crates/vm/src/frame.rs (2)

261-270: ⚠️ Potential issue | 🟠 Major

set_lasti still leaves FrameState.lasti stale in threading mode.

Line 264 updates only the atomic field. But ExecutingFrame::lasti() reads state.lasti (Line 469), so external f_lasti writes can be ignored while executing.

Suggested fix
 pub fn set_lasti(&self, val: u32) {
     #[cfg(feature = "threading")]
     {
         self.lasti.store(val, atomic::Ordering::Relaxed);
+        self.state.lock().lasti = val;
     }
     #[cfg(not(feature = "threading"))]
     {
         self.lasti.set(val);
     }
 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/frame.rs` around lines 261 - 270, The threading build currently
updates only the atomic lasti in Frame::set_lasti, leaving FrameState.lasti
stale and causing ExecutingFrame::lasti to read an outdated value; change
Frame::set_lasti so that in the #[cfg(feature = "threading")] branch it writes
the atomic (self.lasti.store(...)) and also updates the non-atomic backing field
(state.lasti or self.state.lasti.set/assign) so both places remain consistent,
or alternatively change ExecutingFrame::lasti to read the atomic lasti; update
the code paths involving set_lasti, FrameState.lasti, and ExecutingFrame::lasti
accordingly.

3271-3271: ⚠️ Potential issue | 🟠 Major

is_python_call still misclassifies wrapped Python callables as C calls.

Line 3271 only matches bare PyFunction. Wrappers (e.g. bound/static/partial-like delegates) can still execute Python frames, which can lead to wrong C_* event classification.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/frame.rs` at line 3271, is_python_call currently only checks
callable.downcast_ref::<PyFunction>(), misclassifying wrapped Python callables;
change the detection to treat any object whose __call__ is a PyFunction as a
Python call too by replacing the single downcast_ref check with a two-step
check: first keep callable.downcast_ref::<PyFunction>().is_some(), and if not,
attempt to retrieve callable.getattr("__call__") and test whether that attribute
downcasts to PyFunction (handling/propagating errors safely), so
bound/static/partial-like wrappers are classified as Python calls; reference the
local variables is_python_call and callable when making the change.
🧹 Nitpick comments (1)
crates/vm/src/builtins/code.rs (1)

916-987: LGTM! Previous NOT_TAKEN concern addressed; minor robustness consideration.

The implementation correctly handles de-instrumentation via to_base() and properly checks for NotTaken presence (lines 958-965) before computing the fall-through offset.

One minor consideration: the EndAsyncFor branch (lines 970-971) performs next_i - oparg as usize which could panic on subtraction underflow if the bytecode is malformed. While valid bytecode should never trigger this, using saturating_sub or checked_sub would make this more defensive against corrupted inputs.

Optional defensive fix
 Instruction::EndAsyncFor => {
     // src = END_SEND position (next_i - oparg)
     let next_i = i + 1;
-    let src_i = next_i - oparg as usize;
+    let src_i = next_i.saturating_sub(oparg as usize);
     (src_i * 2, (src_i + 2) * 2, next_i * 2)
 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/builtins/code.rs` around lines 916 - 987, co_branches currently
computes src_i for the EndAsyncFor case as next_i - oparg as usize which can
underflow on malformed bytecode; update the EndAsyncFor branch in co_branches to
use a safe subtraction (e.g., saturating_sub or checked_sub with an early
continue or error) when deriving src_i from next_i and oparg so you avoid panics
on underflow while preserving existing behavior for valid bytecode.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@crates/compiler-core/src/bytecode/instruction.rs`:
- Around line 464-469: Update the doc comment for the to_base method to clarify
that returning None means “there is no corresponding base opcode” rather than
implying the variant is not instrumented; explicitly note that some instrumented
variants (e.g., InstrumentedLine and InstrumentedInstruction) do not have base
equivalents and will return None, and advise callers not to use
to_base().is_some() as a general instrumentation check but to check the opcode
variant (or provide/point to an is_instrumented helper) when determining
instrumentation. Reference the to_base function and the
InstrumentedLine/InstrumentedInstruction variants in the comment so callers
understand the intended contract.

In `@crates/vm/src/frame.rs`:
- Around line 3274-3304: call_arg0 is only computed when monitoring::EVENT_CALL
is enabled, which prevents emitting C_RETURN/C_RAISE when only those events are
requested; compute/extract the argument whenever any of the relevant events are
enabled (EVENT_CALL, EVENT_C_RETURN, EVENT_C_RAISE) by checking
self.monitoring_mask for that combined set and calling
final_args.args.first().cloned().unwrap_or_else(|| monitoring::get_missing(vm))
into a variable (e.g., call_arg0) before invoking callable.call; then, only call
monitoring::fire_call when EVENT_CALL is set, but still pass the extracted
call_arg0 to monitoring::fire_c_return and monitoring::fire_c_raise so those
events are emitted even if EVENT_CALL was disabled (use existing symbols:
monitoring::fire_call, monitoring::fire_c_return, monitoring::fire_c_raise,
monitoring::get_missing, call_arg0, self.lasti(), callable.call).

---

Duplicate comments:
In `@crates/vm/src/frame.rs`:
- Around line 261-270: The threading build currently updates only the atomic
lasti in Frame::set_lasti, leaving FrameState.lasti stale and causing
ExecutingFrame::lasti to read an outdated value; change Frame::set_lasti so that
in the #[cfg(feature = "threading")] branch it writes the atomic
(self.lasti.store(...)) and also updates the non-atomic backing field
(state.lasti or self.state.lasti.set/assign) so both places remain consistent,
or alternatively change ExecutingFrame::lasti to read the atomic lasti; update
the code paths involving set_lasti, FrameState.lasti, and ExecutingFrame::lasti
accordingly.
- Line 3271: is_python_call currently only checks
callable.downcast_ref::<PyFunction>(), misclassifying wrapped Python callables;
change the detection to treat any object whose __call__ is a PyFunction as a
Python call too by replacing the single downcast_ref check with a two-step
check: first keep callable.downcast_ref::<PyFunction>().is_some(), and if not,
attempt to retrieve callable.getattr("__call__") and test whether that attribute
downcasts to PyFunction (handling/propagating errors safely), so
bound/static/partial-like wrappers are classified as Python calls; reference the
local variables is_python_call and callable when making the change.

---

Nitpick comments:
In `@crates/vm/src/builtins/code.rs`:
- Around line 916-987: co_branches currently computes src_i for the EndAsyncFor
case as next_i - oparg as usize which can underflow on malformed bytecode;
update the EndAsyncFor branch in co_branches to use a safe subtraction (e.g.,
saturating_sub or checked_sub with an early continue or error) when deriving
src_i from next_i and oparg so you avoid panics on underflow while preserving
existing behavior for valid bytecode.

ℹ️ Review info

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 97bd821 and d647b47.

⛔ Files ignored due to path filters (1)
  • Lib/test/test_peepholer.py is excluded by !Lib/**
📒 Files selected for processing (5)
  • crates/codegen/src/ir.rs
  • crates/compiler-core/src/bytecode.rs
  • crates/compiler-core/src/bytecode/instruction.rs
  • crates/vm/src/builtins/code.rs
  • crates/vm/src/frame.rs

Comment on lines +3274 to +3304
let call_arg0 = if self.monitoring_mask & monitoring::EVENT_CALL != 0 {
let arg0 = final_args
.args
.first()
.cloned()
.unwrap_or_else(|| monitoring::get_missing(vm));
let offset = (self.lasti() - 1) * 2;
monitoring::fire_call(vm, self.code, offset, &callable, arg0.clone())?;
Some(arg0)
} else {
None
};

match callable.call(final_args, vm) {
Ok(value) => {
if let Some(arg0) = call_arg0
&& !is_python_call
{
let offset = (self.lasti() - 1) * 2;
monitoring::fire_c_return(vm, self.code, offset, &callable, arg0)?;
}
self.push_value(value);
Ok(None)
}
Err(exc) => {
let exc = if let Some(arg0) = call_arg0
&& !is_python_call
{
let offset = (self.lasti() - 1) * 2;
match monitoring::fire_c_raise(vm, self.code, offset, &callable, arg0) {
Ok(()) => exc,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

C_RETURN/C_RAISE are incorrectly suppressed when CALL is disabled.

call_arg0 is only built when EVENT_CALL is enabled (Lines 3274-3285), but C_RETURN/C_RAISE emission depends on call_arg0 (Lines 3289-3303). If a tool enables only C_RETURN/C_RAISE, those events are dropped.

Suggested fix
-        // Fire CALL event
-        let call_arg0 = if self.monitoring_mask & monitoring::EVENT_CALL != 0 {
-            let arg0 = final_args
-                .args
-                .first()
-                .cloned()
-                .unwrap_or_else(|| monitoring::get_missing(vm));
-            let offset = (self.lasti() - 1) * 2;
-            monitoring::fire_call(vm, self.code, offset, &callable, arg0.clone())?;
-            Some(arg0)
-        } else {
-            None
-        };
+        let monitor_call = self.monitoring_mask & monitoring::EVENT_CALL != 0;
+        let monitor_c_return = self.monitoring_mask & monitoring::EVENT_C_RETURN != 0;
+        let monitor_c_raise = self.monitoring_mask & monitoring::EVENT_C_RAISE != 0;
+        let need_arg0 = monitor_call || (!is_python_call && (monitor_c_return || monitor_c_raise));
+
+        let call_arg0 = if need_arg0 {
+            Some(
+                final_args
+                    .args
+                    .first()
+                    .cloned()
+                    .unwrap_or_else(|| monitoring::get_missing(vm)),
+            )
+        } else {
+            None
+        };
+
+        if monitor_call {
+            let offset = (self.lasti() - 1) * 2;
+            monitoring::fire_call(
+                vm,
+                self.code,
+                offset,
+                &callable,
+                call_arg0.clone().unwrap(),
+            )?;
+        }
@@
-                if let Some(arg0) = call_arg0
-                    && !is_python_call
+                if let Some(arg0) = call_arg0.clone()
+                    && !is_python_call
+                    && monitor_c_return
                 {
                     let offset = (self.lasti() - 1) * 2;
                     monitoring::fire_c_return(vm, self.code, offset, &callable, arg0)?;
                 }
@@
-                let exc = if let Some(arg0) = call_arg0
-                    && !is_python_call
+                let exc = if let Some(arg0) = call_arg0
+                    && !is_python_call
+                    && monitor_c_raise
                 {
                     let offset = (self.lasti() - 1) * 2;
                     match monitoring::fire_c_raise(vm, self.code, offset, &callable, arg0) {
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/frame.rs` around lines 3274 - 3304, call_arg0 is only computed
when monitoring::EVENT_CALL is enabled, which prevents emitting C_RETURN/C_RAISE
when only those events are requested; compute/extract the argument whenever any
of the relevant events are enabled (EVENT_CALL, EVENT_C_RETURN, EVENT_C_RAISE)
by checking self.monitoring_mask for that combined set and calling
final_args.args.first().cloned().unwrap_or_else(|| monitoring::get_missing(vm))
into a variable (e.g., call_arg0) before invoking callable.call; then, only call
monitoring::fire_call when EVENT_CALL is set, but still pass the extracted
call_arg0 to monitoring::fire_c_return and monitoring::fire_c_raise so those
events are emitted even if EVENT_CALL was disabled (use existing symbols:
monitoring::fire_call, monitoring::fire_c_return, monitoring::fire_c_raise,
monitoring::get_missing, call_arg0, self.lasti(), callable.call).

@youknowone youknowone force-pushed the instrumented-instruction branch from d647b47 to f17b4da Compare February 26, 2026 12:37
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

♻️ Duplicate comments (4)
crates/vm/src/frame.rs (2)

3269-3269: ⚠️ Potential issue | 🟠 Major

is_python_call misclassifies wrapped Python callables.

Line 3269 only detects bare PyFunction, so wrappers (e.g. bound/static/partial wrappers around Python functions) can incorrectly emit C_RETURN/C_RAISE in addition to Python-level events.

[suggested fix: unwrap known wrapper callables before classification, then check the terminal callable type.]

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/frame.rs` at line 3269, The current is_python_call check only
tests for PyFunction and thus misclassifies wrapped callables; add logic to
unwrap known wrapper types (e.g. bound method wrappers, functools.partial-like
wrappers, staticmethod/classmethod wrappers) before testing the terminal
callable type. Implement or call a helper (e.g. unwrap_callable or similar) that
strips PyMethod/partial/descriptor wrappers and then replace the direct
downcast_ref::<PyFunction>() check with a downcast on the unwrapped object so
is_python_call reflects the terminal callable type.

2531-2554: ⚠️ Potential issue | 🟠 Major

C_RETURN/C_RAISE emission is incorrectly tied to EVENT_CALL.

At Line 3272 and Line 2531, arg capture is guarded by EVENT_CALL; when a tool enables only EVENT_C_RETURN/EVENT_C_RAISE, those events can be dropped.

Suggested fix pattern
- let call_arg0 = if self.monitoring_mask & monitoring::EVENT_CALL != 0 {
+ let monitor_call = self.monitoring_mask & monitoring::EVENT_CALL != 0;
+ let monitor_c_return = self.monitoring_mask & monitoring::EVENT_C_RETURN != 0;
+ let monitor_c_raise = self.monitoring_mask & monitoring::EVENT_C_RAISE != 0;
+ let need_arg0 = monitor_call || (monitor_c_return || monitor_c_raise);
+ let call_arg0 = if need_arg0 {
     ...
   } else {
     None
   };

- if let Some(arg0) = call_arg0 && !is_python_call {
+ if let Some(arg0) = call_arg0.clone() && !is_python_call && monitor_c_return {
     ...
   }

- if let Some(arg0) = call_arg0 && !is_python_call {
+ if let Some(arg0) = call_arg0 && !is_python_call && monitor_c_raise {
     ...
   }

Also applies to: 3271-3303

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/frame.rs` around lines 2531 - 2554, The code currently ties
capturing call-site arguments to monitoring::EVENT_CALL (via monitoring_mask and
call_args), so handlers that enable only EVENT_C_RETURN or EVENT_C_RAISE will
not receive the captured args; change the logic so capturing of
global_super/arg0 is performed whenever EVENT_C_RETURN or EVENT_C_RAISE is
enabled (i.e. check monitoring::EVENT_C_RETURN|monitoring::EVENT_C_RAISE in
addition to or instead of EVENT_CALL), ensure
nth_value(2)/nth_value(1)/monitoring::get_missing are used to populate the tuple
used by monitoring::fire_c_return and monitoring::fire_c_raise, and keep
fire_call only under EVENT_CALL while using the newly-captured call_args for the
C_RETURN/C_RAISE paths in load_super_attr success and Err branches (symbols:
monitoring_mask, EVENT_CALL, EVENT_C_RETURN, EVENT_C_RAISE, call_args,
fire_call, fire_c_return, fire_c_raise, load_super_attr, nth_value,
get_missing).
crates/compiler-core/src/bytecode/instruction.rs (1)

464-466: ⚠️ Potential issue | 🟡 Minor

Clarify to_base() None semantics.

Line 465 currently implies None means “not instrumented,” but Instruction::InstrumentedLine and Instruction::InstrumentedInstruction are instrumented and also return None. Please document that None can also mean “instrumented opcode without a base equivalent.”

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/compiler-core/src/bytecode/instruction.rs` around lines 464 - 466,
Update the doc comment for the to_base() method to clarify the semantics of a
None return: state that None is returned both for opcodes that are not
instrumented and for instrumented opcodes that do not have a base equivalent
(specifically mention Instruction::InstrumentedLine and
Instruction::InstrumentedInstruction as examples of instrumented variants that
also return None), so callers know None can mean either “not instrumented” or
“instrumented without a base variant.”
crates/compiler-core/src/bytecode.rs (1)

420-423: ⚠️ Potential issue | 🟡 Minor

Unify the safety contract between Deref and replace_op.

Lines 420-423 say shared reads are valid while replace_op mutates, but Lines 433-434 require exclusive access with no concurrent reads/writes. These guarantees conflict; please keep a single contract (exclusive access) to avoid implying racy reads are permitted.

#!/bin/bash
# Verify the conflicting safety statements in CodeUnits.
rg -n "Shared references to the slice are valid even while replace_op|exclusive access to the instruction buffer" crates/compiler-core/src/bytecode.rs -C2

Also applies to: 433-435

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/compiler-core/src/bytecode.rs` around lines 420 - 423, Unify the
safety contract by requiring exclusive access for both the Deref path and
replace_op: update the unsafe comment in the Deref implementation (the unsafe {
&*self.0.get() } block) to state that callers must ensure exclusive,
no-concurrent-reads/writes access to the underlying instruction buffer (remove
the line claiming shared reads are valid), and make the replace_op
documentation/unsafe contract match (i.e., both must assert exclusive access to
the instruction buffer); reference the Deref impl around the unsafe get() and
the replace_op function so the two comments are consistent and no longer imply
racy shared reads are permitted.
🧹 Nitpick comments (1)
crates/vm/src/stdlib/sys/monitoring.rs (1)

341-347: Redundant .take(len) in iterator chain.

The iterator code.code.instructions.iter() already yields exactly len items (since len = code.code.instructions.len()), making .take(len) unnecessary.

Suggested simplification
         for (i, unit) in code
             .code
             .instructions
             .iter()
             .enumerate()
-            .take(len)
             .skip(first_traceable)
         {

The same applies at line 437-442.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/stdlib/sys/monitoring.rs` around lines 341 - 347, Remove the
redundant .take(len) from the iterator chain(s) that enumerate
code.code.instructions (where len = code.code.instructions.len())—e.g., in the
loop using
code.code.instructions.iter().enumerate().take(len).skip(first_traceable) remove
.take(len) so the chain becomes enumerate().skip(first_traceable); do the same
for the second occurrence referenced around the later loop (the other enumerate
over code.code.instructions with .take(len)). This removes unnecessary
truncation while preserving use of len and first_traceable.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@crates/codegen/src/compile.rs`:
- Around line 5190-5206: At the start of compile_for, capture the
for-statement's full source range (e.g., let stmt_range = stmt.range() or
equivalent) and then use that captured stmt_range wherever the code currently
calls iter.range() for loop-boundary attribution; specifically replace
iter.range() in the set_source_range calls before emitting
Instruction::EndAsyncFor / Instruction::EndFor / Instruction::PopIter and for
the implicit-return attribution so the EndFor/PopIter and implicit return are
attributed to the for statement line rather than the iterator expression. Ensure
the new stmt_range is used instead of iter.range() in both places and keep
preserving/restoring saved_range behavior as before.

In `@crates/vm/src/builtins/code.rs`:
- Around line 968-972: The subtraction in the Instruction::EndAsyncFor arm can
underflow when computing src_i (next_i - oparg); modify the arm in the code that
builds branch offsets to use checked_sub on next_i and oparg, validate the
result before computing (src_i * 2, (src_i + 2) * 2, next_i * 2), and skip or
return an error for entries where checked_sub returns None so malformed bytecode
cannot produce invalid offsets (reference Instruction::EndAsyncFor, next_i,
src_i, and oparg).
- Around line 925-927: co_branches() currently de-instruments opcodes using
unit.op.to_base() which loses InstrumentedLine/InstrumentedInstruction variants
and thus skips displaced branch opcodes; change the de-instrument step to look
up the original opcode from self.monitoring_data by the instruction index (the
same index used to read unit/op) and use that restored opcode for matching
(instead of or in addition to to_base()) when computing op and when performing
the target-op checks around the logic that uses unit.op and unit.arg (this also
applies to the code paths around the checks at the later target-op logic), so
branch tuples include instrumented variants and displaced branches are not
missed.

In `@crates/vm/src/stdlib/sys/monitoring.rs`:
- Around line 886-903: The RERAISE_PENDING flag is set true before calling
fire(...) in fire_reraise, but if fire returns an Err the flag remains true and
suppresses future reraise events; change fire_reraise to capture the result of
fire(...) and, on error, reset RERAISE_PENDING.with(|f| f.set(false)) before
returning the Err so the flag is not left stuck; keep references to
RERAISE_PENDING, fire_reraise, fire, and EVENT_RERAISE when locating the change.

---

Duplicate comments:
In `@crates/compiler-core/src/bytecode.rs`:
- Around line 420-423: Unify the safety contract by requiring exclusive access
for both the Deref path and replace_op: update the unsafe comment in the Deref
implementation (the unsafe { &*self.0.get() } block) to state that callers must
ensure exclusive, no-concurrent-reads/writes access to the underlying
instruction buffer (remove the line claiming shared reads are valid), and make
the replace_op documentation/unsafe contract match (i.e., both must assert
exclusive access to the instruction buffer); reference the Deref impl around the
unsafe get() and the replace_op function so the two comments are consistent and
no longer imply racy shared reads are permitted.

In `@crates/compiler-core/src/bytecode/instruction.rs`:
- Around line 464-466: Update the doc comment for the to_base() method to
clarify the semantics of a None return: state that None is returned both for
opcodes that are not instrumented and for instrumented opcodes that do not have
a base equivalent (specifically mention Instruction::InstrumentedLine and
Instruction::InstrumentedInstruction as examples of instrumented variants that
also return None), so callers know None can mean either “not instrumented” or
“instrumented without a base variant.”

In `@crates/vm/src/frame.rs`:
- Line 3269: The current is_python_call check only tests for PyFunction and thus
misclassifies wrapped callables; add logic to unwrap known wrapper types (e.g.
bound method wrappers, functools.partial-like wrappers, staticmethod/classmethod
wrappers) before testing the terminal callable type. Implement or call a helper
(e.g. unwrap_callable or similar) that strips PyMethod/partial/descriptor
wrappers and then replace the direct downcast_ref::<PyFunction>() check with a
downcast on the unwrapped object so is_python_call reflects the terminal
callable type.
- Around line 2531-2554: The code currently ties capturing call-site arguments
to monitoring::EVENT_CALL (via monitoring_mask and call_args), so handlers that
enable only EVENT_C_RETURN or EVENT_C_RAISE will not receive the captured args;
change the logic so capturing of global_super/arg0 is performed whenever
EVENT_C_RETURN or EVENT_C_RAISE is enabled (i.e. check
monitoring::EVENT_C_RETURN|monitoring::EVENT_C_RAISE in addition to or instead
of EVENT_CALL), ensure nth_value(2)/nth_value(1)/monitoring::get_missing are
used to populate the tuple used by monitoring::fire_c_return and
monitoring::fire_c_raise, and keep fire_call only under EVENT_CALL while using
the newly-captured call_args for the C_RETURN/C_RAISE paths in load_super_attr
success and Err branches (symbols: monitoring_mask, EVENT_CALL, EVENT_C_RETURN,
EVENT_C_RAISE, call_args, fire_call, fire_c_return, fire_c_raise,
load_super_attr, nth_value, get_missing).

---

Nitpick comments:
In `@crates/vm/src/stdlib/sys/monitoring.rs`:
- Around line 341-347: Remove the redundant .take(len) from the iterator
chain(s) that enumerate code.code.instructions (where len =
code.code.instructions.len())—e.g., in the loop using
code.code.instructions.iter().enumerate().take(len).skip(first_traceable) remove
.take(len) so the chain becomes enumerate().skip(first_traceable); do the same
for the second occurrence referenced around the later loop (the other enumerate
over code.code.instructions with .take(len)). This removes unnecessary
truncation while preserving use of len and first_traceable.

ℹ️ Review info

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d647b47 and d0d1d1e.

⛔ Files ignored due to path filters (8)
  • Lib/opcode.py is excluded by !Lib/**
  • Lib/test/dis_module.py is excluded by !Lib/**
  • Lib/test/profilee.py is excluded by !Lib/**
  • Lib/test/test_compile.py is excluded by !Lib/**
  • Lib/test/test_dis.py is excluded by !Lib/**
  • Lib/test/test_monitoring.py is excluded by !Lib/**
  • Lib/test/test_peepholer.py is excluded by !Lib/**
  • crates/codegen/src/snapshots/rustpython_codegen__compile__tests__nested_double_async_with.snap is excluded by !**/*.snap
📒 Files selected for processing (14)
  • .cspell.dict/cpython.txt
  • crates/codegen/src/compile.rs
  • crates/codegen/src/ir.rs
  • crates/compiler-core/src/bytecode.rs
  • crates/compiler-core/src/bytecode/instruction.rs
  • crates/vm/src/builtins/code.rs
  • crates/vm/src/builtins/frame.rs
  • crates/vm/src/frame.rs
  • crates/vm/src/stdlib/sys.rs
  • crates/vm/src/stdlib/sys/monitoring.rs
  • crates/vm/src/types/slot.rs
  • crates/vm/src/vm/interpreter.rs
  • crates/vm/src/vm/mod.rs
  • scripts/update_lib/deps.py
🚧 Files skipped from review as they are similar to previous changes (5)
  • crates/vm/src/types/slot.rs
  • crates/vm/src/stdlib/sys.rs
  • crates/vm/src/vm/mod.rs
  • crates/vm/src/builtins/frame.rs
  • scripts/update_lib/deps.py

Comment on lines +5190 to +5206
// End-of-loop instructions are on the `for` line, not the body's last line
let saved_range = self.current_source_range;
self.set_source_range(iter.range());
if is_async {
emit!(self, Instruction::EndAsyncFor);
} else {
// END_FOR + POP_ITER pattern (CPython 3.14)
// FOR_ITER jumps to END_FOR, but VM skips it (+1) to reach POP_ITER
emit!(self, Instruction::EndFor);
emit!(self, Instruction::PopIter);
}
self.set_source_range(saved_range);
self.compile_statements(orelse)?;

self.switch_to_block(after_block);

// Implicit return after for-loop should be attributed to the `for` line
self.set_source_range(iter.range());

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Use the for statement range, not iter.range(), for loop-boundary attribution.

iter.range() can point to a different line in multiline for headers, so EndFor / PopIter / implicit return can be attributed to the wrong line. Capture the statement range at compile_for entry and reuse it.

💡 Proposed fix
 fn compile_for(
     &mut self,
     target: &ast::Expr,
     iter: &ast::Expr,
     body: &[ast::Stmt],
     orelse: &[ast::Stmt],
     is_async: bool,
 ) -> CompileResult<()> {
     self.enter_conditional_block();
+    let for_stmt_range = self.current_source_range;
@@
-        self.set_source_range(iter.range());
+        self.set_source_range(for_stmt_range);
         if is_async {
             emit!(self, Instruction::EndAsyncFor);
         } else {
             emit!(self, Instruction::EndFor);
             emit!(self, Instruction::PopIter);
         }
         self.set_source_range(saved_range);
@@
-        self.set_source_range(iter.range());
+        self.set_source_range(for_stmt_range);
 
         self.leave_conditional_block();
         Ok(())
     }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/codegen/src/compile.rs` around lines 5190 - 5206, At the start of
compile_for, capture the for-statement's full source range (e.g., let stmt_range
= stmt.range() or equivalent) and then use that captured stmt_range wherever the
code currently calls iter.range() for loop-boundary attribution; specifically
replace iter.range() in the set_source_range calls before emitting
Instruction::EndAsyncFor / Instruction::EndFor / Instruction::PopIter and for
the implicit-return attribution so the EndFor/PopIter and implicit return are
attributed to the for statement line rather than the iterator expression. Ensure
the new stmt_range is used instead of iter.range() in both places and keep
preserving/restoring saved_range behavior as before.

Comment on lines +925 to +927
// De-instrument: use base opcode for instrumented variants
let op = unit.op.to_base().unwrap_or(unit.op);
let raw_arg = u32::from(u8::from(unit.arg));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

co_branches() misses branches when opcodes are displaced by INSTRUMENTED_LINE/INSTRUMENTED_INSTRUCTION.

Line 926 only de-instruments via to_base(). That does not recover InstrumentedLine/InstrumentedInstruction, so displaced branch opcodes can be skipped and branch tuples become incomplete when those events are enabled. Use self.monitoring_data to restore original opcodes by index before matching (including target-op checks like Line 942).

Suggested fix direction
+        let monitoring = self.monitoring_data.lock();
+        let original_op = |idx: usize, op: Instruction| -> Instruction {
+            match op {
+                Instruction::InstrumentedLine => monitoring
+                    .as_ref()
+                    .and_then(|m| m.line_opcodes.get(idx).copied())
+                    .filter(|b| *b != 0)
+                    .and_then(|b| Instruction::try_from(b).ok())
+                    .unwrap_or(op),
+                Instruction::InstrumentedInstruction => monitoring
+                    .as_ref()
+                    .and_then(|m| m.per_instruction_opcodes.get(idx).copied())
+                    .filter(|b| *b != 0)
+                    .and_then(|b| Instruction::try_from(b).ok())
+                    .unwrap_or(op),
+                _ => op.to_base().unwrap_or(op),
+            }
+        };
...
-            let op = unit.op.to_base().unwrap_or(unit.op);
+            let op = original_op(i, unit.op);
...
-                        instructions.get(target).map(|u| u.op),
+                        instructions.get(target).map(|u| original_op(target, u.op)),

Also applies to: 942-945

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/builtins/code.rs` around lines 925 - 927, co_branches()
currently de-instruments opcodes using unit.op.to_base() which loses
InstrumentedLine/InstrumentedInstruction variants and thus skips displaced
branch opcodes; change the de-instrument step to look up the original opcode
from self.monitoring_data by the instruction index (the same index used to read
unit/op) and use that restored opcode for matching (instead of or in addition to
to_base()) when computing op and when performing the target-op checks around the
logic that uses unit.op and unit.arg (this also applies to the code paths around
the checks at the later target-op logic), so branch tuples include instrumented
variants and displaced branches are not missed.

@youknowone youknowone force-pushed the instrumented-instruction branch from d0d1d1e to 39c06eb Compare February 26, 2026 17:11
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

♻️ Duplicate comments (2)
crates/vm/src/builtins/code.rs (1)

925-927: ⚠️ Potential issue | 🟠 Major

Restore displaced opcodes from monitoring_data in co_branches().

Line 926 only uses to_base(), so InstrumentedLine / InstrumentedInstruction cannot be mapped back to their original opcodes. That makes branch extraction incomplete on instrumented code, including target checks at Line 942 and NOT_TAKEN fallthrough detection at Line 961.

Proposed fix
 pub fn co_branches(&self, vm: &VirtualMachine) -> PyResult<PyObjectRef> {
     use bytecode::Instruction;

     let instructions = &self.code.instructions;
+    let monitoring = self.monitoring_data.lock();
+    let original_op = |idx: usize, op: Instruction| -> Instruction {
+        match op {
+            Instruction::InstrumentedLine => monitoring
+                .as_ref()
+                .and_then(|m| m.line_opcodes.get(idx).copied())
+                .filter(|b| *b != 0)
+                .and_then(|b| Instruction::try_from(b).ok())
+                .unwrap_or(op),
+            Instruction::InstrumentedInstruction => monitoring
+                .as_ref()
+                .and_then(|m| m.per_instruction_opcodes.get(idx).copied())
+                .filter(|b| *b != 0)
+                .and_then(|b| Instruction::try_from(b).ok())
+                .unwrap_or(op),
+            _ => op.to_base().unwrap_or(op),
+        }
+    };
     let mut branches = Vec::new();
     let mut extended_arg: u32 = 0;

     for (i, unit) in instructions.iter().enumerate() {
-        let op = unit.op.to_base().unwrap_or(unit.op);
+        let op = original_op(i, unit.op);
         let raw_arg = u32::from(u8::from(unit.arg));
@@
-                    let right = if matches!(
-                        instructions.get(target).map(|u| u.op),
+                    let right = if matches!(
+                        instructions.get(target).map(|u| original_op(target, u.op)),
                         Some(Instruction::EndFor) | Some(Instruction::InstrumentedEndFor)
                     ) {
@@
                     let next_op = instructions
                         .get(i + 1)
-                        .map(|u| u.op.to_base().unwrap_or(u.op));
+                        .map(|u| original_op(i + 1, u.op));

Also applies to: 942-945, 958-961

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/builtins/code.rs` around lines 925 - 927, In co_branches(),
when de-instrumenting opcodes (currently using
unit.op.to_base().unwrap_or(unit.op)), consult monitoring_data to restore any
displaced InstrumentedLine/InstrumentedInstruction back to their original opcode
first (e.g., look up the original opcode by the unit's offset/index in
monitoring_data and use it), then fall back to to_base() and finally unit.op;
apply the same restoration logic at the other locations that compute branch
targets and NOT_TAKEN fallthrough (the checks around the target computation and
fallthrough detection currently at the later blocks referenced in the comment)
so instrumented instructions map correctly to their original opcodes.
crates/vm/src/frame.rs (1)

3378-3379: ⚠️ Potential issue | 🟠 Major

Unwrap delegated Python callables before deciding C-return/C-raise emission.

Line 3378 only recognizes bare PyFunction. Wrapper callables that delegate to Python functions can be misclassified as C calls, which can produce incorrect/duplicate monitoring events on Line 3396 and Line 3406.

Also applies to: 3396-3410

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/frame.rs` around lines 3378 - 3379, The code currently sets
is_python_call via callable.downcast_ref::<PyFunction>().is_some(), which
misclassifies wrapper/delegating callables as non-Python and causes incorrect
duplicate monitoring events at the C-return/C-raise emission sites; change the
logic that computes is_python_call to first unwrap delegated Python callables
(detect known wrapper/proxy types and obtain their inner target, or use the
existing unwrap utility if present) and then test the unwrapped value with
downcast_ref::<PyFunction>() so the subsequent C-return/C-raise emission
branches (the code paths that emit the monitoring events) correctly treat
delegated Python functions as Python calls.
🧹 Nitpick comments (2)
crates/compiler-core/src/bytecode/instruction.rs (1)

435-513: Consider a single source of truth for opcode pair mappings.

Line 440 onward and Line 473 onward duplicate the same mapping pairs in opposite directions. This is easy to desync on future opcode additions. Consider defining pairs once (macro/table) and deriving both to_instrumented and to_base matches from it.

As per coding guidelines: "When branches differ only in a value but share common logic, extract the differing value first, then call the common logic once to avoid duplicate code."

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/compiler-core/src/bytecode/instruction.rs` around lines 435 - 513, The
two match lists in to_instrumented and to_base duplicate opcode pair mappings
and should be centralized; create a single source-of-truth mapping (e.g., a
static array, const slice, or macro) that lists tuples of (base_variant,
instrumented_variant) and then implement to_instrumented and to_base by looking
up that mapping (construct the base variants with Arg::marker() where needed and
match on variant discriminants or pattern keys). Update to_instrumented to
return the instrumented variant from the shared table and to_base to return the
base variant (using Arg::marker() for typed fields) so new opcode pairs are
added only once and both functions derive their behavior from the same mapping.
crates/vm/src/stdlib/sys/monitoring.rs (1)

230-230: Replace decorative section separators with plain comments.

Please remove // === ... === separators and keep short plain comments/doc-comments only.

As per coding guidelines "Do not add decorative section separators (e.g. // -----------, // ===, /* *** */). Use /// doc-comments or short // comments only when they add value".

Also applies to: 277-277

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/stdlib/sys/monitoring.rs` at line 230, Replace the decorative
section separator comment "// === Phase 1-3: De-instrument all layers (outermost
first) ===" (and the similar "// === ... ===" at the other occurrence) with a
short plain comment or doc-comment—e.g., change to "// Phase 1-3: De-instrument
all layers (outermost first)" or "/// Phase 1-3: De-instrument all layers"—so
remove the "===...===" decoration; update the comment in the surrounding
function/section in monitoring.rs where that text appears to follow the
project's short/plain comment guideline.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@crates/vm/src/builtins/frame.rs`:
- Around line 169-180: The loop that accumulates EXTENDED_ARG prefixes can leave
i == len if the bytecode ends with an ExtendedArg, causing the final access
instructions[i] to panic; update the logic around Instruction::ExtendedArg in
the frame analysis (the while handling of Instruction::ExtendedArg, variables
instructions, i, len, oparg, stacks, next_stack) to guard the trailing case by
checking i < len before doing the final oparg combine and accessing
instructions[i] (e.g., break out and handle the incomplete EXTENDED_ARG safely
or skip the final combination when i >= len) so no out-of-bounds indexing
occurs.
- Around line 181-183: mark_stacks() currently calls
opcode.to_base().unwrap_or(opcode) which leaves
InstrumentedLine/InstrumentedInstruction placeholders (stack effect 0,0) when
to_base() returns None; instead have mark_stacks consult the real opcode stored
in CoMonitoringData (from PyCode.monitoring_data) to compute correct stack
effects. Update the mark_stacks signature to accept a reference to the
monitoring data (or a &PyCode that exposes monitoring_data), and where you
currently call to_base()/unwrap_or(opcode) (and the similar spots noted around
lines 243-256 and 483-590) add logic to look up the underlying opcode in
monitoring_data and use that opcode’s stack effect when available; fall back to
opcode.to_base() only if monitoring data is absent. Ensure callers are updated
to pass the monitoring_data reference.

In `@crates/vm/src/stdlib/sys/monitoring.rs`:
- Around line 691-712: The snapshot captures state.disable but doesn't ensure
the DISABLE sentinel exists, so if it was None a callback returning
sys.monitoring.DISABLE won't be recognized; before taking the snapshot in the
block that builds (callbacks, disable_sentinel) (the code reading state.disable
and producing disable_sentinel) ensure state.disable is initialized (create and
store the sentinel PyObjectRef for sys.monitoring.DISABLE when None) then clone
it into disable_sentinel; apply the same initialization to the analogous
snapshot at lines 727-742 so both dispatch sites see a valid disable sentinel.

---

Duplicate comments:
In `@crates/vm/src/builtins/code.rs`:
- Around line 925-927: In co_branches(), when de-instrumenting opcodes
(currently using unit.op.to_base().unwrap_or(unit.op)), consult monitoring_data
to restore any displaced InstrumentedLine/InstrumentedInstruction back to their
original opcode first (e.g., look up the original opcode by the unit's
offset/index in monitoring_data and use it), then fall back to to_base() and
finally unit.op; apply the same restoration logic at the other locations that
compute branch targets and NOT_TAKEN fallthrough (the checks around the target
computation and fallthrough detection currently at the later blocks referenced
in the comment) so instrumented instructions map correctly to their original
opcodes.

In `@crates/vm/src/frame.rs`:
- Around line 3378-3379: The code currently sets is_python_call via
callable.downcast_ref::<PyFunction>().is_some(), which misclassifies
wrapper/delegating callables as non-Python and causes incorrect duplicate
monitoring events at the C-return/C-raise emission sites; change the logic that
computes is_python_call to first unwrap delegated Python callables (detect known
wrapper/proxy types and obtain their inner target, or use the existing unwrap
utility if present) and then test the unwrapped value with
downcast_ref::<PyFunction>() so the subsequent C-return/C-raise emission
branches (the code paths that emit the monitoring events) correctly treat
delegated Python functions as Python calls.

---

Nitpick comments:
In `@crates/compiler-core/src/bytecode/instruction.rs`:
- Around line 435-513: The two match lists in to_instrumented and to_base
duplicate opcode pair mappings and should be centralized; create a single
source-of-truth mapping (e.g., a static array, const slice, or macro) that lists
tuples of (base_variant, instrumented_variant) and then implement
to_instrumented and to_base by looking up that mapping (construct the base
variants with Arg::marker() where needed and match on variant discriminants or
pattern keys). Update to_instrumented to return the instrumented variant from
the shared table and to_base to return the base variant (using Arg::marker() for
typed fields) so new opcode pairs are added only once and both functions derive
their behavior from the same mapping.

In `@crates/vm/src/stdlib/sys/monitoring.rs`:
- Line 230: Replace the decorative section separator comment "// === Phase 1-3:
De-instrument all layers (outermost first) ===" (and the similar "// === ...
===" at the other occurrence) with a short plain comment or doc-comment—e.g.,
change to "// Phase 1-3: De-instrument all layers (outermost first)" or "///
Phase 1-3: De-instrument all layers"—so remove the "===...===" decoration;
update the comment in the surrounding function/section in monitoring.rs where
that text appears to follow the project's short/plain comment guideline.

ℹ️ Review info

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d0d1d1e and 233f0f4.

⛔ Files ignored due to path filters (1)
  • Lib/test/test_sys_settrace.py is excluded by !Lib/**
📒 Files selected for processing (5)
  • crates/compiler-core/src/bytecode/instruction.rs
  • crates/vm/src/builtins/code.rs
  • crates/vm/src/builtins/frame.rs
  • crates/vm/src/frame.rs
  • crates/vm/src/stdlib/sys/monitoring.rs

Comment on lines +169 to +180
// Accumulate EXTENDED_ARG prefixes
while matches!(opcode, Instruction::ExtendedArg) {
oparg = (oparg << 8) | u32::from(u8::from(instructions[i].arg));
i += 1;
if i >= len {
break;
}
stacks[i] = next_stack;
opcode = instructions[i].op;
}
oparg = (oparg << 8) | u32::from(u8::from(instructions[i].arg));

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Guard trailing EXTENDED_ARG to prevent out-of-bounds indexing.

If bytecode ends with ExtendedArg, i can become len, and Line 179 dereferences instructions[i]. That can panic during f_lineno stack analysis.

Proposed fix
                 while matches!(opcode, Instruction::ExtendedArg) {
                     oparg = (oparg << 8) | u32::from(u8::from(instructions[i].arg));
                     i += 1;
                     if i >= len {
-                        break;
+                        next_stack = OVERFLOWED;
+                        break;
                     }
                     stacks[i] = next_stack;
                     opcode = instructions[i].op;
                 }
+                if i >= len {
+                    break;
+                }
                 oparg = (oparg << 8) | u32::from(u8::from(instructions[i].arg));
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/builtins/frame.rs` around lines 169 - 180, The loop that
accumulates EXTENDED_ARG prefixes can leave i == len if the bytecode ends with
an ExtendedArg, causing the final access instructions[i] to panic; update the
logic around Instruction::ExtendedArg in the frame analysis (the while handling
of Instruction::ExtendedArg, variables instructions, i, len, oparg, stacks,
next_stack) to guard the trailing case by checking i < len before doing the
final oparg combine and accessing instructions[i] (e.g., break out and handle
the incomplete EXTENDED_ARG safely or skip the final combination when i >= len)
so no out-of-bounds indexing occurs.

Comment on lines +181 to +183
// De-instrument: get the underlying real instruction
let opcode = opcode.to_base().unwrap_or(opcode);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

# First, let's look at the file structure and the specific lines mentioned
wc -l crates/vm/src/builtins/frame.rs

Repository: RustPython/RustPython

Length of output: 100


🏁 Script executed:

# Now let's examine the code around lines 181-183
sed -n '175,190p' crates/vm/src/builtins/frame.rs

Repository: RustPython/RustPython

Length of output: 616


🏁 Script executed:

# Let's also look at lines 243-256
sed -n '237,260p' crates/vm/src/builtins/frame.rs

Repository: RustPython/RustPython

Length of output: 1447


🏁 Script executed:

# And lines 483-590
sed -n '475,595p' crates/vm/src/builtins/frame.rs

Repository: RustPython/RustPython

Length of output: 4420


🏁 Script executed:

# Find co_branches to understand the comparison
rg -n "fn co_branches" crates/vm/src/builtins/

Repository: RustPython/RustPython

Length of output: 177


🏁 Script executed:

# Let's look at the co_branches implementation
sed -n '917,980p' crates/vm/src/builtins/code.rs

Repository: RustPython/RustPython

Length of output: 2792


🏁 Script executed:

# Search for InstrumentedLine or InstrumentedInstruction definitions
rg -n "InstrumentedLine|InstrumentedInstruction" -t rust

Repository: RustPython/RustPython

Length of output: 1445


🏁 Script executed:

# Let's search for all uses of to_base() to understand the pattern
rg -n "\.to_base\(\)" -t rust -B 2 -A 2

Repository: RustPython/RustPython

Length of output: 7317


🏁 Script executed:

# Let's look at the to_base() implementation to understand what it does
sed -n '420,480p' crates/compiler-core/src/bytecode/instruction.rs

Repository: RustPython/RustPython

Length of output: 3078


🏁 Script executed:

# Let's look at the actual to_base method implementation
rg -n "pub fn to_base" -A 20 crates/compiler-core/src/bytecode/instruction.rs

Repository: RustPython/RustPython

Length of output: 1152


🏁 Script executed:

# Let's see the full mark_stacks function to understand the context
sed -n '155,210p' crates/vm/src/builtins/frame.rs

Repository: RustPython/RustPython

Length of output: 2303


🏁 Script executed:

# Let's look at the documentation comment more carefully and understand CoMonitoringData
rg -n "CoMonitoringData" -t rust -B 3 -A 3

Repository: RustPython/RustPython

Length of output: 3028


🏁 Script executed:

# Let's see how InstrumentedLine and InstrumentedInstruction are handled in the execution
sed -n '2820,2900p' crates/vm/src/frame.rs

Repository: RustPython/RustPython

Length of output: 3726


🏁 Script executed:

# Check if there's a way to get the real opcode from InstrumentedLine/InstrumentedInstruction in mark_stacks
rg -n "mark_stacks" -B 5 -A 30 crates/vm/src/builtins/frame.rs | head -80

Repository: RustPython/RustPython

Length of output: 3107


🏁 Script executed:

# Let's check what fields bytecode::CodeObject has
rg -n "pub struct CodeObject" crates/compiler-core/src/bytecode/code.rs -A 20

Repository: RustPython/RustPython

Length of output: 146


🏁 Script executed:

# Let's check if the PyCode has monitoring_data and if we can access it
rg -n "pub struct PyCode" crates/vm/src/builtins/code.rs -A 20

Repository: RustPython/RustPython

Length of output: 1357


🏁 Script executed:

# Let's see how mark_stacks is called to understand what data is available
rg -n "mark_stacks\(" crates/vm/src/builtins/frame.rs -B 2 -A 2

Repository: RustPython/RustPython

Length of output: 184


🏁 Script executed:

# Let's look at the full set_f_lineno function to see the context
sed -n '485,600p' crates/vm/src/builtins/frame.rs

Repository: RustPython/RustPython

Length of output: 4190


🏁 Script executed:

# Let's search for how bytecode is instrumented to see if InstrumentedLine/InstrumentedInstruction are actually used
rg -n "InstrumentedLine|InstrumentedInstruction" crates/vm/src/stdlib/sys/monitoring.rs | head -20

Repository: RustPython/RustPython

Length of output: 217


🏁 Script executed:

# Let's check if there are any comments or discussions about this issue
rg -n "mark_stacks" crates/ -B 5 -A 5 | grep -A 10 -B 10 "InstrumentedLine\|InstrumentedInstruction\|monitoring_data"

Repository: RustPython/RustPython

Length of output: 47


🏁 Script executed:

# Let's check what match cases exist in mark_stacks for instrumented opcodes
sed -n '194,350p' crates/vm/src/builtins/frame.rs | grep -A 2 "Instruction::" | head -40

Repository: RustPython/RustPython

Length of output: 1944


🏁 Script executed:

# Let's see the complete match statement structure in mark_stacks
sed -n '194,400p' crates/vm/src/builtins/frame.rs

Repository: RustPython/RustPython

Length of output: 10069


🏁 Script executed:

# Let's check if InstrumentedLine or InstrumentedInstruction are ever handled in mark_stacks
rg -n "InstrumentedLine|InstrumentedInstruction" crates/vm/src/builtins/frame.rs

Repository: RustPython/RustPython

Length of output: 47


🏁 Script executed:

# Let's verify if InstrumentedLine/InstrumentedInstruction fall into default match case
rg -n "fn stack_effect_info" crates/compiler-core/src/bytecode/instruction.rs -A 5

Repository: RustPython/RustPython

Length of output: 1306


🏁 Script executed:

# Check what happens with stack_effect_info on InstrumentedLine/InstrumentedInstruction
rg -n "InstrumentedLine\|InstrumentedInstruction" crates/compiler-core/src/bytecode/instruction.rs -B 2 -A 2

Repository: RustPython/RustPython

Length of output: 47


🏁 Script executed:

# Verify that mark_stacks doesn't have access to monitoring_data
sed -n '145,160p' crates/vm/src/builtins/frame.rs

Repository: RustPython/RustPython

Length of output: 681


🏁 Script executed:

# Verify how set_f_lineno uses mark_stacks and has access to PyCode
sed -n '520,530p' crates/vm/src/builtins/frame.rs

Repository: RustPython/RustPython

Length of output: 530


🏁 Script executed:

# Let's check what stack_effect InstrumentedLine and InstrumentedInstruction have
sed -n '650,665p' crates/compiler-core/src/bytecode/instruction.rs

Repository: RustPython/RustPython

Length of output: 894


🏁 Script executed:

# Let's see the full stack_effect_info implementation for both
rg -n "InstrumentedLine.*=>.*\(0, 0\)|InstrumentedInstruction.*=>.*\(0, 0\)" crates/compiler-core/src/bytecode/instruction.rs -B 2 -A 2

Repository: RustPython/RustPython

Length of output: 519


🏁 Script executed:

# Let's check the actual bytecode instructions to understand the problem better
sed -n '355,375p' crates/compiler-core/src/bytecode/instruction.rs

Repository: RustPython/RustPython

Length of output: 1074


🏁 Script executed:

# Let's understand the problem more clearly: check if mark_stacks has any way to access monitoring_data
sed -n '151,155p' crates/vm/src/builtins/frame.rs

Repository: RustPython/RustPython

Length of output: 313


🏁 Script executed:

# Now let's see how set_f_lineno calls mark_stacks and what it has access to
sed -n '515,530p' crates/vm/src/builtins/frame.rs

Repository: RustPython/RustPython

Length of output: 728


🏁 Script executed:

# Let's check if there are similar functions that might handle this correctly
rg -n "mark_stacks\|marklines" crates/vm/src/builtins/frame.rs

Repository: RustPython/RustPython

Length of output: 47


🏁 Script executed:

# Let's look at whether there are any TODOs or FIXMEs related to this issue
rg -n "TODO|FIXME|BUG" crates/vm/src/builtins/frame.rs | grep -i "instrum\|monitoring"

Repository: RustPython/RustPython

Length of output: 47


🏁 Script executed:

# Let's verify that InstrumentedLine/InstrumentedInstruction are actually inserted in real bytecode
rg -n "replace_op.*InstrumentedLine\|replace_op.*InstrumentedInstruction" crates/vm/src/stdlib/sys/monitoring.rs -B 5 -A 5

Repository: RustPython/RustPython

Length of output: 47


🏁 Script executed:

# Let's check if there's a version of mark_stacks that handles monitoring_data
rg -n "mark_stacks" crates/ --type rust -B 3 -A 3

Repository: RustPython/RustPython

Length of output: 1020


🏁 Script executed:

# Let's see if set_f_lineno has access to PyCode's monitoring_data
sed -n '480,530p' crates/vm/src/builtins/frame.rs

Repository: RustPython/RustPython

Length of output: 1897


🏁 Script executed:

# Check if monitoring is actually enabled in typical usage
rg -n "instrumentation_version\|monitoring_data" crates/vm/src/stdlib/sys/monitoring.rs | head -20

Repository: RustPython/RustPython

Length of output: 47


🏁 Script executed:

# Verify the issue exists by understanding when InstrumentedLine appears
sed -n '270,290p' crates/vm/src/stdlib/sys/monitoring.rs

Repository: RustPython/RustPython

Length of output: 685


mark_stacks() cannot access displaced opcodes for InstrumentedLine/InstrumentedInstruction and will compute incorrect stack states.

Lines 182 uses to_base() which returns None for these placeholder instructions. The unwrap_or(opcode) returns the InstrumentedLine/InstrumentedInstruction itself, which has stack effect (0, 0) in the default case. However, the real underlying opcode stored in CoMonitoringData may have a different stack effect. This causes incorrect stack-state propagation and leads to set_f_lineno potentially accepting or rejecting jumps incorrectly.

The issue is that mark_stacks() only receives &bytecode::CodeObject and has no access to PyCode.monitoring_data where the real opcodes are stored. Lines 243-256 and 483-590 have the same problem.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/builtins/frame.rs` around lines 181 - 183, mark_stacks()
currently calls opcode.to_base().unwrap_or(opcode) which leaves
InstrumentedLine/InstrumentedInstruction placeholders (stack effect 0,0) when
to_base() returns None; instead have mark_stacks consult the real opcode stored
in CoMonitoringData (from PyCode.monitoring_data) to compute correct stack
effects. Update the mark_stacks signature to accept a reference to the
monitoring data (or a &PyCode that exposes monitoring_data), and where you
currently call to_base()/unwrap_or(opcode) (and the similar spots noted around
lines 243-256 and 483-590) add logic to look up the underlying opcode in
monitoring_data and use that opcode’s stack effect when available; fall back to
opcode.to_base() only if monitoring data is absent. Ensure callers are updated
to pass the monitoring_data reference.

Comment on lines +691 to +712
let (callbacks, disable_sentinel): (Vec<(usize, PyObjectRef)>, Option<PyObjectRef>) = {
let state = vm.state.monitoring.lock();
let mut cbs = Vec::new();
for tool in 0..TOOL_LIMIT {
let global = state.global_events[tool];
let local = state
.local_events
.get(&(tool, code_id))
.copied()
.unwrap_or(0);
if ((global | local) & check_bit) == 0 {
continue;
}
if state.disabled.contains(&(code_id, offset as usize, tool)) {
continue;
}
if let Some(cb) = state.callbacks.get(&(tool, event_id)) {
cbs.push((tool, cb.clone()));
}
}
(cbs, state.disable.clone())
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Initialize DISABLE sentinel before dispatching callbacks.

If state.disable is None at snapshot time, a callback returning sys.monitoring.DISABLE in that first dispatch will not be recognized.

💡 Proposed fix
-    let (callbacks, disable_sentinel): (Vec<(usize, PyObjectRef)>, Option<PyObjectRef>) = {
-        let state = vm.state.monitoring.lock();
+    let (callbacks, disable_sentinel): (Vec<(usize, PyObjectRef)>, Option<PyObjectRef>) = {
+        let mut state = vm.state.monitoring.lock();
+        if state.disable.is_none() {
+            let d: PyObjectRef = sys_monitoring::MonitoringSentinel.into_ref(&vm.ctx).into();
+            state.disable = Some(d);
+        }
         let mut cbs = Vec::new();
         for tool in 0..TOOL_LIMIT {
             let global = state.global_events[tool];
@@
-        (cbs, state.disable.clone())
+        (cbs, state.disable.clone())
     };

Also applies to: 727-742

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/stdlib/sys/monitoring.rs` around lines 691 - 712, The snapshot
captures state.disable but doesn't ensure the DISABLE sentinel exists, so if it
was None a callback returning sys.monitoring.DISABLE won't be recognized; before
taking the snapshot in the block that builds (callbacks, disable_sentinel) (the
code reading state.disable and producing disable_sentinel) ensure state.disable
is initialized (create and store the sentinel PyObjectRef for
sys.monitoring.DISABLE when None) then clone it into disable_sentinel; apply the
same initialization to the analogous snapshot at lines 727-742 so both dispatch
sites see a valid disable sentinel.

@youknowone youknowone force-pushed the instrumented-instruction branch 3 times, most recently from 71ebc4a to 13a7676 Compare February 26, 2026 19:26
@youknowone youknowone force-pushed the instrumented-instruction branch from 13a7676 to 8c74023 Compare February 26, 2026 19:37
youknowone and others added 5 commits February 27, 2026 09:06
Add CoMonitoringData with line_opcodes and per_instruction_opcodes
side-tables. INSTRUMENTED_LINE and INSTRUMENTED_INSTRUCTION read
original opcodes from side-tables and re-dispatch after firing events.

- Add decode_exception_table() and CodeUnits::replace_op()
- Add Instruction::to_instrumented/to_base/is_instrumented mappings
- Three-phase instrument_code: de-instrument, re-instrument regular,
  then layer INSTRUCTION and LINE with side-table storage
- Mark exception handler targets as line starts in LINE placement
- InstrumentedLine resolves side-table chain atomically when wrapping
  InstrumentedInstruction
- InstrumentedForIter fires both BRANCH_LEFT and BRANCH_RIGHT
- Remove callback on DISABLE return for non-local events
- Implement set_f_lineno with stack analysis and deferred unwinding
- Add Frame::set_lasti() for trace callback line jumps
- Implement co_branches() on code objects
- Clear _cache_format in opcode.py (no inline caches)
- Fix getattro slot inheritance: preserve native slot from inherit_slots
- Fix BRANCH_RIGHT src_offset in InstrumentedPopJumpIf*
- Move lasti increment before line event for correct f_lineno
- Skip RESUME instruction from generating line events
- Defer stack pops via pending_stack_pops/pending_unwind_from_stack
  to avoid deadlock with state mutex
- Fix ForIter exhaust target in mark_stacks to skip END_FOR
- Prevent exception handler paths from overwriting normal-flow stacks
- Replace #[cfg(feature = "threading")] duplication with PyAtomic<T>
  from rustpython_common::atomic (Radium-based unified API)
- Remove expectedFailure from 31 now-passing jump tests
test_peepholer
test_bdb
@youknowone youknowone force-pushed the instrumented-instruction branch from 8c74023 to 4567c68 Compare February 27, 2026 00:07
@youknowone youknowone merged commit 684001f into RustPython:main Feb 27, 2026
24 of 25 checks passed
@youknowone youknowone deleted the instrumented-instruction branch February 27, 2026 04:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants