You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Preserve Unicode in shared qualified type names - #9700
Preserve Python string contents throughout shared qualified type names and their consumers.
Return WTF-8 from the existing PyType::fully_qualified_name helper. Static types use their native slot name; heap types read stored module/qualname metadata without metaclass or string-subclass callbacks.
Keep lone surrogates, embedded NULs and empty names through all 53 helper invocations on 51 source lines, using the existing wtf8_concat! mechanism rather than duplicate formatters.
Carry that representation through numeric warnings, percent-format context, AST/type/iterator diagnostics and weakref/ZoneInfo type-name prefixes. Extend the existing warning helper to accept WTF-8 while preserving String callers.
This is 367 substantive changed lines across 23 production Rust files. No Python, Lib, test, marker or workflow files change.
Verification
Compared against pinned CPython 3.15 source d220409a13b77de7a6c30abe16ea90db01528edc, including Objects/typeobject.c, numeric, weakref and ZoneInfo consumers.
335 bounded CPython exact-message/representation assertions passed, plus 17 recorded metadata/warning edge probes. Independent review also checked three exact warning messages, all consumer paths and all 50 warning-helper callers.
Focused offline locked cargo check and ordinary all-targets cargo clippy for host-env, VM and stdlib with threading passed. Clippy reports exactly the four existing, unchanged-main diagnostics in posix/elementtree; none added or suppressed.
Independent source review, formatting, diff checks and normal commit hooks passed.
Native RustPython execution, full workspace/C-API tests and platform runtime suites were not run. Hosted CI remains the runtime validation gate; this draft does not claim a complete runtime suite pass.
Composition and limits
Compatible formatting companions are prepared for the additional consumers in #9693, #9697 and #9698. Preserve #9693's extracted hash-error helper, #9688's traceback arguments and #9690's removal of the old unraisable block when composing those changes. Older infrastructure overlaps were checked separately; this does not promise conflict-free merging of every historical refactor.
ZoneInfo's separately cached file-object repr still uses its existing String storage, and the native stderr fallback is not a new lossless OS-encoding implementation. These paths are not claimed fixed. The Rust helper's return type changes, so out-of-tree Rust callers may need to use WTF-8-aware formatting too.
AI assistance: OpenAI Codex performed source analysis, implementation, reference probing and independent review. Model version is not exposed. This contribution follows the maintainer-directed Python 3.15 upgrade work.
Summary by CodeRabbit
Bug Fixes
Improved handling of Python strings containing unusual Unicode characters in error messages, warnings, object representations, and diagnostic output.
Preserved fully qualified type names in messages, including names that cannot be represented as standard UTF-8.
Kept existing validation behavior and error conditions unchanged.
Reuse WTF-8 concatenation through native type metadata, diagnostics, warning context and representation consumers.
Assisted-by: OpenAI Codex:model version not exposed
No actionable comments were generated in the recent review.
Walkthrough
Selected RustPython error messages, warnings, representations, and unraisable output now use WTF-8 construction or buffers. Some paths retain non-UTF-8 names that previously required conversion to Rust String.
Changes: WTF-8 String Handling
Layer / File(s)
Summary
WTF-8 type names and representations crates/vm/src/builtins/type.rs, crates/vm/src/builtins/weakref.rs, crates/stdlib/src/_zoneinfo.rs
Fully qualified type names and weak-reference names retain WTF-8 data. ZoneInfo representations use WTF-8 values.
Warning and unraisable output crates/vm/src/stdlib/_warnings.rs, crates/vm/src/vm/mod.rs
The warning wrapper accepts messages convertible to Wtf8Buf. Unraisable output is assembled and written as WTF-8.
Formatting error messages and the message helper use WTF-8 buffers and concatenation.
VM builtin and protocol messages crates/vm/src/builtins/asyncgenerator.rs, crates/vm/src/builtins/{complex,descriptor,dict,function,genericalias,int,module}.rs, crates/vm/src/{coroutine.rs,frame.rs,protocol/{iter,mapping,number}.rs,types/slot.rs}
Selected builtin, coroutine, and protocol error or warning messages use WTF-8 concatenation.
Standard-library and AST messages crates/stdlib/src/{_asyncio.rs,socket.rs}, crates/vm/src/stdlib/_ast/python.rs
Selected asyncio, socket, and AST error or warning messages use WTF-8 concatenation.
The WTF-8 migration has no identified merge-blocking issue in the supplied evidence. Merge after normal CI checks pass.
Pre-merge checks: 4 · 1
Failed checks: 1 inconclusive
Check name
Status
Explanation
Resolution
Docstring Coverage
Docstring coverage is 16.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documenta…
Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed checks: 4 passed
Check name
Status
Explanation
Description Check
Check skipped - CodeRabbit’s high-level summary is enabled.
Title check
The title "Preserve Unicode in shared qualified type names" matches the main change. PyType::fully_qualified_name now returns WTF-8 (Wtf8Buf) instead of String, so qualified type names keep Python str…
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Full details: Title check
Explanation
The title "Preserve Unicode in shared qualified type names" matches the main change. PyType::fully_qualified_name now returns WTF-8 (Wtf8Buf) instead of String, so qualified type names keep Python string contents, including lone surrogates. The other changes route the new representation through error messages, warnings, and weakref/ZoneInfo prefixes. The title is concise and specific, and it does not describe anything absent from the changeset. The title says "Unicode" where WTF-8 is more precise, but this does not make it misleading.
Full details: Docstring Coverage
Explanation
Docstring coverage is 16.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 36 functions across 22 files. (1 skipped: 1 too large.)
Resolution
Write docstrings for the functions missing them to satisfy the coverage threshold.
Finishing Touches
Generate unit tests (beta)
Create a new PR
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Reviewing files that changed from the base of the PR and between 3485237 and 8fcc099.
Files selected for processing: 23
crates/stdlib/src/_asyncio.rs
crates/stdlib/src/_zoneinfo.rs
crates/stdlib/src/socket.rs
crates/vm/src/builtins/asyncgenerator.rs
crates/vm/src/builtins/complex.rs
crates/vm/src/builtins/descriptor.rs
crates/vm/src/builtins/dict.rs
crates/vm/src/builtins/function.rs
crates/vm/src/builtins/genericalias.rs
crates/vm/src/builtins/int.rs
crates/vm/src/builtins/module.rs
crates/vm/src/builtins/type.rs
crates/vm/src/builtins/weakref.rs
crates/vm/src/cformat.rs
crates/vm/src/coroutine.rs
crates/vm/src/frame.rs
crates/vm/src/protocol/iter.rs
crates/vm/src/protocol/mapping.rs
crates/vm/src/protocol/number.rs
crates/vm/src/stdlib/_ast/python.rs
crates/vm/src/stdlib/_warnings.rs
crates/vm/src/types/slot.rs
crates/vm/src/vm/mod.rs
Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 2 remain after this review.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Preserve Python string contents throughout shared qualified type names and their consumers.
PyType::fully_qualified_namehelper. Static types use their native slot name; heap types read stored module/qualname metadata without metaclass or string-subclass callbacks.wtf8_concat!mechanism rather than duplicate formatters.This is 367 substantive changed lines across 23 production Rust files. No Python, Lib, test, marker or workflow files change.
Verification
d220409a13b77de7a6c30abe16ea90db01528edc, includingObjects/typeobject.c, numeric, weakref and ZoneInfo consumers.cargo checkand ordinary all-targetscargo clippyfor host-env, VM and stdlib with threading passed. Clippy reports exactly the four existing, unchanged-main diagnostics in posix/elementtree; none added or suppressed.Native RustPython execution, full workspace/C-API tests and platform runtime suites were not run. Hosted CI remains the runtime validation gate; this draft does not claim a complete runtime suite pass.
Composition and limits
Compatible formatting companions are prepared for the additional consumers in #9693, #9697 and #9698. Preserve #9693's extracted hash-error helper, #9688's traceback arguments and #9690's removal of the old unraisable block when composing those changes. Older infrastructure overlaps were checked separately; this does not promise conflict-free merging of every historical refactor.
ZoneInfo's separately cached file-object repr still uses its existing String storage, and the native stderr fallback is not a new lossless OS-encoding implementation. These paths are not claimed fixed. The Rust helper's return type changes, so out-of-tree Rust callers may need to use WTF-8-aware formatting too.
AI assistance: OpenAI Codex performed source analysis, implementation, reference probing and independent review. Model version is not exposed. This contribution follows the maintainer-directed Python 3.15 upgrade work.
Summary by CodeRabbit