Skip to content

Stabilize jacobian helper FX graphs#183756

Draft
jansel wants to merge 1 commit into
gh/jansel/639/basefrom
gh/jansel/639/head
Draft

Stabilize jacobian helper FX graphs#183756
jansel wants to merge 1 commit into
gh/jansel/639/basefrom
gh/jansel/639/head

Conversation

@jansel
Copy link
Copy Markdown
Contributor

@jansel jansel commented May 14, 2026

Stack from ghstack (oldest at bottom):

Compute jacobian diagonal offsets in Python and use functional split calls so Dynamo readable graphs avoid unstable CPU tensor bookkeeping and method/function spelling differences.

Fixes #180264

Generated by my agent

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @kadeng @chauhang @amjames @jataylo @azahed98

[ghstack-poisoned]
@pytorch-bot
Copy link
Copy Markdown

pytorch-bot Bot commented May 14, 2026

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/183756

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

❌ 1 Unclassified Failure

As of commit df04807 with merge base f513851 (image):

UNCLASSIFIED FAILURE - DrCI could not classify the following job because the workflow did not run on the merge base. The failure may be pre-existing on trunk or introduced by this PR:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@jansel
Copy link
Copy Markdown
Contributor Author

jansel commented May 14, 2026

@claude review these changes

@claude
Copy link
Copy Markdown

claude Bot commented May 14, 2026

Claude finished @jansel's task in 1m 32s —— View job


Review of PR #183756: Stabilize jacobian helper FX graphs

  • Gather context and read the diff
  • Review torch/_functorch/eager_transforms.py changes
  • Review test/dynamo/test_higher_order_ops.py changes
  • Post review feedback

Overall: LGTM — this is a clean, correct change that makes the jacobian FX graphs more stable for Dynamo.

Summary of changes

The PR does two things:

  1. Replaces CPU tensor bookkeeping with plain Python in _chunked_standard_basis_for_ (lines 900-904): The original code used torch.tensor(tensor_numels).cumsum(dim=0)[:-1].neg().unbind() to compute diagonal start indices, which produced tensor operations that appeared in Dynamo-traced graphs. The new code computes the same values via a simple Python loop, so these operations no longer pollute the FX graph.

  2. Uses torch.split (functional form) instead of Tensor.split (method form) in jacrev (line 781) and jacfwd (line 1409): This ensures the traced graph always spells the op as torch.functional.split(...) rather than .split(...), eliminating method vs. function spelling instability across traces.

Correctness verification

The diagonal offset computation is semantically equivalent:

  • Original: (0, *torch.tensor([a, b, c]).cumsum(0)[:-1].neg().unbind())(0, -a, -(a+b))
  • New: iterative subtraction → [0, -a, -(a+b)]

Both produce the same sequence of offsets.

Minor observations

  • The list() wrapping in torch.split(result, list(flat_output_numels), dim=0) is needed because flat_output_numels is a tuple (from tuple(out.numel() for out in flat_output) at line 662), and torch.split with a sequence argument expects a list for the split sections. This is fine.

  • In test_jacrev_two_tensors_argnums (around line 4107-4162), the assertion was restructured to use empty_line_normalizer on both sides to handle cross-platform empty line differences — a pragmatic fix for test stability.

  • The test changes are purely mechanical: removing the now-absent tensor/cumsum/getitem/neg/unbind nodes and updating the split spelling from .split((12,), dim=...) to torch.functional.split(..., [12], dim=...).

No issues found. The change is minimal, well-targeted, and directly addresses the graph instability reported in #180264.


Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant