Repository navigation
Fix MiniMax DSpark anchor predictions and enable native projection activations - #162
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
MiniMax DSpark inherited Qwen DFlash2's mask-only prediction convention: eight draft queries, discarding the anchor row. The pinned upstream MiniMax contract uses seven queries
[anchor, six masks]and predicts from rows 0–6. Every proposal was shifted, with an extra bidirectional attention query. Target rejection preserved final transcripts, so those checks alone missed the broken drafter.Separate draft query count/proposal offset from the eight-row target verification feed. Apply the corrected offset to proposals, confidence and candidate publication; retain predicted-token positions for sampled proposal RNG. Qwen's existing convention remains covered by shared regression tests.
DSpark now honors native activations for all MXFP8 projections, including context K/V. Reuse context and gate/up quantization, fuse SwiGLU/native quantization, and include scratch in the memory plan. BF16 control and checkpoint BF16 auxiliary matrices remain available. Withdraw the invalid DSpark benchmark comparison and document a tested adaptive schedule. The shipped sixteen-slot plain template is unchanged.
Validation
Executable source:
5a3a5b7c034897338edae8d0a1e8d80317658a56; SHA-256:bd23e4b27371d757a2e50555b561eeee2d8fbeb3a0cd5a1849ecf99e40ed5227. Subsequent changes are documentation only. Checkpoint:2f33a8b9ae49ce7b51c994df97166a7fd059bc2e.Short-context tuning diagnostics, one repetition of five classes, 256 tokens, eight configured slots; these are not replacement shipped-template campaign rows:
Ranges span classes, not confidence intervals. Adaptive beats plain in every measured class at C1 and C8. Adaptive settings: depths 1/3/5/7, row cost 5.6 ms, base 51.6 ms, adaptive lambda with the reservation floor. Costs were fitted from the corrected native C1 passes.
Cold prefill, measured separately with zero cached tokens: plain 0.632 / 2.103 / 7.629 s, adaptive 0.659 / 2.146 / 7.778 s, at approximately 512/2K/8K tokens. Prior plain-template campaign rows retain their original source attribution; invalid DSpark rows are withdrawn rather than replaced with these single-repeat diagnostics.
Fresh native API checks pass 9/9, including sampled requests. All eight video cases match fresh plain controls; video cache reuse, future-frame invalidation and streaming continuation pass. Eight completed/cancelled conversation replay comparisons are exact, with two completed-answer reuse checks. All five successful diagnostic boots ran the recorded executable with zero swap on every rank, matching operation streams, and no other world/build/benchmark on the fabric during measurements.
Production and head swap were restored once after validation. All four production processes use the original frozen executable with zero swap; HTTP health and a chat smoke request pass.
The initial reduced-depth driver invocation passed the concurrency list incorrectly and stopped before measurement; it was corrected and rerun. Its failed attempt supplies no timing data.
Documentation