Expand description
DFlash drafter weight loader.
Loads z-lab/Qwen3.6-{27B,35B-A3B}-DFlash-style drafter checkpoints into
the typed DflashWeights structure consumed by
crate::layers::BlockDiffusionDraftHead. The drafter is a small
Qwen3-architecture transformer (8 layers, hidden=2048, GQA 32:4) with
these distinctive parts vs. a vanilla Qwen3:
model.fc—[len(target_layer_ids) * target_hidden, draft_hidden]BF16 projection that maps the stack of captured target hidden states into the drafter’s input space.model.hidden_norm— RMSNorm applied to the projected target context before mixing with token embeddings.lm_head— drafter ships its own (NOT tied to target’s), sotie_word_embeddings=false.- Optional
d2t— draft-vocab → target-vocab id remap (absent when drafter shares vocab with target, as in Qwen3.6-35B-A3B-DFlash where both = 248320). - Special
mask_token_id(248070for Qwen3.6-DFlash) used for the γ “to-be-predicted” positions in block diffusion.
Under TP the drafter is not sharded — it’s small (~1–2 GB BF16),
every rank loads the full set. Mirrors the existing MTP-under-TP pattern
(MTP loads ALL experts on every rank — no EP all_reduce needed).
Structs§
- Dflash
Config - Drafter HF
config.json(subset Atlas consumes). Mirrorsz-lab/Qwen3.6-35B-A3B-DFlash/config.jsonfield names verbatim soserde_json::from_strworks directly on the raw file. - Dflash
Layer Weights - Per-drafter-layer raw weights (BF16). Same shape across all 8 layers.
- Dflash
Rope Scaling - Subset of HF
rope_scalingblock consumed by Atlas. Mirrors the field names intransformers’ Qwen3 config soserde_json::from_strworks directly on the drafter’sconfig.json. - Dflash
SubConfig - Nested
dflash_configblock in the drafter’sconfig.json. - Dflash
Weights - Raw weight bundle for the DFlash drafter, post-load.
Functions§
- load_
dflash_ weights - Load DFlash drafter weights from a separate
WeightStorepointing at the drafter checkpoint. - parse_
dflash_ config - Parse a DFlash drafter’s
config.jsoninto aDflashConfig. Used bymain.rsafter fetching the drafter’s HF metadata to size the runtimeBlockDiffusionDraftHead(layer count, head_dim, vocab_size, thetarget_layer_idscapture indices). - store_
has_ dflash_ weights - Probe a
WeightStorefor the presence of DFlash drafter weights. Returns true if the store contains the uniquefc.weighttensor that DFlash drafters ship — a lightweight detection that doesn’t load any data. Both bare-key andmodel.-prefixed layouts are accepted; the canonicalz-lab/Qwen3.6-{27B,35B-A3B}-DFlashcheckpoints ship the bare layout (verified against commit 42d3b34, May 2026).