Expand description
Weight loader for Mistral Small 4 (MLA + MoE architecture).
GQA fallback: MLA LoRA projections are expanded to dense at load time
via GPU matmul. Q = wq_b @ wq_a, K/V split from wkv_b @ wkv_a[:kv_lora].
Loses MLA’s 12.8x KV cache compression but produces coherent output.