Module quant_format 

Weight-quantization format abstraction.

Atlas must load quantized checkpoints produced by several toolchains, each of which serializes the same fundamental numeric format (e.g. NVFP4) with a different tensor-name convention. Historically we sniffed those names at load time via detect_nvfp4_variant in weight_map.rs, but community re-quants that advertise their scheme in quantization_config (HF standard) and keep some modules unquantized via an ignore list (lukealonso/MiniMax-M2.7-NVFP4) broke that heuristic: the detector saw MLP gates without .weight_scale and returned Bf16Raw, which then read uint8-packed FP4 as BF16 — a 4× byte overrun that surfaced as CUDA_ERROR_ILLEGAL_ADDRESS ten seconds into model construction (reported on Discord 2026-04-17 by energyburns, henryous).

The mitigation is to match vLLM / TensorRT-LLM / SGLang: prefer the quantization_config signal, fall back to tensor-name sniffing only when it is absent. This module formalizes that with a trait plus one implementation per supported serialization layout:

• CompressedTensorsFormat — Neural Magic llm-compressor (weight_packed + weight_global_scale + input_global_scale)
• ModeloptFormat — NVIDIA TensorRT ModelOpt (weight + weight_scale + weight_scale_2 + input_scale)
• Fp8BlockScaledFormat — FP8 E4M3 with weight_scale_inv

detect_quant_format is the single entry point. It inspects config.quantization_config first and only falls back to a heuristic on the weight store when the config is silent (emitting a warning, since a silent fallback is precisely what caused the original bug).

Structs

CompressedTensorsFormat

compressed-tensors NVFP4 checkpoint.

Fp8BlockScaledFormat

FP8 block-scaled checkpoint.

ModeloptFormat

ModelOpt-style NVFP4 checkpoint.

Traits

QuantFormat

A serialization layout for quantized weights.

Functions

detect_quant_format

Pick the right QuantFormat for a checkpoint.