pub fn detect_quant_format(
config: &ModelConfig,
store: &WeightStore,
) -> Box<dyn QuantFormat>Expand description
Pick the right QuantFormat for a checkpoint.
Decision order:
- If
config.quantization_configis present, use its declaredquant_method/quant_algo— this is the authoritative signal every other inference stack (vLLM, TRT-LLM, SGLang) keys off. - Otherwise scan the weight store for the convention actually on
disk (legacy path, preserved for the many checkpoints in the
wild that ship without a
quantization_configblock). - If neither the config nor the store yields a recognized scheme,
return a
ModeloptFormatwith empty ignore list but emit atracing::warn!. This preserves existing behavior for the rarely-used pure-BF16 checkpoints while making the guess loud.