detect_quant_format

Function detect_quant_format 

Source
pub fn detect_quant_format(
    config: &ModelConfig,
    store: &WeightStore,
) -> Box<dyn QuantFormat>
Expand description

Pick the right QuantFormat for a checkpoint.

Decision order:

  1. If config.quantization_config is present, use its declared quant_method / quant_algo — this is the authoritative signal every other inference stack (vLLM, TRT-LLM, SGLang) keys off.
  2. Otherwise scan the weight store for the convention actually on disk (legacy path, preserved for the many checkpoints in the wild that ship without a quantization_config block).
  3. If neither the config nor the store yields a recognized scheme, return a ModeloptFormat with empty ignore list but emit a tracing::warn!. This preserves existing behavior for the rarely-used pure-BF16 checkpoints while making the guess loud.