Enum Nvfp4Variant 

pub enum Nvfp4Variant {
    Standard,
    CompressedTensors,
    Fp8Dequanted,
    Bf16Raw,
}

Weight quantization variant (on-disk format).

Variants

Standard

Standard ModelOpt: weight, weight_scale, weight_scale_2, input_scale. Attention/SSM projections are BF16 dense.

CompressedTensors

Sehyo/compressed-tensors: weight_packed, weight_global_scale, input_global_scale. Attention/SSM projections are NVFP4 quantized.

Fp8Dequanted

FP8 block-scaled (e.g. Qwen/Qwen3.5-35B-A3B-FP8, Qwen/Qwen3.6-35B-A3B-FP8): weight (float8_e4m3fn) + weight_scale_inv (BF16 per-[128,128]-block).

Loaded NATIVELY as FP8 in Qwen3 and Qwen3.5/3.6 model families. Attention uses w8a16_gemv (decode) + w8a16_gemm (prefill). MoE uses the FP8 fused grouped-GEMM batch1/2/3 path. SSM uses w8a16_gemv decode + fp8_gemm_n128 prefill (single-scale). No silent FP8→BF16→NVFP4 triple-conversion.

Historical note: the variant name retains “Dequanted” because the Bf16Raw cousin and the pre-2026-05-24 NVFP4 detour did dequant on load. The dispatch tables in qwen35/load_layers.rs (LayerType:: FullAttention if native_fp8) and qwen3.rs (line 176) now branch to native FP8 paths when quant_format == QuantFormat::Fp8.

Bf16Raw

Raw BF16/FP16 fine-tunes (e.g. samuelcardillo/Carnice-MoE-35B-A3B): only .weight tensors exist (no quantization metadata). Runtime-quantize from BF16 to NVFP4 at load time. Quality is suboptimal vs. a pre-calibrated NVFP4 release — the user gets a warning at startup.

Trait Implementations

impl Clone for Nvfp4Variant

fn clone(&self) -> Nvfp4Variant

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Nvfp4Variant

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for Nvfp4Variant

fn eq(&self, other: &Nvfp4Variant) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for Nvfp4Variant

impl Eq for Nvfp4Variant

impl StructuralPartialEq for Nvfp4Variant