Enum MtpQuantization 

pub enum MtpQuantization {
    Nvfp4,
    Fp8,
    Bf16,
}

MTP head weight precision.

Variants

Nvfp4

NVFP4 E2M1 (0.5 bytes/weight) — fastest MTP forward, lowest accuracy.

Fp8

FP8 E4M3 (1 byte/weight) — balanced.

Bf16

BF16 (2 bytes/weight) — highest accuracy, slowest MTP forward.

Implementations

impl MtpQuantization

pub fn supports_drafter_prefill(self) -> bool

Whether the batched drafter prefill can run at this precision.

NECESSARY, not sufficient — prefill::prefill_drafter remains the authority and re-checks the actual weight variants and kernel handles. This predicate exists so the caller can skip the max_seq_len x hidden BF16 prompt-hidden buffer (335 MB at 32k/h=5120, 2.7 GB at 256k) for a head that could never use it. It is exact by construction: quantize_proj produces ProjectionWeight::Bf16 for, and only for, MtpQuantization::Bf16.

Trait Implementations

impl Clone for MtpQuantization

fn clone(&self) -> MtpQuantization

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for MtpQuantization

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl FromStr for MtpQuantization

type Err = Error

The associated error which can be returned from parsing.

fn from_str(s: &str) -> Result<Self>

Parses a string s to return a value of this type.

impl PartialEq for MtpQuantization

fn eq(&self, other: &MtpQuantization) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for MtpQuantization

impl Eq for MtpQuantization

impl StructuralPartialEq for MtpQuantization