Struct Fp8Weight 

pub struct Fp8Weight {
    pub weight: DevicePtr,
    pub row_scale: DevicePtr,
    pub n: u32,
    pub k: u32,
    pub scale_format: WeightQuantFormat,
}

FP8 E4M3 checkpoint weight loaded directly from safetensors.

This struct carries an FP8 weight buffer along with its dequantization scale. The exact scale layout depends on the WeightQuantFormat tag in scale_format:

• WeightQuantFormat::Fp8PerRow — scale is [N] f32 per-row.
• WeightQuantFormat::Fp8BlockScaled — scale is [N/BS, K/BS] BF16 per-block (BS = 128 typically, the Qwen FP8 release convention).
• WeightQuantFormat::Fp8SingleScale — scale is the NULL DevicePtr; a single global scale is baked into the kernel that consumes this.

Always check scale_format before reading scale as a particular shape. Prior to the format tag (Phase 2c day-3 follow-up), the Fp8Weight struct silently mixed all three layouts in a single field, causing a cuMemcpyDtoDAsync_v2 INVALID_VALUE crash when the SSM build path tried to concat per-row F32 scales out of a buffer that actually held per-block BF16 scales (lower memory than expected).

Fields

weight: DevicePtr

[N, K] FP8 E4M3 weight bytes on GPU.

row_scale: DevicePtr

Dequantization scale pointer. Shape and dtype depend on scale_format — see struct docs.

n: u32

Output dimension (rows).

k: u32

Input dimension (columns).

scale_format: WeightQuantFormat

Tag for the row_scale buffer’s actual format. Asserted at kernel call sites via WeightQuantFormat::expect(...).

Implementations

impl Fp8Weight

pub fn transpose_for_gemm( &self, gpu: &dyn GpuBackend, transpose_k: KernelHandle, transpose_scale_k: KernelHandle, stream: u64, ) -> Result<Fp8WeightTransposed>

Transpose this FP8 weight for coalesced prefill GEMM. Allocates new GPU buffers for B_t[K,N] (FP8 bytes) and scale_t[K/128, N/128] (FP32; row_scale is already FP32).

Trait Implementations

impl Clone for Fp8Weight

fn clone(&self) -> Fp8Weight

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Fp8Weight

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl From<Fp8Weight> for QuantWeight

fn from(w: Fp8Weight) -> Self

Converts to this type from the input type.

impl Copy for Fp8Weight