shard_fp8_block_scaled

Function shard_fp8_block_scaled 

Source
pub fn shard_fp8_block_scaled(
    src: &Fp8Weight,
    kind: TpShardKind,
    tp_rank: usize,
    tp_size: usize,
    block_size: usize,
    gpu: &dyn GpuBackend,
) -> Result<Fp8Weight>
Expand description

Shard an FP8 block-scaled weight. weight is [N, K] FP8 bytes; row_scale is [N/block_size, K/block_size] FP32 (widened at load). Both slice on the same axis at block granularity.