shard_gdn_out_proj_row_parallel

Function shard_gdn_out_proj_row_parallel 

Source
pub fn shard_gdn_out_proj_row_parallel(
    src: DevicePtr,
    dims: &TpGdnDims,
    gpu: &dyn GpuBackend,
) -> Result<(DevicePtr, usize, usize)>
Expand description

Shard the out_proj BF16 weight [h, full_value_dim] row-parallel on its input dim (value_dim). Rank r keeps columns [r*local_value_dim, (r+1)*local_value_dim) of every output row; the partial products are summed with an all-reduce after the GEMM (mirrors attention o_proj). Returns (ptr, h, local_value_dim).