decode_attention

Function decode_attention 

Source
pub fn decode_attention(
    gpu: &dyn GpuBackend,
    kernel: Glm5NextDsaDecodeKernel,
    cfg: &Glm5NextDsaConfig,
    geom: &DsaSelectGeometry,
    paging: &DsaDecodePaging,
    inputs: &DsaDecodeInputs,
    stream: u64,
) -> Result<()>
Expand description

Launch the selected-index decode. Enqueued on stream, not synchronised.

q_rows in geom must equal paging.num_seqs: this is the decode path, one query row per sequence. A mismatch would index the selection rows with the wrong stride, so it is refused rather than trusted.