Function sigmoid_gate_mul_head_broadcast 

pub fn sigmoid_gate_mul_head_broadcast(
    gpu: &dyn GpuBackend,
    kernel: KernelHandle,
    input: DevicePtr,
    gate: DevicePtr,
    output: DevicePtr,
    nq: u32,
    hd: u32,
    num_tokens: u32,
    stream: u64,
) -> Result<()>

Per-head sigmoid gate multiply with broadcast over head_dim.

Step 3.7 attention gate: g_proj produces one BF16 scalar per head. This kernel applies output[t,h,d] = input[t,h,d] * sigmoid(gate[t,h]) where the sigmoid gate is broadcast across all hd dimensions of each head.

Kernel: sigmoid_gate_mul_head_broadcast(input, gate, output, nq, hd, total) Grid: (ceil(total/256), 1, 1) Block: (256, 1, 1)