pub fn sigmoid_gate_mul(
gpu: &dyn GpuBackend,
kernel: KernelHandle,
input: DevicePtr,
gate: DevicePtr,
output: DevicePtr,
num_elements: u32,
stream: u64,
) -> Result<()>Expand description
Element-wise sigmoid gate: output[i] = input[i] * sigmoid(gate[i]).
Used for gated attention in Qwen3: attn_output = attn_output * sigmoid(q_gate).
Kernel: sigmoid_gate_mul(input, gate, output, n)
Grid: (ceil(n/256), 1, 1) Block: (256, 1, 1)