pub fn forward_dense(
gpu: &dyn GpuBackend,
k: &Glm5NextMlpKernels,
cfg: &Glm5NextMlpConfig,
w: &Glm5NextDenseMlpWeights,
inter: usize,
x: DevicePtr,
out: DevicePtr,
m: usize,
ws: &Glm5NextMlpWorkspace,
stream: u64,
) -> Result<()>Expand description
A BF16 SwiGLU MLP of width inter: down(clamped_swiglu(gate(x), up(x))).
Used for both the dense layers and the shared expert — identical math, different widths.
With tp_world_size > 1 the result is a partial sum; the caller reduces.