REDUCE_ATTN_ENQ

Constant REDUCE_ATTN_ENQ 

Source
pub const REDUCE_ATTN_ENQ: usize = 15;
Expand description

Host-side ENQUEUE cost of the two collectives — the driver/NCCL calls only, no sync. Paired with REDUCE_ATTN/REDUCE_MLP, which then time ONLY the synchronize that follows, i.e. the device + network + rank-skew wait. Splitting them is the difference between “the fabric is slow” and “we call it 90 times a token”.