Crate spark_comm

Crate spark_comm 

Source
Expand description

Communication backend abstraction (SBIO IORouter for collective ops).

All distributed communication flows through CommBackend. Business logic never calls NCCL or MPI directly.

  • SingleGpuBackend — all ops are no-ops (single GPU).
  • NcclBackend — real multi-GPU via NCCL (expert parallelism; available with the nccl feature).

Re-exports§

pub use nccl_backend::NcclBackend;

Modules§

nccl
Raw NCCL FFI bindings (minimal surface for EP all-reduce + broadcast).
nccl_backend
NCCL-based communication backend for expert parallelism.

Structs§

SingleGpuBackend
Single-GPU backend: all collective ops are no-ops.

Traits§

CommBackend
Communication backend trait for distributed operations.