Struct SafetensorsLoader 

pub struct SafetensorsLoader {
    pub ep_rank: usize,
    pub ep_world_size: usize,
    pub num_experts: usize,
    pub peak_memory_multiplier: Option<f64>,
    pub skip_activation_scales: bool,
    pub skip_mtp: bool,
}

Loads weights from safetensors files using mmap.

Fields

ep_rank: usize

EP rank (0-based). Only used when ep_world_size > 1.

ep_world_size: usize

EP world size. When > 1, remote expert tensors are skipped.

num_experts: usize

Total number of MoE experts in the model (for EP partitioning).

peak_memory_multiplier: Option<f64>

Override for the peak memory multiplier in the pre-flight OOM check. Set from QuantFormat::peak_memory_multiplier() in the caller. When None, the pre-flight uses its own heuristic (1.3x NVFP4 / 1.5x FP8).

skip_activation_scales: bool

Skip the W4A4 *.input_scale activation scales at load.

ModelOpt NVFP4 checkpoints ship one 0-dim F32 scalar per quantized projection. On a 512-expert model that is ~74k four-byte allocations, each taking a full allocation granule — GBs of padding for values Atlas never reads, because it serves w4a16 (BF16 activations) and the NVFP4 loader already treats the key as optional.

OPT-IN: step3p7 reads this key on its own path, so it must stay off unless the model’s loader is known not to need it.

skip_mtp: bool

Skip mtp.* tensors at load.

For models whose loader deliberately does not build an MTP head, uploading its weights is pure waste — on Qwen3.8-Flash-Next that is a 1.49 GB expert shard plus the MTP backbone, held resident while the KV cache goes without.

OPT-IN: a model that DOES build an MTP head must keep them, so this is set only where load_mtp_weights is known to return None.

Implementations

impl SafetensorsLoader

pub fn new() -> Self

Create a loader with no expert parallelism (loads all tensors).

pub fn with_ep(ep_rank: usize, ep_world_size: usize, num_experts: usize) -> Self

Create a loader with EP-aware filtering.

pub fn should_skip_tensor(&self, name: &str) -> bool

Check if a tensor should be skipped under EP. Skips *.experts.{E}.* tensors where E is not in local range. MTP head experts are never skipped (small, fully replicated).

🪤 The MTP exemption keys on a leading mtp. — a DeepSeek-style name. GLM-5.3 puts its MTP head at model.language_model.layers.45.* with no mtp. prefix, so that layer’s routed experts ARE sharded on GLM. Fine while the MTP head is out of scope; revisit before enabling it.

pub so residency can be PROVEN against a real checkpoint index without collectives (see spark-model/tests/glm53_ep_residency.rs).

Trait Implementations

impl Default for SafetensorsLoader

fn default() -> Self

Returns the “default value” for a type.

impl WeightLoader for SafetensorsLoader

fn load( &self, model_dir: &Path, gpu: &dyn GpuBackend, oom_reserve_bytes: usize, ) -> Result<WeightStore>