pub struct MoeExpertRoute {
pub a_table: DevicePtr,
pub b_table: DevicePtr,
pub scale_table: DevicePtr,
pub n_experts: u32,
pub k_in: u32,
pub n_out: u32,
pub max_rank: u32,
}Expand description
Per-EXPERT routing tables for the grouped down fold — the expert-keyed
analogue of the slot-keyed super::lora_delta::LoraRoute. Built once at
adapter install from the layer’s Down pairs; load-time-fixed device
addresses, so they are stable kernel args across capture/replay (adapter
identity for a mixed batch flows through the per-row moe_row_adapter, not
these tables).
n_experts is the TABLE LENGTH = max adapted expert id + 1 (NOT the
layer’s full num_experts): the grid launches grid.z = n_experts, and
every adapted expert has index < n_experts, so any higher-index expert is
unadapted and correctly folds nothing. expert_offsets[e] / [e+1] are read
for e < n_experts <= num_experts, always in range of the [num_experts+1]
prefix sum.
Fields§
§a_table: DevicePtr[n_experts] u64 device array of A_e addresses (0 = expert unadapted).
b_table: DevicePtr[n_experts] u64 device array of B_e addresses (0 = expert unadapted).
scale_table: DevicePtr[n_experts] f32 device array of per-expert scale_e (0.0 where unadapted).
n_experts: u32Table length = max adapted expert id + 1 (== grid.z).
k_in: u32Contraction dim of the shrink stage (moe_intermediate_size).
n_out: u32Output dim of the expand stage (hidden_size).
max_rank: u32Padded rank (contraction dim of the expand stage; row stride of B_e).
Trait Implementations§
Source§impl Clone for MoeExpertRoute
impl Clone for MoeExpertRoute
Source§fn clone(&self) -> MoeExpertRoute
fn clone(&self) -> MoeExpertRoute
1.0.0 · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read more