pub fn router_dims(cfg: &ModelConfig) -> (usize, usize)Expand description
(out_dim, in_dim) of the base router (mlp.gate) projection: [num_experts, hidden]. A router LoRA perturbs the pre-selection routing logits.
pub fn router_dims(cfg: &ModelConfig) -> (usize, usize)(out_dim, in_dim) of the base router (mlp.gate) projection: [num_experts, hidden]. A router LoRA perturbs the pre-selection routing logits.