pub enum MoeLoraRoute {
Fold,
Skip,
Refuse,
}Expand description
Feature-1 MoE-LoRA fold decision for a single forward pass.
The MoE router/expert delta is a SINGLE globally-installed adapter (phase 1):
this gate makes the prefill fold per-request without a device kernel, exactly
mirroring the attention BGMV seq_slot < 0 skip but at request granularity.
A base request pays nothing; a packed/mixed batch refuses loudly rather than
fold one adapter onto every row (the device-side per-row fold that would let
a mixed batch skip base rows individually is the documented follow-up —
docs/design/lora-solid.md Incr 1/3).
Variants§
Fold
Single-request pass whose request owns the installed active MoE adapter:
FOLD. Genuine single-seq decode now folds the router + expert gate/up/down
deltas at this altitude (mirroring prefill); the remaining back-compat
paths (multi-seq / verify) still bail via reject_decode_lora before the
fold, so their Fold value is inert there. Also the default.
Skip
Single-request pass that is base (adapter_slot < 0, no adapter) or
routes to a different, non-installed adapter: SKIP the fold entirely.
Base tokens pay nothing (request-granularity mirror of the attention
BGMV seq_slot < 0 early-return).
Refuse
Multi-request / packed / codispatch batch whose per-row adapter identity cannot be honored without the device-side per-row fold (follow-up): the fold REFUSES loudly rather than mis-apply one adapter to every row.
Trait Implementations§
Source§impl Clone for MoeLoraRoute
impl Clone for MoeLoraRoute
Source§fn clone(&self) -> MoeLoraRoute
fn clone(&self) -> MoeLoraRoute
1.0.0 · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read moreSource§impl Debug for MoeLoraRoute
impl Debug for MoeLoraRoute
Source§impl Default for MoeLoraRoute
impl Default for MoeLoraRoute
Source§fn default() -> MoeLoraRoute
fn default() -> MoeLoraRoute
Source§impl PartialEq for MoeLoraRoute
impl PartialEq for MoeLoraRoute
impl Copy for MoeLoraRoute
impl Eq for MoeLoraRoute
impl StructuralPartialEq for MoeLoraRoute
Auto Trait Implementations§
impl Freeze for MoeLoraRoute
impl RefUnwindSafe for MoeLoraRoute
impl Send for MoeLoraRoute
impl Sync for MoeLoraRoute
impl Unpin for MoeLoraRoute
impl UnwindSafe for MoeLoraRoute
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
§fn equivalent(&self, key: &K) -> bool
fn equivalent(&self, key: &K) -> bool
key and return true if they are equal.