Enum LoraModule 

pub enum LoraModule {
    QProj,
    KProj,
    VProj,
    OProj,
    GateProj,
    UpProj,
    DownProj,
    OutProj,
}

Variants

QProj

KProj

VProj

OProj

GateProj

UpProj

DownProj

OutProj

GDN block OUTPUT projection (linear_attn.out_proj), value_dim -> hidden. Not OProj: different contract dim, disjoint layer set.

Implementations

impl LoraModule

pub const ALL: [LoraModule; 8]

pub fn is_dense_ffn(&self) -> bool

Dense SwiGLU FFN module — see Self::applies_to_layer.

pub fn is_gdn_out(&self) -> bool

Whether this module is the GDN block’s output projection.

pub fn applies_to_layer(&self, cfg: &ModelConfig, layer: usize) -> bool

Which layers may carry this module: attention q/k/v/o on full-attention layers only; dense gate/up/down on EVERY layer of a dense-FFN model (a hybrid’s linear-attention layers carry the SwiGLU FFN too); GDN out_proj on linear-attention layers only. THE authority for BOTH the pool layout and the packing walk, so reserved and written bytes cannot disagree.

pub fn peft_name(&self) -> &'static str

PEFT suffix name (target_modules vocabulary).

pub fn dims(&self, cfg: &ModelConfig) -> (usize, usize)

(out_dim, in_dim) of the base projection. Holo-3.1-0.8B (verified against the checkpoint header): k/v [512,1024], o [1024,2048], gate/up [3584,1024], down [1024,3584].

q_proj: on a gated-attention model (attn_gated) the raw projection emits the interleaved [Q|gate] at width 2·q_heads·head_dim (the FULL width the PEFT lora_B was trained against — verified [8192,16] on holo-3.1-35b); ungated q is q_heads·head_dim.

Trait Implementations

impl Clone for LoraModule

fn clone(&self) -> LoraModule

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LoraModule

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Ord for LoraModule

fn cmp(&self, other: &LoraModule) -> Ordering

This method returns an Ordering between self and other.

fn max(self, other: Self) -> Self

where Self: Sized,

Compares and returns the maximum of two values.

fn min(self, other: Self) -> Self

where Self: Sized,

Compares and returns the minimum of two values.

fn clamp(self, min: Self, max: Self) -> Self

where Self: Sized,

Restrict a value to a certain interval.

impl PartialEq for LoraModule

fn eq(&self, other: &LoraModule) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl PartialOrd for LoraModule

fn partial_cmp(&self, other: &LoraModule) -> Option<Ordering>

This method returns an ordering between self and other values if one exists.

fn lt(&self, other: &Rhs) -> bool

Tests less than (for self and other) and is used by the < operator.

fn le(&self, other: &Rhs) -> bool

Tests less than or equal to (for self and other) and is used by the <= operator.

fn gt(&self, other: &Rhs) -> bool

Tests greater than (for self and other) and is used by the > operator.

fn ge(&self, other: &Rhs) -> bool

Tests greater than or equal to (for self and other) and is used by the >= operator.

impl Copy for LoraModule

impl Eq for LoraModule

impl StructuralPartialEq for LoraModule