Enum LayerType 

pub enum LayerType {
    FullAttention,
    SlidingAttention,
    LinearAttention,
    Moe,
    SparseAttention,
}

Layer type in a hybrid transformer model.

Variants

FullAttention

SlidingAttention

LinearAttention

Moe

Standalone MoE FFN layer (Nemotron-H: no mixer, just expert routing + FFN).

SparseAttention

Sparse attention over a per-query selected subset of the KV cache (deepseek_sparse_attention): a full-rank mixer whose visible key set is chosen at runtime by an indexer, not fixed by a window.

Distinct from Self::FullAttention on purpose. Both attend over the whole cache in principle, but a sparse layer additionally needs indexer state, an indexer weight family, and a per-query top-k selection step — so scheduling, cache sizing and weight binding all have to be able to tell them apart. GLM-5.3 was previously flattened onto FullAttention at parse time, which round-tripped deepseek_sparse_attention into a lie.

Implementations

impl LayerType

pub fn is_attention(self) -> bool

Does this layer attend over a KV cache (as opposed to carrying recurrent state or being FFN-only)?

pub fn hf_name(self) -> &'static str

The string this layer type round-trips to in a HuggingFace layer_types array.

Trait Implementations

impl Clone for LayerType

fn clone(&self) -> LayerType

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LayerType

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for LayerType

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl PartialEq for LayerType

fn eq(&self, other: &LayerType) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for LayerType

impl Eq for LayerType

impl StructuralPartialEq for LayerType