Struct NgramDims 

pub struct NgramDims {
    pub vocab_size: u64,
    pub ratio: u64,
    pub neighbor_num: usize,
    pub split_num: usize,
    pub eos_token_id: u32,
    pub hidden_size: usize,
}

The n-gram trio + derived dims, extracted from ModelConfig.

Fields

vocab_size: u64

ratio: u64

neighbor_num: usize

Largest n-gram size N (>= 2).

split_num: usize

Hash splits K per n-gram size.

eos_token_id: u32

hidden_size: usize

Implementations

impl NgramDims

pub fn from_config(c: &ModelConfig) -> Option<Self>

pub fn num_tables(&self) -> usize

pub fn table_dim(&self) -> usize

Per-table embedding dim = hidden / num_tables (validated at parse).

pub fn table_rows(&self, index: usize) -> u64

Row count of table index: ratio*vocab + 2*index + 1 — the consecutive odd offsets give the K tables of one n-gram size distinct (near-coprime) sizes so a collision in one split is independent of the others.

pub fn vocab_mods(&self, ngram: usize, split: usize) -> Vec<u64>

[V^1 mod T, ..., V^(i-1) mod T] for table (i, j).

Trait Implementations

impl Clone for NgramDims

fn clone(&self) -> NgramDims

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for NgramDims

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Copy for NgramDims