Enum MetalKvDtype 

pub enum MetalKvDtype {
    Bf16,
    Turbo8,
    Turbo4,
    Turbo3,
    Turbo2,
    Bf16KTurbo4V,
    Bf16KTurbo3V,
    Bf16KTurbo2V,
}

KV cache storage format for the Metal contiguous cache.

Variants

Bf16

Raw bfloat, 2 bytes/elem.

Turbo8

FP8 E4M3 data + bf16 group-of-16 scales, WHT-rotated basis. 2.13× smaller than bf16.

Turbo4

4-bit Lloyd-Max codebook indices + FP8 group-of-16 scales (matched-norm L2), WHT-rotated basis. 3.56× smaller than bf16.

Turbo3

3-bit Lloyd-Max (8 values → 3 bytes) + FP8 group scales. 4.57× smaller than bf16.

Turbo2

2-bit Lloyd-Max (4 elems/byte) + FP8 group scales. 6.4× smaller than bf16.

Bf16KTurbo4V

Safer-asym: K raw bf16 (un-rotated), V Turbo4. Production- recommended frontier — K precision dominates retrieval quality.

Bf16KTurbo3V

Safer-asym: K raw bf16, V Turbo3.

Bf16KTurbo2V

Safer-asym: K raw bf16, V Turbo2.

Implementations

impl MetalKvDtype

pub fn k_is_rotated(self) -> bool

K side stored in the WHT-rotated basis (gates K rotation at append and the WHT(Q) decode bookend).

pub fn v_is_rotated(self) -> bool

V side stored in the WHT-rotated basis (gates V rotation at append and the iWHT(out) decode bookend).

Trait Implementations

impl Clone for MetalKvDtype

fn clone(&self) -> MetalKvDtype

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for MetalKvDtype

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl FromStr for MetalKvDtype

type Err = Error

The associated error which can be returned from parsing.

fn from_str(s: &str) -> Result<Self>

Parses a string s to return a value of this type.

impl PartialEq for MetalKvDtype

fn eq(&self, other: &MetalKvDtype) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for MetalKvDtype

impl Eq for MetalKvDtype

impl StructuralPartialEq for MetalKvDtype