pub struct DenseFfnWeightsQ2 {
pub gate_proj: PackedQ2Weight,
pub up_proj: PackedQ2Weight,
pub down_proj: PackedQ2Weight,
}Expand description
Native keep-packed ternary Q2_0 dense MLP weights — loaded directly from a
PrismML Q2_0 GGUF (ATLAS_GGUF_NATIVE_Q2=1) with NO dequant / NVFP4 requant.
Each projection is a raw block_q2_0 buffer (2-bit codes + inline fp16 scale
per group). When installed via set_q2_weights, decode dispatches
q2_0_gemv (BF16 act × 2-bit weight, dequant-in-dot-product), mirroring the
FP8 path but with the weights ~4× smaller resident.
Fields§
§gate_proj: PackedQ2Weight§up_proj: PackedQ2Weight§down_proj: PackedQ2WeightAuto Trait Implementations§
impl Freeze for DenseFfnWeightsQ2
impl RefUnwindSafe for DenseFfnWeightsQ2
impl Send for DenseFfnWeightsQ2
impl Sync for DenseFfnWeightsQ2
impl Unpin for DenseFfnWeightsQ2
impl UnwindSafe for DenseFfnWeightsQ2
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more