pub struct HcHeadWeights {
pub hc_fn: DevicePtr,
pub hc_base: DevicePtr,
pub hc_scale: DevicePtr,
pub lowrank: Option<HcLowRank>,
}Expand description
Both HC sites for a DeepSeek-V4 block: the attention site runs before/after attention, the FFN site before/after the MoE FFN. Model-level HC head parameters (final collapse before LM head). Loaded once, attached to every layer, but only used by the last layer.
Fields§
§hc_fn: DevicePtrMix projection head_fn: [hc_mult, hc_mult*hidden] f32.
hc_base: DevicePtrMix bias head_base: [hc_mult] f32.
hc_scale: DevicePtrMix scale: [1] f32.
lowrank: Option<HcLowRank>Qwen low-rank variant of the model-level mixer. Some => low-rank
kernels; its inject_w is NULL (use_combine=False).
Trait Implementations§
Source§impl Clone for HcHeadWeights
impl Clone for HcHeadWeights
Source§fn clone(&self) -> HcHeadWeights
fn clone(&self) -> HcHeadWeights
Returns a duplicate of the value. Read more
1.0.0 · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source. Read moreAuto Trait Implementations§
impl Freeze for HcHeadWeights
impl RefUnwindSafe for HcHeadWeights
impl Send for HcHeadWeights
impl Sync for HcHeadWeights
impl Unpin for HcHeadWeights
impl UnwindSafe for HcHeadWeights
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more