pub struct KdaWeights<'a> {
pub w_f_a: &'a [f32],
pub w_f_b: &'a [f32],
pub dt_bias: &'a [f32],
pub a_log: &'a [f32],
pub w_b: &'a [f32],
pub w_g_a: &'a [f32],
pub w_g_b: &'a [f32],
pub o_norm_w: &'a [f32],
pub w_o: &'a [f32],
}Expand description
Weights for one KDA layer, reference-only. Torch Linear layout [out, in] throughout.
Note what is absent relative to Atlas’s Qwen GDN: there is no Z tensor. The output gate is
g_a/g_b, and the decay source is f_a/f_b. Neither has a slot in Atlas’s fused
QKVZ + BA layout, so weight binding is not a rename map.
Fields§
§w_f_a: &'a [f32]§w_f_b: &'a [f32]§dt_bias: &'a [f32]§a_log: &'a [f32]§w_b: &'a [f32]§w_g_a: &'a [f32]§w_g_b: &'a [f32]§o_norm_w: &'a [f32]§w_o: &'a [f32]Auto Trait Implementations§
impl<'a> Freeze for KdaWeights<'a>
impl<'a> RefUnwindSafe for KdaWeights<'a>
impl<'a> Send for KdaWeights<'a>
impl<'a> Sync for KdaWeights<'a>
impl<'a> Unpin for KdaWeights<'a>
impl<'a> UnwindSafe for KdaWeights<'a>
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more