pub struct OverlayKernels {
pub rowdiff: KernelHandle,
pub embed_overlay: KernelHandle,
pub lmhead_overlay_bf16: KernelHandle,
pub lmhead_overlay_f32: KernelHandle,
}Expand description
The four token-overlay kernels, resolved once at model construction via
try_kernel (null-on-miss ⇒ the feature is silently unused rather than a
hard init failure on a kernel image that predates the overlay).
Fields§
§rowdiff: KernelHandle§embed_overlay: KernelHandle§lmhead_overlay_bf16: KernelHandle§lmhead_overlay_f32: KernelHandleImplementations§
Source§impl OverlayKernels
impl OverlayKernels
pub fn new(gpu: &dyn GpuBackend) -> Self
Trait Implementations§
Source§impl Clone for OverlayKernels
impl Clone for OverlayKernels
Source§fn clone(&self) -> OverlayKernels
fn clone(&self) -> OverlayKernels
Returns a duplicate of the value. Read more
1.0.0 · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source. Read moreSource§impl Default for OverlayKernels
impl Default for OverlayKernels
impl Copy for OverlayKernels
Auto Trait Implementations§
impl Freeze for OverlayKernels
impl RefUnwindSafe for OverlayKernels
impl Send for OverlayKernels
impl Sync for OverlayKernels
impl Unpin for OverlayKernels
impl UnwindSafe for OverlayKernels
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more