pub struct MixedForwardResult {
pub decode_logits: DevicePtr,
pub prefill_logits: DevicePtr,
}Expand description
Result of a mixed forward pass (decode + prefill in one pass).
Fields§
§decode_logits: DevicePtrLogits for decode sequences: [N, vocab_size] BF16. NULL if no decode sequences.
prefill_logits: DevicePtrLogits for the prefill sequence’s last token: [1, vocab_size] BF16.
NULL if is_last_chunk was false (intermediate chunk, no logits).
Auto Trait Implementations§
impl Freeze for MixedForwardResult
impl RefUnwindSafe for MixedForwardResult
impl Send for MixedForwardResult
impl Sync for MixedForwardResult
impl Unpin for MixedForwardResult
impl UnwindSafe for MixedForwardResult
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more