Struct AttnMetadataDev 

pub struct AttnMetadataDev {
    pub positions: DevicePtr,
    pub positions_h: DevicePtr,
    pub positions_w: DevicePtr,
    pub slot: DevicePtr,
    pub seq_len: DevicePtr,
    pub block_table: DevicePtr,
    pub max_blocks_per_seq: u32,
    pub num_seqs: u32,
    pub seq_slot: DevicePtr,
    pub moe_row_adapter: DevicePtr,
}

Pre-uploaded attention metadata device pointers.

Uploaded once per decode step in the model loop, reused across all 12 attention layers. Eliminates 44 redundant H2D copies per step.

For batched decode (num_seqs > 1), arrays are contiguous:

• positions: [N] u32
• slots: [N] i64
• seq_lens: [N] i32
• block_table: [N * max_blocks_per_seq] i32 (row-major)

Fields

positions: DevicePtr

Position values: [N] u32 at this device address. For multi-modal MRoPE this is the temporal (T) stream; callers set positions_h/positions_w to distinct buffers only when the token stream contains image or video patches.

positions_h: DevicePtr

Height (H) position stream for MRoPE-interleaved. When identical to positions (same pointer) the rope reduces to scalar RoPE. Default: same as positions.

positions_w: DevicePtr

Width (W) position stream for MRoPE-interleaved. Same fallback as positions_h.

slot: DevicePtr

Slot mappings: [N] i64 at this device address.

seq_len: DevicePtr

Sequence lengths (+1): [N] i32 at this device address.

block_table: DevicePtr

Block tables: [N * max_blocks_per_seq] i32 at this device address.

max_blocks_per_seq: u32

Number of blocks per sequence row in block_table.

num_seqs: u32

Number of sequences in this batch (1 for single-sequence decode).

seq_slot: DevicePtr

M2 per-request LoRA routing: [num_seqs] i32 at this device address, one adapter SLOT index per row (< 0 = base / no delta; pad rows are -1). Uploaded each decode step to a stable address (like positions / block_table), so the batched bgmv stays inside the captured decode graph. DevicePtr(0) on every non-routed path (single-seq decode, prefill, verify, MLA, MTP) — the bgmv apply sites no-op when it is null.

moe_row_adapter: DevicePtr

SOLID Incr-4 (batched decode MoE fold): [num_seqs] i32 per-row adapter map for the MoE expert gather-BGMV fold, at this device address. MoE semantics (distinct from seq_slot): < 0 = base / no fold (device kernel skips the row); >= 0 = fold the installed active adapter’s per-expert delta on that row. Built by crate::lora::build_moe_row_adapter_decode and uploaded each decode step to a stable address (a dedicated fixed-address buffer, TransformerModel::moe_row_adapter_buf, alloc’d once at init), so the batched fold stays inside the captured decode graph and is route-agnostic across replays (base rows no-op individually). DevicePtr(0) when no adapter is resident and on every non-batched path (the fold hooks then fall back to the request-granularity moe_route_gate). NOT the seq_slot buffer — that resolves -1 → active (attention defer-to-active), which would fold the adapter onto base rows here.

Trait Implementations

impl Clone for AttnMetadataDev

fn clone(&self) -> AttnMetadataDev

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for AttnMetadataDev