Struct VisionEncoder 

pub struct VisionEncoder {

    pub patch_embed_w: DevicePtr,
    pub patch_embed_b: DevicePtr,
    pub pos_embed: DevicePtr,
    pub blocks: Vec<ViTBlock>,
    pub deepstack: Vec<MergerLayer>,
    pub deepstack_indexes: Vec<usize>,
    pub merger: MergerLayer,
    pub hidden_size: usize,
    pub num_heads: usize,
    pub head_dim: usize,
    pub spatial_merge_size: usize,
    pub out_hidden_size: usize,
    pub intermediate_size: usize,
    pub p_max: usize,
    pub num_grid_per_side: usize,
    /* private fields */
}

Fields

patch_embed_w: DevicePtr

patch_embed_b: DevicePtr

pos_embed: DevicePtr

blocks: Vec<ViTBlock>

deepstack: Vec<MergerLayer>

deepstack_indexes: Vec<usize>

merger: MergerLayer

hidden_size: usize

num_heads: usize

head_dim: usize

spatial_merge_size: usize

out_hidden_size: usize

intermediate_size: usize

p_max: usize

num_grid_per_side: usize

Implementations

impl VisionEncoder

pub fn forward( &self, pixels: &[f32], grid_h: usize, grid_w: usize, gpu: &dyn GpuBackend, stream: u64, ) -> Result<usize>

Single-image forward (back-compat shim). For N=1 this issues the SAME kernels with the SAME args in the SAME order as the old per-image path → byte-identical output. Returns total_rows = (1+n_deepstack)*merged_p (the OLD return value; only ever tested > 0 downstream).

pub fn forward_batched( &self, images: &[(&[f32], usize, usize)], gpu: &dyn GpuBackend, stream: u64, ) -> Result<Vec<(usize, usize, usize)>>

Batched forward over N images. M-agnostic ops (patch_embed + all 27 blocks’ GEMMs/norms/gelu/residuals) run ONCE over M=Σpᵢ; per-image- geometry stages (host pos/rope prep, attention, mergers) loop per image.

buf_out layout (rows of out_hidden_size BF16): [0 .. Σmerged_p) = final merger, IMAGE-ORDER packed ← splicer reads here [(k+1)*Σmerged_p .. ) = deepstack-k, image-order packed ← LLM-unused

IN-BOUNDS INVARIANT: the deepstack high-water row is 4·Σmerged_p = Σp ≤ p_max (all four mergers emit exactly merged_p rows each), and buf_out is p_max rows → no realloc, no overrun.

Returns per-image (post_h, post_w, merged_p) in image order.

impl VisionEncoder

pub fn new( patch_embed_w: DevicePtr, patch_embed_b: DevicePtr, pos_embed: DevicePtr, num_position_embeddings: usize, blocks: Vec<ViTBlock>, deepstack: Vec<MergerLayer>, deepstack_indexes: Vec<usize>, merger: MergerLayer, hidden_size: usize, num_heads: usize, spatial_merge_size: usize, out_hidden_size: usize, intermediate_size: usize, patch_size: usize, max_pixels: Option<usize>, gpu: &dyn GpuBackend, ) -> Result<Self>