Expand description
Fixed-stride batched-decode metadata layout, derived from the serve
max_batch_size (SSOT — consumed by sizes.rs for the scratch envelope
and by spark-model’s upload_batch_metadata_fixed/_at for the
upload offsets, replacing the former hardcoded 0/128/256/512/768 gaps
that fit exactly 32 rows).
Region shapes (byte offsets, R = rows):
positions u32 [0, 4R)
seq_slot i32 [4R, 8R) (per-request LoRA routing)
slots i64 [8R, 16R)
seq_lens i32 [16R, 20R)
(gap [20R, 24R)) — legacy [640,768) pad, scaled
block_tbl i32 [24R, 24R + R·max_blocks·4)
At R = 32 this reproduces the legacy layout BYTE-FOR-BYTE
(0/128/256/512/768), so every boot with max_batch_size <= 32 is
byte-identical in addresses, strides and upload sizes.
Structs§
- Decode
Meta Layout - Derived fixed-stride decode-metadata layout.
Constants§
- DECODE_
META_ MAX_ ROWS - Layout ceiling, checked at serve time (
serve.rs). The metadata gaps themselves derive cleanly to any width; the binding constraints on this tip are downstream row consumers sized at 96+ rows: - DECODE_
META_ MIN_ ROWS - Layout floor: the legacy fixed layout was sized for exactly 32 rows;
deriving
rows = max(32, bs)keeps everybs <= 32boot byte-identical.