Struct CompressorWeights 

pub struct CompressorWeights {
    pub wkv: DenseWeight,
    pub wgate: DenseWeight,
    pub norm: DenseWeight,
    pub ape: DevicePtr,
    pub ratio: usize,
    pub proj_dim: usize,
    pub is_csa: bool,
    pub pool: DevicePtr,
    pub pool_blocks: usize,
    pub ring: DevicePtr,
    pub prev_win: DevicePtr,
    pub stage: DevicePtr,
}

DeepSeek-V4 compressed-attention compressor weights (one per compressed layer). Produces n_win = usable/ratio compressed KV entries that are concatenated to the raw sliding-window KV before core attention. CSA (ratio 4) uses a 2×ratio overlap window (Ca/Cb); HCA (ratio 128) uses a single non-overlapping window.

Fields

wkv: DenseWeight

kv_proj: [proj_dim, hidden]. proj_dim = 2*head_dim (CSA) or head_dim (HCA).

wgate: DenseWeight

gate_proj: same shape as wkv.

norm: DenseWeight

kv_norm weight [head_dim] — HF-vanilla RMSNorm (loaded exactly).

ape: DevicePtr

position_bias / ape: [ratio, proj_dim] F32 (checkpoint-native; csa_compress indexes it as const float*), added to the gate before the per-dim softmax.

ratio: usize

compress_rate for this layer (4 = CSA, 128 = HCA).

proj_dim: usize

proj_dim of wkv/wgate output (2*head_dim for CSA, head_dim for HCA).

is_csa: bool

true = CSA (2×ratio overlap window); false = HCA (single window).

pool: DevicePtr

4b: persistent flat compressed-KV pool (decode reads it; inc-3 appends). Layout [pool_blocks × hd_mla] FP8-E4M3, each block = one rope’d comp_k entry quantized at the raw KV arm’s scale (k_scale=1.0 for V4) so decode reads raw+compressed at one dtype/scale (single online softmax). Flat per-seq (V4 serves max_batch=1), NOT paged — mirrors the reference Compressor.kv_cache contiguous buffer so block_idx = pos/ratio matches prefill’s index set exactly (no ring, no block-table remap). Prefill fills blocks [0, n_win); decode appends after.

pool_blocks: usize

Capacity in compressed blocks = max_position_embeddings.div_ceil(ratio).

ring: DevicePtr

4b inc-3: persistent decode-time normed-x ring [ratio × hidden] BF16. Each decode token’s compressor input (normed, the layer-input RMSNorm output — the SAME tensor prefill’s cache_skip_v4 feeds wkv/wgate) is written to slot pos % ratio. At a window boundary the ring holds the ratio tokens of the just-completed window in order, and decode reruns the prefill compress pipeline over it to append one pool block. BF16 (not FP8): quantize only at the pool write, so decode’s compressor input matches prefill’s bit-for-bit (fp8-ing the input would add a stage prefill never sees and make the golden-vector gate uninterpretable).

prev_win: DevicePtr

4b inc-3 (CSA only): previous completed window’s normed-x [ratio × hidden] BF16. CSA reads a 2×ratio overlap (prev window’s Ca + current window’s Cb); after each append the ring is copied here to feed the next window’s Ca. DevicePtr::NULL for HCA (no overlap). The first decode window has no valid prev (it would be a prefill window absent from the decode ring) → Ca masked.

stage: DevicePtr

4b inc-3 (CSA only): concat staging [2×ratio × hidden] BF16 = prev_win ‖ ring, the 2×ratio-token input the CSA compress kernel indexes for one overlapped window. DevicePtr::NULL for HCA.

Trait Implementations

impl Clone for CompressorWeights

fn clone(&self) -> CompressorWeights

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for CompressorWeights

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Copy for CompressorWeights