Devlog 07: Bounded 128MB Staging Ring & The 11.5x Recovery

Architected the fix for the 27B memory collapse: a bounded double-buffered staging ring in src/xdna-gemv-engine.cpp.

// Allocate exactly two 64 MB host-only staging buffers (128 MB total pinned memory)
static constexpr size_t STAGING_BO_SIZE = 64 * 1024 * 1024;
bo_staging[0] = std::make_unique<xrt::bo>(*device, STAGING_BO_SIZE, xrt::bo::flags::host_only, 0);
bo_staging[1] = std::make_unique<xrt::bo>(*device, STAGING_BO_SIZE, xrt::bo::flags::host_only, 0);

// Double-buffered stream from mmap page cache
size_t stage_idx = current_op % 2;
std::memcpy(bo_staging_map[stage_idx], mmap_weight_ptr, weight_bytes);
kernel_dpu->operator()(..., *bo_staging[stage_idx], ...);

• flags::host_only: Limits pinned kernel memory strictly to $2 \times 64\text{ MB} = \mathbf{128\text{ MB}}$.
• mmap_weight_ptr: Weights stay in standard page cache (MAP_SHARED), freely managed by Linux VM.

Result: pgmajfault and swap dropped to 0. 27B decode speed jumped from 0.15 tok/s to 1.72 tok/s—an 11.5× end-to-end recovery.