Devlog 06 The 27B Disaster & 12.6 GB Unevictable BO Crisis

Tried loading Qwen3.8-27B-Q4_0 (15.2 GB). Everything collapsed: token speed plummeted to 0.15 tok/s (6.67 seconds/token).

Ran /proc/meminfo and /proc/vmstat diffs during decode to find the bottleneck:

/proc/meminfo during 27B crash:
  MemTotal:    31,842 MB
  Unevictable: 12,640 MB  <-- 369 persistent XRT BOs locked in RAM!
  pgmajfault:  520,118    <-- 520k disk page reads PER TOKEN
  pswpout:      18,400    <-- active swap storm

• The bug: Allocating persistent xrt::bo objects for all 369 model tensors pinned 12.6 GB into Unevictable kernel memory.
• The result: On a 32 GB laptop with OS and KV cache, Linux was starved of clean page cache. Every layer forced synchronous swap-outs and major page faults to disk.

Persistent BO allocation for large models is fundamentally broken on consumer RAM.