Devlog 2

Built and validated a real native Vulkan RAYPAGE path on the RTX 5050. RT cores perform BVH traversal and page candidate pruning entirely on-device, with no CPU routing, readback, per-token BVH rebuild, or host synchronization.

What passed:

Native RT integration and live router execution.
BVH rejection: ~99.57%.
Full-model deterministic chat execution.
Fidelity: final-logit cosine ~0.9964 and transformed hidden cosine >0.998.
14B Qwen2.5-Q2_K model loads, chats, stays deterministic, and fits within 8 GB VRAM (~5.64 GB peak).

What failed:

End-to-end speed: native RT Qwen3 reached ~88 tok/s versus dense CUDA ~362 tok/s.
Software sparse was also slower (~115 tok/s), proving routing was not the main issue.
Precomputed perfect page IDs still achieved only ~106 tok/s, eliminating RT traversal as the bottleneck.
Indirect quantized MUL_MAT_ID was ~5.15 ms versus ~0.75 ms dense: ~6.9× slower.
Useful-page recall was ~98.69%, below the 99.5% gate.
14B page-major execution reached only ~0.5× dense.
Fine-grained arbitrary pages destroy contiguous reads, tensor-core tiling, warp coherence, and fused-kernel efficiency.

New work:

Implemented quantization-safe coarse 128-channel superblocks.
Built a 6-block trunk + 18-page Q8_0 model with contiguous page-major storage.
Added prefix slicing so active bundle counts can use ordinary contiguous MUL_MAT kernels without rebuilding artifacts.
Pulled CUTLASS and confirmed explicit SM120 NVFP4/BF16 GEMM examples exist.
Initial superblock active-1 fast path was ~68 tok/s, so it has not yet solved the problem.

Most promising directions:

RT selects 1–2 coarse bundles, then one fused contiguous kernel executes gate/up/SwiGLU/down.
Native block-scaled NVFP4/FP4 kernels, avoiding FP4→BF16 materialization.
Long-context KV-page pruning, where avoided attention work is much larger than FFN routing overhead.
Hybrid bundle templates: fewer fixed layouts, but optimized for real decode shapes.

Current conclusion: RT pruning works; arbitrary sparse FFN execution is the failure.