What I’ve Done:

1. Fixed KV Cache Indexing in Graph Mode: The previous cudaErrorStreamCaptureUnsupported error and out-of-bounds access were caused by assigning python integers dynamically during capture, and failing to
   respect block sizing. I modified the KV cache’s _in_graph_mode behavior to compute the tensor-based slot offset dynamically ((slot % block_size).view(1)) directly inside the graph capture. This worked
   successfully and fully restored the model’s coherent output without breaking graph logic.
2. Experimented with MLP Block Sizing: I attempted to increase block_m and num_warps in the 1-bit Triton kernel (mxfp_lowbit_selected_swiglu) to reduce redundant memory loads of the intermediate x tensor.
   Unfortunately, register limits and memory layout meant it actually resulted in worse throughput, so I reverted those changes.
3. Experimented with Attention Loop Early Exit: The single_token_gqa_attention_kernel is statically unrolled up to token_bucket (256). I tried adding a dynamic while loop to break early for short context
   sizes, but Triton’s compiler handles static for loops much better.

reached around 65tk/s on gpt-oss-20b, constantly improvising MoE performance, next ima lock in for dense models but for now on MoE