You are browsing as a guest. Sign up (or log in) to start making projects!

19m 54s logged

ThinRuntime GPT-OSS-20B now achieves:

  • 50.3445 tok/s median across three explicit 200-token runs.
  • llama.cpp b9964 Q4_K_M: 40.3631 tok/s mean.
  • ThinRuntime is 24.73% faster.
  • Exact embedding and LM head; Q1/Q2 applies only to experts.
  • Zero expert transfers during decode.
  • Top-1 unchanged through step 4.
  • Cosine: 0.987376 step 1, 0.983299 step 4.
  • Accepted one-step configuration: 0.991415 cosine.
  • llama.cpp Q4_K_M source-relative cosine: 0.792350.
0
12

Comments 1

@Harman

This is a very interesting project and you explained it in very simple language