GLM5.2 on AMD CDNA3

Well.
That’s been a huge pain.

But I’s got a native unix shell
Should be amazing to not rely on this slugish p9

windows is like sweet poison
so nice
but so sloowwwww

bitlocker is scary guys

ubuntu is so nice – linux without the added pain of knowing how linux works (:

last 6 spills super nasty. LLVM be complex
should also hoist the loads before the stores (ugly stalls are swaits).
but look at that pretty no-sigpgmend-barrier work

I don’t hate LLVM anymore!!

So easy to just hook up to its headers and do what the hell I want with it (mostly in terms of what opt passes to run on the device code because I’m scared of that fat binary linking 😭)

LICM is being stupid? Kick that sucker off the top the stairs!!

possibly could have also done this with opt but C++ is way cooler and going to pay off once I really dig deep into this.

Ima get my 0 spills even if it’s going to cost me all my finals. probably already did 🥲

phhh that poor-people normal kernel… Hardware scheduled… Meh. No match for my persistent kernel!

not actually sure why I beat it, shouldn’t have. Stores are not acked on at the end of the kernel.
But we first smile, then run them back to back and earn that smile! Look at me utilizing smile debt like finance bro.

Maybe its my very cool scheduling algo. but I doubt it.

I know how I’ll do it!
We will unfraternally have to have kernel boundaries
but really just because the L1i cache is too small (and GPU architecture is too dumb)
But it’s fine for now.

Going to have the following kernels:
1:1 Residual + Norm
8:1 Residual + Norm (for the MoE summing)
1:1 Residual + Norm + MPT norm
MoE
Full DSA Fused attention (with or without indexer)
Head + Sampling +Embedding retrieval + normed MTP embedding
EH_Proj + MLP

And then!
For the HBM sync we need between so many things (mostly the attention kernel)
I’m going to extend HipKittens with a flagging mechanism that latches onto the stores, and add many more load methods that depend on it (like a stream of available blocks. the coolest thing is that I can have the flagging work at a certain granularity, and the load can infer if it’s OK based on a different one!

Don’t do kernels kids.

(90h) I hate LLVM so much! fun! Have you written manual ASM loops because the best OS compiler can’t handle a backedge? I did (:

And since AMD are the worst and I can’t get access to CDNA4, will be moving forward with MI300s.

All my ❤️ to RunPod and HotAisle.

And to whoever is hogging all the capacity from from cloudrift – may something bad happen to you, pig. (: