Doubling numbers

Turns out you have to go back to square one to run a neural net on the GPU. For me that meant writing a program that multiplies every number in an array by 2, on the GPU.

Limited learning resources

Rust has a great library called wgpu for talking to the GPU. It’s cross-platform and can even run in WebAssembly if you do it right. The problem: most tutorials only cover rendering, and my neural net doesn’t care about triangles on a screen, it needs compute. I never found a tutorial on compute shaders from scratch, so I read the rendering material, stripped out what didn’t apply (windows, vertices, …), and added what did (workgroup size, how a compute pass works, …). Not trivial, but the official wgpu compute example got me on track. I mostly just copied it.

Why it’s hard

GPU programming isn’t straightforward. Here’s everything required just to multiply an array by 2:

1. Get a wgpu instance, adapter, device, and queue
2. Create a shader module
3. Create the input, output, and download buffers
4. Create the bind group layout and bind group
5. Create the pipeline layout
6. Create an encoder and a compute pass
7. Copy data CPU → GPU, run the shader, then copy data GPU → CPU

After 190 lines and some unexpected debugging (on code copied straight from an example, no less), I could finally double numbers fast, but only with huge arrays. Below a certain size, shuttling data to the GPU and back is slower than just doing the math on the CPU.

I don’t fully understand every step yet. Hopefully I won’t need to, or I’ll pick it up along the way.

Up next

Matrix multiplication is next, and most of the boilerplate above should carry over. Big questions remain about the final network, but taking it one step at a time should get me there eventually… or to a complete surrender, who knows.