Spent a few hours trying to get a DQN to learn the game and it never really worked.

Got the DQN model built and training against itself. Found a real bug early where the training math treated every move as if the same player went again next instead of the opponent, and fixed it.

After that it just wouldn’t budge. Tried a handful of fixes: better reward shaping, a much faster version of the game engine so it could run way more games at once, features that pointed out immediate wins, playing against older versions of itself with separate buffers so wins didn’t get drowned out by everyday moves. None of it stuck. The network kept settling back into picking almost the same move no matter what the board looked like.

Decided the algorithm itself was the problem, not the tuning, and switched to Monte Carlo tree search with a value and policy network instead, closer to how AlphaZero plays these kinds of games. Kept the DQN code in git history instead of deleting it.

I have to admit, I gave up on coding and debugging this manually so I used Claude Code… Next devlog soon with the actual changes.