Rebuilt the whole training setup around Monte Carlo tree search instead of DQN, and it’s actually finding forced wins now.

Wrote a real spec and plan before touching code this time. Built the game engine again first, same rules as before, no changes needed, then added the value and policy network, then layered the tree search on top, all with tests along the way.

Writing the tests for the tree search caught a genuine sign bug right away. The code assumed the turn always switches after a move, but it doesn’t when a move wins the game, so the value was flipped for exactly the moves that mattered most. Also realized my own test scenario was wrong, it only completed a small board instead of the actual game. Fixed both, and the search now finds a forced win or a forced block on its own, even with a completely untrained network, since it’s real search finding the outcome, not a guess.

Got self-play, training, and evaluation working, ran a short validation loop and it actually worked, loss came down and it was already beating random play early on. Scaled it up for a real run.

Then the real run barely printed anything for ten minutes. Turned out it was running fine, just slow, because the tree search was calling the network one board at a time for every single simulation instead of batching them together, the exact same slowness from the DQN version, just in new code that never got the fix. Built batched search for self-play and evaluation both, checked it’s actually faster with a real timing test, and added logging that writes to a file and prints a lot more often so it’s obvious it’s alive.

Kicked off a real training run right after with the fix in, that’s going now.