Super Tic Tac Toe

Added a scoreboard to the browser game today.

Now it tracks wins, losses, and draws between me and the AI, shown right above the board.

It saves to local storage too, so the score sticks around even if I close the tab or hit New Game.

Got the game playable in an actual browser tab today.

Moved the whole thing over from Python to a real webpage, board, AI opponent, all of it, so anyone can just open a link and play.

Had a hard time getting the trained model working in the browser, but got it working and double checked it gives the same results as the original.

Then spent a while making it actually feel good to play. The AI’s moves were popping in too suddenly, so I slowed that down and added a marker so you can see what it just played. Cleaned up the spacing and added a New Game button too.

Deployed it, thought it looked fine, then realized it was actually still showing an old broken version because of caching. Tracked that down and fixed it, also caught one more small thing where the board looked clickable during the AI’s turn when it shouldn’t have been.

You can try it at: https://raczjonathan12.github.io/super-tic-tac-toe/

Rebuilt the whole training setup around Monte Carlo tree search instead of DQN, and it’s actually finding forced wins now.

Wrote a real spec and plan before touching code this time. Built the game engine again first, same rules as before, no changes needed, then added the value and policy network, then layered the tree search on top, all with tests along the way.

Writing the tests for the tree search caught a genuine sign bug right away. The code assumed the turn always switches after a move, but it doesn’t when a move wins the game, so the value was flipped for exactly the moves that mattered most. Also realized my own test scenario was wrong, it only completed a small board instead of the actual game. Fixed both, and the search now finds a forced win or a forced block on its own, even with a completely untrained network, since it’s real search finding the outcome, not a guess.

Got self-play, training, and evaluation working, ran a short validation loop and it actually worked, loss came down and it was already beating random play early on. Scaled it up for a real run.

Then the real run barely printed anything for ten minutes. Turned out it was running fine, just slow, because the tree search was calling the network one board at a time for every single simulation instead of batching them together, the exact same slowness from the DQN version, just in new code that never got the fix. Built batched search for self-play and evaluation both, checked it’s actually faster with a real timing test, and added logging that writes to a file and prints a lot more often so it’s obvious it’s alive.

Kicked off a real training run right after with the fix in, that’s going now.

Spent a few hours trying to get a DQN to learn the game and it never really worked.

Got the DQN model built and training against itself. Found a real bug early where the training math treated every move as if the same player went again next instead of the opponent, and fixed it.

After that it just wouldn’t budge. Tried a handful of fixes: better reward shaping, a much faster version of the game engine so it could run way more games at once, features that pointed out immediate wins, playing against older versions of itself with separate buffers so wins didn’t get drowned out by everyday moves. None of it stuck. The network kept settling back into picking almost the same move no matter what the board looked like.

Decided the algorithm itself was the problem, not the tuning, and switched to Monte Carlo tree search with a value and policy network instead, closer to how AlphaZero plays these kinds of games. Kept the DQN code in git history instead of deleting it.

I have to admit, I gave up on coding and debugging this manually so I used Claude Code… Next devlog soon with the actual changes.

Started the super tic-tac-toe project.
Wrote the first version of the game engine in train.py. There’s a Game class that tracks the 9x9 board, whose turn it is, and the status of each of the 9 sub-boards.
It handles the annoying part of this game: winning a small board, checking if that wins the whole meta-board, and figuring out which sub-board the other player is forced to play in next (or opening up the whole board if that one’s already finished).Also pulled in TensorFlow/Keras imports up top since the plan is to train a DQN on it once the game logic is solid.
Nothing runs as an actual game yet, just the class and its rules.