@kamix

The bot now has a brain

I just made a commit implementing the neural network, that will be used to guide the MCTS I created a few days ago.

The role of the NN in AlphaZero is really huge - it takes in the current position and predicts the probability for each move to be played, and how good or bad the position is for the current player.

As you can see from the screenshot, the NN is implemented in Python (PyTorch), but the MCTS is made in C++. You can probably guess it took quite a bit of time to get those two working together, but I think I got it under control.

Next up is the training loop for the NN

The search engine for HexZero is complete!

Over the last few days, I’ve been implementing Monte Carlo Tree Search, which is the algorithm AlphaZero uses to explore possible moves.

Since HeXO has a two-move turns, I had to make some modifications from the original approach, but I think I finally got it working!

With the C++ MCTS implementation complete, the next goal for the project is to make the neural network, that will guide the MCTS on which moves are worth exploring in the first place.

Made some pretty good progress working on the training environment for my AlphaZero for HeXO today

For those curious:

AlphaZero is an ML program by DeepMind, that was able to master complex games like chess, go and shogi with no human input

HeXO is a hexagonal tic-tac-toe variant, with a “connect 6” win condition