Added easy, medium and hard mode - each being UCBs but with different runtimes. Also updated the buttons as they were outdated code

UCB - Upper Confidence Bound

The UCB algorithm is an algorithm that I created to play connect 4.

How it works

The philosphy of UCB is doing random simulations of the game, and the move with the highest winrate in the random simulations is the best move.

The problem UCB solves is given a set amount of simulations - called rollouts, how do you allocate the amount of rollouts each move gets? You want to spend more time on promising moves, and less on unpromising moves, but you still need to try these unpromising moves because they may be good, but have low winrates just due to random chance. This is decided by the UCB rule:

UCBᵢ = avg(Xᵢ) + √( 2 ln(T) / nᵢ )

Which is the average winrate of the move i plus an optimism score. The more times you play a move, the chance that it is secretely a good move decreases. This ensures that you spend more time on good moves (high average) but also spend some times on worse moves that may turn out to be better by investing more rollouts on them.

Implementation

I implemented this into my connect 4 game I made, and allocated rollouts to 1000, so it makes 1000 random simulations to choose each move, and it kept on beating me. I had to reduce rollouts to 250 so I had a chance of winning.

I added a clear button so you can clear the board and try out you own creations.

Life Like Cellular Automata

Cellular Automata is a discrete grid based system which consists of cells that update themselves following a certain ruleset over discrete time chunks called generations. Conway’s game of life is the most famous one of these, but is only 1 subset of a larger set of life-like cellular automata.

In Conways game of life, each cell can either be one of 2 states, dead or alive. Its’s updating ruleset is:

• A node is ‘born’ if it has 3 neighbours, changing state from dead to alive
• A node survives if it is alive and has either 2 or 3 neighbours, else it reverts back to a dead state

This ruleset can be written in the form of B3/S23 and is just one of many other rules that can exist in life-like Cellular Automata.

In this update, I have added boxes that you can turn on or off to customise the cellular automa to any rule you want.
My next goals are to:

• Make the grid editable so you can create your own systems
• Make a rewind button so you can see how different structures are formed
• Create a slider to control simulation speed

Mainly, I just added checkboxes. The vision is to have these boxes control at what numbers nodes spawn and die.

Now that I had a drawable grid, i converted that grid to a pytorch tensor, that I plugged into my MNIST classifer that I made 2 devlogs ago, to create an AI number guesser! Honestly, while it does ocassionally slip up and make a few mistakes, especially confusing 7s and 2s and 6s and 0s, it is amazing!

I created a pygame grid that you can draw on, left click to add, right click to remove.

Using my knoeledge on classification I created a convolutional neural network classifier on the Mnist numbers dataset:

Added a Mouse scroll to make the zoom mechanic more intuitave - and its much less laggy than a slider. Programming a scroll wheel was both easier and harder than I expected, but I have now added a new tool to my pygame toolkit.

I Implemented the slider mechanic I was working on before in order to make it so that you can scale the simulation, as can be seen in this video.

I followed learn pytorch to make my first Convolutional Neural Network in pytorch! I have it sucessfully trained on the black and white mnist fashion dataset so it can identify fashion objects. Now I think I have enough foundations to build my final classification project.

I tried to make a classifier on skit-learns moon dataset that creates 2 classes of blobs in a cresent shape. Using my knowledge on how to create a binary classifer with ReLu and Binary cross entropy loss, I created a classifier that sucessful ‘solved’ the toy dataset. After completing this, I began to work on learnpytorch’s Computer vision module. My vision is to use my knowledge of computer vision and classification to build an interactive CNN visualiser.

I removed my old colour stuff and implemented conways game of line my changing the update code. I also made it screen size.

I made my first working classifier in pytorch with 0 training loss!

I am following learnpytorch.io’s code to create a simple binary classifier. It seems that I have a long way to go.

Finally finished my project. I expected this project to only take like 5 hours max, but it took more than double that! I have learnt a lot about pygame and pytorch in this project. My final additons were a databox that showed ML metrics like epochs, and added a formula to the line on the graph.

Made my program much smoother and added a stop button.

Finally, After heaps of normalisation and code and bugfixing, I finally, have a working linear regressor!!!

Created a graph wher eyou can click nodes, delete them and clear the screen