DEVLOG 8: I think I may ship soon…

Ok so I’ve hit a road block. Stardance ends soon which means I don’t really have much time left. My project is a completely working reinforcement learning AI which can play Tetris but not well.

Originally I wanted to perfect a model and then ship the project but due to Stardance ending soon I may change that.

Recent additions

• Added the ability to resume training on a model you made at your most recent checkpoint
• Added setup automation scripts for windows and linux
• Refined README with a getting started section and more detail in the Model History section to give the project more story

Because I’m essentially prepped for shipping and it is a completely working project I will probably ship soon. Wish me luck.

The screenshots shown are the results from training Tetr.AI-V2

DEVLOG 7: more AI training (hint: FAILED)

This graph shows the results of my first training session of my AI showing how well the AI does over time through the reward. In total it lasted ~17 hours and its obviously not nearly enough time. The reward is supposed to go up over time, which it is. But not fast enough. Overall it only seems to increase in reward over time just by VERY little.

Moving onto the other graph which shows the rows cleared over time. As you can see it never really increases on average. This shows that:

1. MY AI SUCKSSSSSSS
2. My AI is dumb

Overall my AI can’t learn and I need to just keep on adjusting numbers over and over again.

DEVLOG 6: actual learning

Wow two devlogs in pretty much a day… I need to touch grass…

But that is not the point. IT IS FINALLY LEARNING (just very inefficiently) Currently it only learns from two things:

1. How many rows it cleared on a single run
2. Whether or not it sold the game

Now pretty much the rest of this project will just be tweaking and experimenting with things. Before I ship I do want to train and release a demo model that you can test locally on your PC ( and I may want to test if it would work on android phones too ) that will come with the project. I also want to make it to where you can create and train your own models and potentially share them with other people. CAN’T WAIT FOR EVERYTHING TO BE DONE!!!!!!

DEVLOG 5: Random movements

An update from my previous devlog where I started learning pytorch. I have not learned pytorch to the point to where I can explain everything down to its finest details but enough to make my “AI” do some random junk with no direction.

Pretty much I’m just initializing random weights and letting it go. I have 200 input neurons for my 20x10 board. The video I attached below shows the AI just doing some random stuff like rotating non stop and moving left and right.

DEVLOG 4: The beginning of learning PyTorch & Working AI-Tetris-Environment

Pretty much the title sums up what I’ve been doing for the past 3h and 9m of this. I have some prior knowledge of the basics of machine learning ( from The Nature Of Code by Daniel Shiffman awesome book about coding would recommend ) by making a single layer perceptron ( basically a single neuron algorithm ) that can separate linearly separable data. I attempted to make a multi layered perceptron which actually puts the network in neural network. But I got stuck on back propagation. I don’t really know what to put as the picture for this devlog so I’ll just show a visualization of a neural network.

DEVLOG 3: TETRIS ✅

THE TETRIS GAME IS PRETTY MUCH COMPLETE (besides some finishing touches) the game is completely playable with:

• Moving pieces with arrow keys
• Rotating pieces
• Deleting rows
• And the game restarts after there is no more valid places to go

So pretty much complete. Now it is time to dive into the deep waters that is…

NEURAL NETWORKS

Oh boy this is going to be fun (and hard). I already know that the kind of training that I have to do for this is reinforcement learning. Which basically is like giving dopamine to your artificial brain when it does something good and giving it a slap in the neck when it does something bad. Which in return it changes its weights based on its previous performance.

But for the first thing I got to do is set up all of the mumbo jumbo that will actually allow the neural network to see. And I think I have a rough understanding on how I’m going to implement that.

Stay tuned on this awesome journey of making an AI play Tetris!

DEVLOG 2

Ok it’s been a couple days but I’ve added key presses where:

-Left Arrow = Piece moves to the left
-Right Arrow = Piece moves to the right
-Down Arrow = Piece soft drops.
-Space Bar = Piece hard drops.

Overall it wasn’t that hard to implement but I had this bug where after soft dropping it saves that position to the locked pieces array. Which in return kind of broke the whole game. Here is the broken code block:

if event.type == pygame.KEYDOWN:
    if event.key == pygame.K_LEFT:
        self.currentPiece.x -= 1
        if not validSpace(self.currentPiece, self.grid):
            self.currentPiece.x += 1

            lockPiece(self.currentPiece, self.locked_pos)

The problem was the last line where it says

lockPiece(self.currentPiece, self.locked_pos)

So all I had to do was remove that and it works perfectly. Now I need to make the game rules where if you finish a row it deletes it and then the hardest part… ROTATION.

DEVLOG 1: Basic foundations

It took me way longer than it should have but I finally did it. I learned how to use PyGame and now I’m on my way to remaking Tetris. I set up a function that puts a random Tetris shape onto the 2D array that is essentially the one source of truth in Tetris at least that’s how I think of it. With PyGame I then update every frame what it should display based on that 2D array. The picture below shows me spinning up 3 instances of this code working. Currently the blocks don’t do much but I’ll start working on that tomorrow. Plus I really want to get the Tetris remake done quick so I can start building and training the neural network to actually play the game for me.