King of rock paper scissors

Ship #1 Pending review

@busy_cheesy on King of rock paper scissors · about 2 hours ago

This project is a real time rock, paper scissors ai predictor built entirely from scratch in python, displayed with a retro look made using pygame. Yes, this probably could’ve been a website to make it more accessible for more people, but I have absolutely no experience with js css or html, my experience is almost exclusively in python, so for the sake of goal of this project, which was to learn about how machine learning actually works, I made this project in python.

To truly understand how these algorithms work, I decided to build one myself from scratch instead of relying on prebuilt machine learning libraries. This includes the full neural network with backpropagation and dynamic hidden layer scaling. I started with just one input given to the algorithm: the last 8 player choices. There were also just two parameters the bot had: number of games it receives (8) and the learning rate (how much it should learn from each move). I first experimented with different prediction strategies, starting from the simple highest likelyhood that the ai predicted, then opting for weighted choices (ex: if the ai predicted 40% for rock, 10% for paper and 50% for scissors, there would be a 40% chance it chose rock, 10% it chose paper, and 50% it chose scissors. this was decided randomly, with those weights). I finally settled on the epsilon-greedy strategy, which is like the highest likelyhood strategy except there is a 10% chance it takes a random choice to keep the bot a little unpredictable. I then added momentum, which accumulates the updates made to the model, which lets it notice patterns much quicker. Finally I added feature engineering, that adds more inputs that the bot could have made from the game histroy, but instead I give it to the bot. This gives more information to the bot such as shoft transitions, outcome history, and move frequencies, which it doesnt have to deduce on its own, which helps the bot find patterns quicker and predict better.

The biggest challenge for me was tuning the bot by changing the parameters to find the best combination. I kept tweaking the values until it seemed like the bot was absolutely crushing me, but the next run it would be like I was playing against a toddler, an abysmal performace put on by the same values. To solve this problem, I ran a “championship” of bots, where i made 1400 combinations that then played 10 rounds of 100 games against the best bot I made so far. If they won more than 50%, they moved on. Around 140 bots made it to the second stage, a round-robin style tournament where each bot played the others in 20 rounds, each with 100 games. In total, there were about 13 000 000 games played. The winner was absolutely not who I was expecting: 2 game memory, 0.2 learning rate (pretty high) and 0.1 momentum. While the models with more game memomry and lower learning rate performed better in the q3 and q4, this proved that quick adaptation was much better suited for this project, and after testing that was certainly the case.

I am most proud of how the interface turned out. In my humble opinion, the retro arcade aesthetic was perfectly executed (no glaze) with the scannlines, the font, and the pixel art. The prediction bars showing how the model predicted your next move are also a nice touch, giving a peek into the algorithm’s mind. When you play, expect some trash talk from the bot. From experience, it gets better and better the more you play. It varies from person to person, as some people are more predictable than others. That’s about the whole project! Thanks for having read the whole thing (if you did)!

8 devlogs
17h

Try project → See source code →

Open comments for this post

@busy_cheesy on King of rock paper scissors · 2 days ago

2h 14m 18s logged

This will probably be my last devlog before the ship. I have overhauled the manager to include feature engineering, which should help the model make connections much faster. It now receives choice shifts, match outcomes and overall move frequencies instead of just the history of past rounds. Changed the algorithm to predict the user’s next move instead of the counter. This makes it easier to understand how it was thinking that you would act. It still returns the counter to the player move tho. Also changed back from the probability weighted selection method to the epsilon greedy strategy (highest probability move is chosen 90% of the time and 10% is random). The only thing left is to tweak all the values until I am satisfied with the result and then off it goes into the sea.

Open comments for this post

@busy_cheesy on King of rock paper scissors · 3 days ago

2h 21m logged

Completely overhauled the UI and I think it looks much better now. The bot has taunts now, but performance is marginal at best right now. Taking all the different strategies I’ve tried to make the bot better and consolidating them to find the best possible parameters for it will be the next and probably final problem the project will face before shipping. Might try the round robbin style face offs with different parametered bots facing each other. Overall it’s been a very interesting journey learning about all these concepts in machine learning.

Open comments for this post

@busy_cheesy on King of rock paper scissors · 4 days ago

1h 7m logged

So I did a lot of digging to find out how to improve the model and found some interesting stuff, but the main problem was that i was telling the model that the target move was a draw (immitating the player’s move) instead of beating them -_-. Also the idea of the round robbin to find the best bot parametersalso fell out because the models just trained to draw each other, but it took forever to execute so I don’t know if I should maybe retry it. There is still the very valid concern that the best bot would be the one that’s the best against other bots and not players so. On a more positive note, I introduced momentum. It keeps a running memory of the past updates, like a rolling bolder that builds speed. This enables it to recongnize patterns much quicker by stacking the past changes together. Caught onto patterns of length 1, 2, 3 and 4 in 1, 4, 12, and 17 moves, respectively. Below is a comparison (100 games of normal human rock paper scissors, not repeating a pattern, or atleast not conciously) vs random ouput (left) and the new bot (right). Glad to see it’s doing better. Will do more research on improving but polishing the game now is a better priority.

Open comments for this post

@busy_cheesy on King of rock paper scissors · 5 days ago

3h 1m logged

So clearly what I said earlier about the algorithm was wrong. I tested it some more and it seems like sometimes it is just reading my mind and other attemps it’s like it’s not even trying. I have fiddled around with the number of games the algorithm sees and it’s learning rate, and every time I think I got it I ask my brother (unbiased third party) to test it where it promptly fails. So, I have devised a plan to find the ultimate round robbin with about 200 different algorithms playing a total of 30 000 000 games of rock paper scissors to find the true champion. Hopefully this will let me find the best learning rate and sample size for the ai, therefore improving it’s game. might take a little while to execute, so we’ll see when that happens. also if there’s anyone who knows more about machine learning, please don’t hesitate to tell me what i could try. Thanks!

Open comments for this post

@busy_cheesy on King of rock paper scissors · 7 days ago

2h 52m logged

Got the basic UI working with a simple animation, just need to transfer the text from the terminal to the app instead. Still have to work on making the algorithm better, as spamming the same move 20 times in a row, the AI was still picking the wrong options sometimes (maybe trying to predict when i would switch it up?). Cool to see everything coming together

Open comments for this post

@busy_cheesy on King of rock paper scissors · 8 days ago

1h 19m logged

Very scary progress, I am barely winning now. All it took was some minor adjustements to the layer sizes, learning rate, and number of games stored, as well as making the output a weighted random choice based on the probabilities given by the algorithm, and changing the input for the bot from just the human inputs to also include the bot inputs. I bet there’s even more to make it better which is even more scary. Very nice to see the progress showing though!

Open comments for this post

@busy_cheesy on King of rock paper scissors · 9 days ago

2h 1m logged

Second deblog working on the machine learning algorithm. I got the backpropagation working after some time, but clearly I have to tweak some things, like the learning rate, game history size, number of neurons per layer, etc. The issue probably also lies beyond that, so more research will have to be done. The algorithm easily gets outsmarted. So for now, humans still reign as rock paper scissors champions!

Open comments for this post

@busy_cheesy on King of rock paper scissors · 10 days ago

2h 4m 1s logged

New project: a machine learning algorithm that plays rock paper scissors against you and learns in real time to predict your next move to beat you. Figured it was a pretty good way to get into machine learning as AI is on the rise, and coding my own will force me to truly understand how it works. I got the forward propagation working, and set everything up for back propagation for the next session. Can’t wait