Poem AI
- 9 Devlogs
- 35 Total hours
An Ai that creates poetry based off the first word you give it, but without any external libraries like torch.
Ship #1
Finished the project! Better outputs, using torch, output streaming, and a readme!
Try it! -> https://k754a.hackclub.app/PoemAi
Ok, a decent amount of stuff! First, after the retraining of the model, I got it up and running on the server; that didn’t take too long.
What took the longest was switching from the AI slop of the chat to using a simpler torch approach. I’m going to be honest, AI did do a lot of the handholding for the inference, but it really helped me learn and did save me 15+ hours for sure, though I did write a large chunk of it, using learning mode, rather than the fast answer.
The next thing was output streaming. Output streaming made it feel so much faster and more efficient and less slow overall and helped a lot!
Finally, the readme. The readme took a bit, as I made a few graphs and some stats and stuff, but it’s all done!
Overall this project was a super fun and interesting project that helped me learn a LOT about AI and how they are trained, and I have mad respect for the people who figured this out from basically nothing!
Small changes to the website, better training data, efficiency, and performance!
Try it! -> https://k754a.hackclub.app/PoemAi
This took so freaking long, just because of all the random bugs, issues, and problems training an AI can have. But some of the biggest changes came from the dataset. Changing the dataset from a lot of junk to more Shakespeare ended up really improving output and structure (even though it doesn’t really rhyme).
The other things were mixing the data. Instead of training on the same text doc, I shuffle the data around, still in the chunks so as to not affect output performance, but because of shuffling each epoch, running longer training (like 400 epochs) created much better results.
Another update was because of these larger parameters and changes to the model; it created a problem where my loader could no longer load the model file, so I switched it to a bin and to store pure binary, allowing much faster loading and a memory drop from 3 GB to 1.1, running on the NEST server!
Overall, this has been a super fun project! There is still one thing I want to change: the main chat loader is basically all AI, and I kind of hate using it, so I’m going to attempt to code my own. I could see that taking a bit, but then I’m done!
EDIT: I’ve also been testing with different learning speeds, epochs, and context windows, and i ended up getting a even better model than the one currently, so i’m going to run the 400 epochs, overnight, and it should be updated by morning!
Web interface up, backend, and changes to CSS and HTML for all devices!
Try it out!! -> https://k754a.hackclub.app/PoemAi
Ok, this was not too hard; I updated a bunch of the CSS to make sure it worked for mobile and other aspect ratios (and stuff like that)!
Next was the harder part: I coded the backend and sent a packet requesting the poem and got a packet back from the server. This was probably the hardest part, but overall it was good!
I ended up changing a lot of the original AI chatbot to update it to take in a request and send it back out, but I did use a bit more AI to make it efficient, as it was using 2 GB of RAM on Nest lol, but that was probably the only AI part, and I really didn’t use AI for anything else, definitely keeping it under 30%.Overall it was good. I want to get a better model, as it’s alright, but it’s lowkey lacking, so yeah!
Faster sorting and work on the chat UI!
Over the last bit, I’ve worked a lot on building a basic chat UI. I’m going to be honest, even though I’ve built a browser engine, I’ve actually not done that much HTML coding (without tons of AI), so without using AI, I built just a basic chat UI that lets you send in your word and get the poem back (but it hasn’t been wired yet). This took me a ridiculously long time, as I’m just not very good at HTML, but everyone has to start somewhere!
The sorting was also another change; this allowed the AI to train so much faster, and I also changed a few compiler settings to have faster training times, but overall that was it.
I’m not a huge fan of all of it, so I want to put in some more work for the UI, get it running and hosted on NEST, then just improve the AI and be done! So yeah.
Finished training the BoW (Bag of Words) custom architecture model, with 0.7m and 3m parameter models.
It’s been a hot minute since I’ve worked on this project, but I decided I gotta finish this, ‘cause those hours won’t earn themselves!
Today, I fixed a few bugs with the training, but the main change was upping the dataset size from 96k chars to 600k and a 3m version as well. This proved to create much higher AI performance and took about 4 hours to train for both (as it’s trained on CPU, with one thread).
While it works, i used some quick ai just code the chat part, to test to make sure the models worked, and to fix any bugs, but i plan to write the chat connection in c++ (as the ai version is in python), then to create the web view, and hook it up to my nest site!
Overall a good day, and I should finish it up soon.
Migrated to C++, as well as made training faster and smarter!
Some of the main things I changed were to add epochs (going through the data more than once). Also, thanks to C++, I was able to improve the speed from about an hour and a half to under 2 seconds; however, with epochs, it’s about a minute, which isn’t too bad overall!
I’m almost done with this project; I’ll build the chat interface, then I’m planning to make a UI for it on Nest so that people can use it!
This project has really taught me a lot about ML and LLMs, and I’m starting to understand (and appreciate) how they are built!
There really isnt much to see still, but next time i should have a vid of it working fully!
Converted the tokenizer and data processing to C++!
Ok, this wasn’t as horrible as I thought it would be; however, it was just super long, as I had to search for similar functions and fix tons of errors, and overall it took a bit!
However, I’m in the perfect place for converting the training part, then testing it! So stay tuned for that. :)
The vid below shows just the basic loop running! (Not much to see, though)
Trained the first version!
Ok, so today, I was able to build a full tokenizer and train the first version!
However, it was super duper slow, but thanks to learning how to do this in Python (with very minimal libraries), I think I can code it in C++!
The AI took about 2 hours for about 100 neurons and 96k samples.
My plan is to make this train faster and smarter, as it has poetry structure but not the poetry part overall!
The video below shows the code, as the AI currently returns bad outputs!
Thanks for reading!
Started work on the AI!
Ok, taking a break from my other project, I wanted to program a simple AI without using libraries like PyTorch, as well as learn Python! The idea for my AI is you give it a word, and it creates poetry from that word (even if it doesn’t make much sense, I’m just trying to get it to rhyme)!
Today, I worked on learning some Python, as well as building the start of the tokenizer! So far it’s not too bad, and I should have it finished by the end of today!
Some challenges I’ve been running into are just learning Python in general, because I keep trying to do std::cout to print things, lol, but overall it’s going well!
Thanks for reading!
