Finished training the BoW (Bag of Words) custom architecture model, with 0.7m and 3m parameter models.
‎
It’s been a hot minute since I’ve worked on this project, but I decided I gotta finish this, ‘cause those hours won’t earn themselves!
‎
Today, I fixed a few bugs with the training, but the main change was upping the dataset size from 96k chars to 600k and a 3m version as well. This proved to create much higher AI performance and took about 4 hours to train for both (as it’s trained on CPU, with one thread).
‎
While it works, i used some quick ai just code the chat part, to test to make sure the models worked, and to fix any bugs, but i plan to write the chat connection in c++ (as the ai version is in python), then to create the web view, and hook it up to my nest site!
‎
Overall a good day, and I should finish it up soon.