Devlog #7
Hello guys! This is probably my last devlog before releasing this project, and I want to share what happened during the last 3 days.
Also, keep in mind that I previously had a version based on a Q&A-type database that I had generated using an LLM.
First off, the version from the last devlog was based on a database from DailyDialog. It was very heavy for the model’s “brain”, and as a result, it wasn’t really able to communicate properly. It would sometimes respond with something completely unrelated to the question.
After that, I tried training it on a Wikipedia dump. As an idea, it was good, but the amount of time it took to finish a training session with 30 epochs was LONG. And even after that, the AI still wasn’t capable of having proper conversations.
After that, I asked ChatGPT for help, and I can say that ChatGPT is not only bad, it is horrible for debugging. It makes stuff up, changes things that aren’t even broken, and generally makes the debugging process harder.
So I tried using Claude. Claude at least tried. It actually computed the project on its server, fixed something, but it still wasn’t enough.
Then I discovered Codex. It’s not entirely new, but I didn’t know about it before!
I gave Codex the project files and asked it to search for the problem. It gave me the files back, and it actually fixed the problems.
When I tried the fixed version, I asked it some questions. It had been trained on both a Q&A database and a Wikipedia database, and it actually answered correctly most of the time.
The problem was that when I asked something that wasn’t in the Q&A database, it retrieved information from the Wikipedia dump. That wasn’t necessarily bad, but it didn’t know how to actually make a sentence using the information. It was basically just a retrieval tool rather than a conversational AI.
Since my Codex tokens expired and I can’t use it for a month, I didn’t bother continuing with that version.
I went back to the version from before the last devlog, which was based on a smaller Q&A database. I improved the database, retrained the model, and now it actually responds.
The problem is that if you’re not using proper grammar, it can still respond with something unrelated to the topic. And if the topic isn’t included in the database, it can make stuff up.
After that, I added an Ollama version alongside my trained model. This way, if the user wants an answer that my trained AI can’t provide, they can use the Ollama model instead.
I also made a UI with the help of an LLM.
So now the app is functional. Both models are available.
I also thought about adding a tool to my trained AI that detects when the user wants math support, but for now it would only be for math and not other topics, so I’ll see if I end up adding it.
About web support
I don’t think I’ll add a web version since the UI is made using PySide6, and making it web-compatible would basically mean rewriting the whole UI.
Or maybe there’s an easier way to do it that I don’t know about.
Linux and macOS support
Linux and macOS support is possible, but I don’t have a Mac to actually test whether the program works properly on that platform.
Linux should be easier since I can install a Linux distro in a VM and test it there, although I’m not completely sure yet.
And that’s basically where the project is right now.
If you have any suggestions leave them in the comments, thanks!