AEVA Devlog #7
Same progress as last time, the model is currently in training. I had some trouble back home and wasn’t able to train a model since my computer kept shutting down however the training will be done soon. Didn’t have an image to share lol.

AEVA Devlog #6
The architecture is now very strong and is more than fast enough to reach the goal in 3 hours however I’m having issues creating the perfect dataset since the model becomes to focused on the roleplay datasets. I’m going to try using some online datasets as is for this to see if that helps.

AEVA Devlog #5
Running into the same issue as last time but this time I think I have a better fix and will be distilling some model. in the meantime, I created a simple app icon for when I release it and will be working on the UI

AEVA Devlog #4
Training failed again, this is getting tiring but there’s no other real way to test the architecture. This time the training failed due to the dataset and how it was distributed. I was actually only giving about 0.13 tokens per parameter which was leading to it overfitting too. now I’m in the process of making the architecture more memory efficient and faster at processing data, as well as increase dataset size.

I also finally started working on creating a github repo as well as an explanation on how it works

• Updating architecture to handle more data
• Increasing dataset size
• Creating github rep
• Creating explanation document

AEVA Devlog #3

Well bad news, the AI infact did find a loophole, although after a lot of fixing as well as creating a better dataset, I’ve started training once again which will complete in another 3 hours, until then I’ll be working on another one of my projects. This is likely going to happen many times since I did decide to start doing such a hard project, I just wish I can finish the project in time. Also, for now I’m going to use this CLI Logo.

• Revising architecture
• Creating new dataset
• Started training new model

AEVA Devlog #2

After another 16 hours of working on the project, The AI has finally reached a CE of 1.0!!!!! actually it reached a CE of 0.9 to be exact in only 3 hours of training which is just spectacular. Of course, the model still has to be thoroughly tested to see if it’s actually conversational since the model could have found a loophole to get a lower CE than it deserves. After some further testing ill post again within the next 7 hours.

• Done training model
• Started testing on the model since goal has apparently been reached
• Creating GUI

AEVA Devlog #1

This is Project AEVA, an AI I’ve been trying to create completely from scratch on consumer hardware I own to compete with the current SOTA architectures. I originally started creating this AI because i thought that LLM’s although usefull and highly advance come with many issues esspecially on low-end hardware, halucinations, memory limit and the frozen weights after training. instead i want my AI to actually live with a human level life-span and continuously learn during that tmie slowly increasing its parameter size.

After months of designing and rewriting, AEVA (The name of the model I’m making) has officially reached its first training run. Instead of using Transformers, I’m building a completely new cognitive AI architecture designed to compete with frontier models while running on consumer hardware. The current architecture scales in O(n), is designed for massive context and persistent memory, and aims to support self-improvement and offline “sleep” learning in the future. Right now, the first model is being trained for 4 hours into training on my RTX 5090, and if everything goes well, it should become conversational soon. This is just the beginning for AEVA.

• Completed designing and programing the architecture
• Started creating a better dataset for the model
• Started Modl Training