Project Prometheus

A LOT HAS CHANGED. Alright so I realized my current capabilities couldn’t make the project I wanted so I used a coding LLM from Nvidia to help. It wrote the entire frontend for me because I have noooooooo clue how to write a frontend. It also modernized how messages are displayed to the user. So for those that don’t know, tokens are essentially parts of words, 1 token being around 3/4th’s of a word. In the past, the model would generate tokens and then deliver them all at once in one output. However, the appearance to the user was that the program was freezing up because after putting in their input nothing happened. This make it so text appears in realtime, or well close to it, as the LLM is making a response.

MEMORY SYSTEM OVERHAUL. Alright for those following along, I previously had a system where I would store memory inside a JSON file. However, I have been researching how real agents store memory and they seem to do it very differently. In short, they store memory by embedding text into a vector. This process is done by turning a string into a vector with 800+ dimensions, like: <v1, v2, v3……. v800>. This way the model can find similar meaning setences, ideas, etc. using what it does best, numbers. It matches the vector to the next closest vectors to it when searching through memory. This makes finding things in memory wayyyyyyyyyyyy easier and faster because the model can both find stuff in memory that may not have any matching keywords but mean the same thing(like “I am broke” and “I have no money” would be plotted next to each other even though the main keywords aren’t shared between each other, this is a rough example though) and it can also do it in a way more efficient manner. This is called semantic searching(searching by meaning, not keywords). I also have an SQLite database, mostly used for project data. The model stores stuff like project id and summaries about projects in neat, clean rows. This helps the model find stuff better because it can now search by id, but it also makes it easier for me to see stuff :).

THIS TOOK FOREVERRRRRRRRRRRRRR. What I did is I made it so the model went from getting the User’s input and guessing on what to do next each iteration to making a predetermined plan and then each iteration would complete a step of the plan and that would be notified to the model, then it works on the next step of the plan and so forth. The terminal output associated with this Devlog would definitely help explain this a little more. To summarize though, I basically revamped the architecture of program to make it so that each time a request is made by the User, the model makes a plan of execution in steps and then iterates through the plan to complete the request.

I am improving the long-term memory but it overall seems to be going fine. I added a way to prevent the conversation history from overflowing, which comes in the form of an if-statement in my main loop to check how large the conversation has gotten, and if it has passed 100 messages, it summarizes the oldest 50 but keeps the newer 50. I am noticing however that my computer has started slowing down considerably when making post requests to the model. I think I am going to try to find a way to stop using such a large context window but this has gotten rather out of hand. I do want this to be accessible to the average person, so I intend to somehow lessen the load when running locally. Saying this, at this time I am not sure how yet so I am going to have to go research that.

LONG-TERM MEMORY!!!!! Well kind of…. I added a way for the model to remember information between sessions, but there is a caveat. The user has to tell the model in the first place that it should remember something. Otherwise that information will be stored in RAM and be wiped when restarting the program. Unless specifically prompted, the model won’t use the remember tool to store information. But this is still pretttttttty nice. All memory is stored in a memory.json file which the model can access and search through to find stuff.

I know this is just a huge block of text but we’re actually getting somewhere!!!!!! The model can know reason after tool results and use that to generate responses!!!! It’s a rather simple thing but important. For example, in one of my tests I asked it to create a file called numbers.txt and put the numbers 1,2,3,4,5 in it. Then I asked it to read the file and tell me the sum. The flow looks like this: Input -> Tool Call(write_file) -> process tool result and check against original user input -> Tool Call(read_file) -> process tool result -> check against original user request -> reasoning -> response!!!! EDIT: the screenshot is an image of the results of some tests I ran.

BIIIIIIIG NEWS!!!!! The terminal response says it all , but the logic is working very well. Just fixed the little bug at the end where the model returned too JSON messages so that should go away, but if you look at the top right of the image the test2.txt file actually generated. This means my model is using the tools correctly and is starting to get the hang of it. WHAT I LEARNED: building the prompt you send to the model after each user input is VERYYYY important. You kind of have to spoon feed models information, or else they don’t know what to do. ESPECIALLLLY with a smaller model.

EDIT: I also changed models to a slightly bigger one from Ollama, the gemma3:12b model. I highly recommend Ollama for projects such as this, as it is free to download and run a tonnnnnn of local models on it. This is a list of all of the models that are available:https://ollama.com/library.

I am trying to give it functionality to add stuff to files and it worked but it’s still rather shaky. I mean at least it worked!!!!! The debugging on this is veryyyyyy annoying. I still don’t know why it is putting “Your final answer here” in chat all the time but I am going to fix it!

I made it so it can launch apps for me! The process goes something like this: User Input -> AI decides if it needs tool and what tool to use -> Tool Output -> AI takes output and decides if its sufficient or it needs another tool -> Tool result or final response -> final response. Reallllly happy with this new update to it.

BIG NEWS!! So I completed some early tools, most of them being based on file management. I added support for the agent to make multiple tool calls as well if necessary. So essentially what it does is that it takes the user’s input, decides what tool to use, uses it, evaluates the result of the tool against the user’s original request and decides if it has sufficient information to respond to the user’s request or if it needs more tool calls. In this scenario, the user asks to find a file in a folder. The model first uses the search_projects tool to find the folder, and then uses that information to find a file!!!!!! Currently focusing on allowing it to launch apps, so far it can launch notebook :).

I am currently trying to give it access to make certain decisions to open tools for me. I successfully got it to open youtube for me!

Currently just finished linking a local model from Ollama, llama3, to my project in the terminal. I eventually want to end up developing my own model because I am not entirely satisfied with the power of this model but for right now this is going prettttttty well.