I am improving the long-term memory but it overall seems to be going fine. I added a way to prevent the conversation history from overflowing, which comes in the form of an if-statement in my main loop to check how large the conversation has gotten, and if it has passed 100 messages, it summarizes the oldest 50 but keeps the newer 50. I am noticing however that my computer has started slowing down considerably when making post requests to the model. I think I am going to try to find a way to stop using such a large context window but this has gotten rather out of hand. I do want this to be accessible to the average person, so I intend to somehow lessen the load when running locally. Saying this, at this time I am not sure how yet so I am going to have to go research that.
Comments 0
No comments yet. Be the first!
Sign in to join the conversation.