New Architecture
After very loooong thinking I figured that letting agents/models generate memories themselves on their session will lead to inconsistent and unpredictable memory.
For that reason I implemented memory generation on the server.
FoR NeRdS
Architecture setup
- User creates new session via
new_session endpoint.
- Using
sessionId user calls add_memory endpoint which expects: newMessages field.
- Servers inserts
newMessages into session_messages table in database
- Background job (Currently
fire-and-forget) is scheduled and response instantly returned
Background Job
Using sessionId and newMessages job fetches memories assigned to session.
With messages and session_memories it calls an llm(currently nvidia/nemotron-3-ultra-550b-a55b:free)
BeCaUsE It Is FrEe
which extract memories from these messages. session_memories are passed as context so that model can understand what each entity means
Thx for reading!
See you soon… ;D