Nothing special

I was busy restructurizing project again because of the fact that I added research folder to repo. In it I will be placing researches that I did to come up with some decision about a project.
One that I did today was embedding formats research.

What did it cover

It used python script to go through multiple formats that will be passed to llm embedder. I used results to pick one that will work the best for our case.
Funny that it contributed to changing current embedding strategy. Ig research is needed after all…

!! THX FOR READING !!

! Working embedder !

This milestone brings me one step closer to engine MVP as it means that now graph is semantically queryable (by meaning not by accuracy)

What I did ?

I integrated qwen-embedder-8b model into my engine which now embedds (Generates 1024 dim vector representing semantic meaning text) this text block for each entity: canonical_name[type]. That allows you to run vector similarity search accross graph entities which makes engine kind of like a little RAG.

~~ Thats all ~~

THX FOR READING

Cool feature xD

I added auto-add-admin feature. If you file a specific admin_request with very specific message you will receive free admin on page which allows you to chat with assistant ;D

What else?

I also made project more self-hostable in a way that I just expanded instructions on how to run it. (I might forgot bout smth though)

! THX FOR READING THIS SHORT AHH DEVLOG !

Finishing touches AND FIXES to playground

I added finishing touches to the playground and also fixed memory_add endpoint which used fire-nd-forget approach which unfortunately is not allowed on vercel :(

This simple problem caused a lot of headache

I needed to migrate entire function to cloudflare.
It landed on claudflare queues which means that now i have an entire queue system build for backend api xD

NEW FEATURE

request_admin_access i added button to request admin access !

** Thx for reading and see you sooon **

! OpenMem playground is now live !

Only caveat is that to chat with llm inside you need to be on admins list but if you comment your user id under this devlog(or message me on slack: bober with black and white pf) I will happily add you here ;D

What changed?

• I expanded entire db schema by a factor of probably 2 xD. This included handling RLS, CLS (Column privileges) in database and also minor fixes to existing schema. Essentially security stuff
• Completed ui for playground with sessions, containers, data_viewer and llm_chat.

Next steps

1. Ship
2. Add indirect_deduper to make engine actually usable
3. Add recaller to be able to use data inside of the engine
   Later idk… ;D

** !! Thx for reading !! See you soon…**

A lot has happened xD

As you can see 8h have elapsed since last time I have written a single devlog. That is because I have been extremely busy and didn’t even still finish what I have been working on

To the point…

For past 8h I have been making these things:

• Web playable playground (very basic, enough for ship to be accepted because running engine by yourself rn is a big pain xD) 4h
• Updates to entity/relation direct deduper 1h
• Relations inserter (including relation_mentions) 1h
• Cleaning up code mess I have created for past time just trying to get engine to work in any way 1h
• Rethinking/reapplying database schema and restructurizing it to allow me to introduce admins and regular users 1h

Still finishing touches needed before first ship…

This ship will not mean “Engine is ready to use check it out”
It will only mean that I need those hours accepted rn (xD)
Engine still requires implementation of indirect entity deduper which will be pretty hard (as I explained in earlier in devlogs).
This ship will let you create account on playground and chat with model and see how your memories are stored in db that’s it for now..

** As always ;D**

❤️ THX FOR READING ❤️

Full Entity deduper + Inserter !

Now system is capable of taking conversation and turning it into entities that are linked by relations and preserve history of changes!

From technical point of view

What actually changed in past 3.5h was that I fully implemented entity deduper and inserter. Now entities land in database which also means that previously implemented db deduper can use those as references to do its job.

Slowly but steadily

Now system is starting to come to a point where in next 20h it might become actually usable. Only things left now are:

• Recaller - System for querying this knowledge graph
• Periodical Merger - Extension of deduper that will periodically run to merge potentially wrong flagged entities for example: Rust (PROGRAMMING_LANGUAGE) AND Rust (TECHNOLOGY), These two refer to the same entity but couldn’t have been merged by direct deduper because this is an Indirect relationship

!!! Thx for reading and see you soon !!!

Partial Entity Deduper!

After loong 3h of constant designing and rethinking my previous architecture choices (xD) I managed to create direct entity deduplication system.

Hold up… Direct??

Direct meaning actual duplicates/cross references. For example:

1. entity x, aliases: [y,z,c]
2. entity y, aliases: [a,b,c]
   Current deduper setup will merge these two because name y appears in aliases for entity x.

Not so fast

This is just to ilustrate how to distinguish direct from indirect. But in reality hardest part of this implementation was to figure out how to merge these two together (How properties should behave). For example:

• How confidence for given entity should be calculated?
• How to merge unique properties?
• How to preserve history of entity merging (kinda like git diffs)

Still more to come…

I still didn’t design a solution for indirect entity matching (What I will be working on now). How to figure out that entity of name: “Alice” is the same as entity named: “Dr. Alice” when neither of those is present in subject’s aliases array. :((((

There is hope!

After entity deduper is finally finished I will be able to test this system on actual tests checking how well it handles entity resolution, memory understanding etc…

Btw I changed my model to xiaomi-mimo-v2.5 but don’t tell anyone…. shhh

That’s all for today. As always…

!! THX FOR READING AND SEE YOU NEXT TIME !!

Finally some real progress!

For the past 4h (And even more spent chatting with tencent-hy3) I have been making a NER (Named entity recognition) system for my engine.

What does this mean?

It means that now after system generates memories from chat history they get passed into entity resolver which extracts entities(people, locations, events) from them and forms relationships between them.

What comes next?

Next I will be adding entity deduper (even harder problem).
What I am missing right now is: What if NER system generates direct duplicates of entities or (even worse) indirect ones.

• Direct duplicate would be: Same entity name appearing in different memories. This is easy to fix. Just a regular == scan.
• Indirect duplicate would be: Same entity referenced by slightly different names/properties. For example: Rust[Language] / Rust[Programming Language] .
  This problem is still unsolved by me although I have some ideas about how to go around it. However this is theme for next devlog…

Thx for reading this ;D. See you soon

Nvim setup

It will happen to all of us at certain time….

What was I doing

Migrating myself from vscode to NeoVim with NvChad preconfig.
This took a long time for me to get used to it.
I also removed @openrouter/sdk because it absolutely
SUCKS

My question to openrouter team

How can you be a leader in api space and be unable to have up-to-date docs????

New Architecture

After very loooong thinking I figured that letting agents/models generate memories themselves on their session will lead to inconsistent and unpredictable memory.
For that reason I implemented memory generation on the server.

FoR NeRdS

Architecture setup

• User creates new session via new_session endpoint.
• Using sessionId user calls add_memory endpoint which expects: newMessages field.
• Servers inserts newMessages into session_messages table in database
• Background job (Currently fire-and-forget) is scheduled and response instantly returned

Background Job

Using sessionId and newMessages job fetches memories assigned to session.
With messages and session_memories it calls an llm(currently nvidia/nemotron-3-ultra-550b-a55b:free)
BeCaUsE It Is FrEe
which extract memories from these messages. session_memories are passed as context so that model can understand what each entity means

Thx for reading!

See you soon… ;D

Recall & Sketch

For the past ~2h I created a basic sketch of algorithm that I will use and also recalled basics of C++ syntax.

Thx for reading one sentence xD

(Btw graph was made in excalidraw)

Thinker ;D

Currently just trying to figure out structure of this entire project and also…

How in the world!

Will I make usable memory knowledge graph out of text

ThX fOr ReAdInG tHaT sHoRt AhHhH dEvLoG!