Finally some real progress!
For the past 4h (And even more spent chatting with tencent-hy3) I have been making a NER (Named entity recognition) system for my engine.
What does this mean?
It means that now after system generates memories from chat history they get passed into entity resolver which extracts entities(people, locations, events) from them and forms relationships between them.
What comes next?
Next I will be adding entity deduper (even harder problem).
What I am missing right now is: What if NER system generates direct duplicates of entities or (even worse) indirect ones.
- Direct duplicate would be: Same entity name appearing in different memories. This is easy to fix. Just a regular
==scan. - Indirect duplicate would be: Same entity referenced by slightly different names/properties. For example: Rust[Language] / Rust[Programming Language] .
This problem is still unsolved by me although I have some ideas about how to go around it. However this is theme for next devlog…
Comments 1
No comments yet. Be the first!
Sign in to join the conversation.