Hey everyone! I’m building “Cena”, a local-first visual copilot system that provides real-time geometric telemetry and asynchronous semantic understanding.

Note for the Shipwrights: I actually spent around 35-40 hours programming the core architecture and multi-threading structure last week before finding out about Stardance and installing the Hackatime extension. You can verify my daily development dates and progress through my GitHub commit history!

How the system works:

1. Geometric Layer (~30 FPS): Uses OpenCV to capture frames, feeding them to YOLO-World and MediaPipe Hands (running with GPU Metal/MPS acceleration on my Mac M4).
2. Semantic Layer: A background thread invokes Moondream-2 every 10 seconds to get a text description of the scene.
3. Agent Layer: As shown in the screenshot, the engine merges everything into plain markdown files inside Obsidian (LiveContext.md and PersistentContext.md). Inspired by Karpathy’s LLM-as-an-OS concept, this avoids vector database indexing latency on edge devices.

The code is currently split into two versions (“Cena” and “CenaAmd”) representing the performance updates and VLM optimization of the pipeline. Check out the site here: https://cenadev.vercel.app/

Check out the full repository here: https://github.com/harciufff/cena