EV devlog: iterations 12-14

~10k lines added. kinda unhinged week ngl.

Iteration 12: ev learns who’s talking and gets self aware

• whisper kept mishearing me so now EV keeps track of how i sound. it learns the words i say a lot (ev/hearing.py) and loosens its “didn’t catch that” threshold if i’m just a quieter/mumblier speaker. it only ever loosens, never gets stricter, so it can’t drift into acting on noise
• stuff it learns is stored in a json, tests don’t touch it so the real profile stays clean
• wake word got a glow up: saying just “ev” while it’s on standby wakes it. before you had to say the whole “wake up” thing like a password. bro it’s literally your name
• face now knows it’s a face believe it or not. chat can take a mood so “look angry” or “wink at me” actually does something, and there’s a demo mode that cycles through every expression
• readme got gutted (585 lines → basically nothing) and moved into OVERVIEW.md, cause nobody was reading that

Iteration 13: EV can use any app, not just the browser

i forgot what all i did. but here is the best recap of what i remember.

• new tools/desktop/ folder. basically three tiers now, cheapest first:
  1. window stuff (list/focus/close/minimise) straight through the OS, no model call
  2. accessibility tree (AT-SPI on linux, UI Automation on windows, System Events on mac). EV reads buttons and fields by name instead of guessing pixels
  3. screenshots + vision, only when the first two can’t cook
• new app_control and app_agent tools so “turn on dark mode in settings” doesn’t burn a screenshot
• wayland was the final boss. x11 screenshots come back fully black there lol, so ev was describing a black rectangle. fixed with mutter’s screencast + remote desktop stuff
• wrote a tiny gnome shell extension ([email protected]) so wayland windows have real positions and focus works
• closing apps is polite by default. if an editor asks “save?” ev reads it out and waits, doesn’t just nuke your work. force kill needs a confirm. learnt that the hard way.
• added jeepney and comtypes to requirements, both lazy imported so nothing changes if you never use em

Iteration 14: memory that actually remembers + overlay on linux

• semantic memory (ev/semantic_memory.py). remember_fact with no key stores a free note, and ev pulls up the ones relevant to what you just said. all local search (BM25 + fuzzy for misheard words), gemini embeddings are optional and only used inside tool calls. has dedup, archiving, backups, so one bad write doesn’t wipe everything. forgetting stuff asks for a confirm
• overlay on linux. the red “ev is driving your screen” frame was windows only. now it works through xwayland (tools/desktop/xhud.py), click-through and everything. the kill hotkey on gnome wayland goes through the extension (bumped to v2) since wayland won’t let apps grab global keys
• fixed a crash where tk objects got freed on the wrong thread and the process died with exit 134 after every takeover. cool
• vision got better on desktop, new test_desktop_vision.py
• tool schema had to get squished again to stay under the groq token limit. every new feature = something else gets shorter, it’s the circle of life

where things are at

works: voice, typed mode, browser errands, desktop apps on windows + ubuntu, memory.
still janky: wayland without the extension (no window positions), snapped apps refusing accessibility, and my commit messages.

next up: idk, tests probably. maybe sleep.