side devlog 3: less clutter + a manager page

the page had way too much going on fr. cut the face readouts, the demo stepper, extra stats and the giant verify box.

• headline rises in word by word, the face boots up with its eyes opening, sections fade in once, and the nav has a little pill that slides around. all of it switches off if u have reduced motion on
• made a site manager (npm run admin). edit any text, pick which commit stable points at, update the roadmap, all next to a live preview. saves as u type
• drag an installer onto a platform and it gets hashed on upload, so the page shows the real sha-256. no more copy pasting hashes by hand
• it only runs on localhost and never ships in the build, so nobody’s editing my site but me. hehe nice try.

Now that i think about it. i think i did way too much unneccessary stuff.

side devlog 1: ev has a website now

vite + tailwind + plain js. no react, cause ev’s whole thing is being lightweight and it’d be kinda hypocritical ngl

• nothing on it is made up. a script reads my git log before every build, so the commit list, the iteration number and the test count (950 btw) come straight from the repo
• the demo replays real ev errands with the real tool names. the amazon mouse one asks “Confirm?” before it touches the basket, same as the actual thing
• downloads are github zips pinned to the iteration commit since ev ships as source, plus a one liner to check which commit u got
• ctrl+k menu to jump around cause i’m lowk addicted to command palettes

works on phones too, tested it at like 5 screen sizes

side devlog 2: the face made it onto the site

the hero used to be a particle orb. it was mid. ev literally has a face so now the site uses that instead

• ported the qt renderer (ev/face/render.py) to a canvas and it reads the same expressions.json as the real face. change a mood in python and the site picks it up on the next build
• eyes follow ur cursor, it dozes off if u leave it alone for 15s and wakes back up when u move. click it and it gets startled
• big bug: it kept going “surprised” every few seconds and stopped following the mouse. turns out mouse events can be 1ms apart so a tiny wiggle read as 5000px/s 💀 now it measures over 100ms and only near the face
• glow is additive now so the eyes don’t have that weird dark rim anymore

EV devlog: iterations 12-14

~10k lines added. kinda unhinged week ngl.

Iteration 12: ev learns who’s talking and gets self aware

• whisper kept mishearing me so now EV keeps track of how i sound. it learns the words i say a lot (ev/hearing.py) and loosens its “didn’t catch that” threshold if i’m just a quieter/mumblier speaker. it only ever loosens, never gets stricter, so it can’t drift into acting on noise
• stuff it learns is stored in a json, tests don’t touch it so the real profile stays clean
• wake word got a glow up: saying just “ev” while it’s on standby wakes it. before you had to say the whole “wake up” thing like a password. bro it’s literally your name
• face now knows it’s a face believe it or not. chat can take a mood so “look angry” or “wink at me” actually does something, and there’s a demo mode that cycles through every expression
• readme got gutted (585 lines → basically nothing) and moved into OVERVIEW.md, cause nobody was reading that

Iteration 13: EV can use any app, not just the browser

i forgot what all i did. but here is the best recap of what i remember.

• new tools/desktop/ folder. basically three tiers now, cheapest first:
  1. window stuff (list/focus/close/minimise) straight through the OS, no model call
  2. accessibility tree (AT-SPI on linux, UI Automation on windows, System Events on mac). EV reads buttons and fields by name instead of guessing pixels
  3. screenshots + vision, only when the first two can’t cook
• new app_control and app_agent tools so “turn on dark mode in settings” doesn’t burn a screenshot
• wayland was the final boss. x11 screenshots come back fully black there lol, so ev was describing a black rectangle. fixed with mutter’s screencast + remote desktop stuff
• wrote a tiny gnome shell extension ([email protected]) so wayland windows have real positions and focus works
• closing apps is polite by default. if an editor asks “save?” ev reads it out and waits, doesn’t just nuke your work. force kill needs a confirm. learnt that the hard way.
• added jeepney and comtypes to requirements, both lazy imported so nothing changes if you never use em

Iteration 14: memory that actually remembers + overlay on linux

• semantic memory (ev/semantic_memory.py). remember_fact with no key stores a free note, and ev pulls up the ones relevant to what you just said. all local search (BM25 + fuzzy for misheard words), gemini embeddings are optional and only used inside tool calls. has dedup, archiving, backups, so one bad write doesn’t wipe everything. forgetting stuff asks for a confirm
• overlay on linux. the red “ev is driving your screen” frame was windows only. now it works through xwayland (tools/desktop/xhud.py), click-through and everything. the kill hotkey on gnome wayland goes through the extension (bumped to v2) since wayland won’t let apps grab global keys
• fixed a crash where tk objects got freed on the wrong thread and the process died with exit 134 after every takeover. cool
• vision got better on desktop, new test_desktop_vision.py
• tool schema had to get squished again to stay under the groq token limit. every new feature = something else gets shorter, it’s the circle of life

where things are at

works: voice, typed mode, browser errands, desktop apps on windows + ubuntu, memory.
still janky: wayland without the extension (no window positions), snapped apps refusing accessibility, and my commit messages.

next up: idk, tests probably. maybe sleep.

E.V. got a face (and moved to Ubuntu)

Iterations 9, 10 and 11. Lowkey the biggest glow up E.V. has had so far.

9 + 10: WE GOT A FACEEEE

E.V. has a face now. It’s a little floating dark screen with two glowing eyes that sits on your desktop and reacts to whatever it’s doing. Listening, thinking, talking, happy, confused, sleepy, and even angry for some reason. yeah dont ask.

• It’s its own process. The face runs on PySide6 in a separate process, so the main assistant never loads Qt and stays tiny. If the face crashes, E.V. doesn’t care, it just keeps going. No PySide6 installed? No face, nothing else changes.

• Every mood is just JSON. All the expressions live in expressions.json as eye shapes, lid angles, glow and colour. The eyes ease between moods smoothly, blink on their own and glance around so it never looks dead inside.

• Captions. What you said and what E.V. says back pop up in a speech bubble next to the face. The bubble is click-through so it never gets in the way.

• It only pulls up when you call it. Say “E.V.” and it slides in, stop talking and it dips. If you’re watching something full screen it stays hidden, because nobody wants a robot photobombing their movie.

• Media control. “Pause the music”, “next song”, “turn it down”. Works on Linux and Windows without installing anything extra.

• Finally Stopped Ignoring Ubuntu. I switched to Ubuntu and half of E.V. broke ngl. The mic needed a fallback to pw-record / parec / arecord, app launching had to learn .desktop files, and the voice plays through GStreamer. Wayland doesn’t let apps float on top, so the face sneaks through XWayland instead.

Bugs that were lowkey unhinged

• The face crashed instantly when started from VS Code because the snap terminal leaked its own libraries into Qt. Fixed by cleaning the environment before Qt loads.- On Ubuntu the “default microphone” was actually my speakers. E.V. was literally recording YouTube. Now it picks a real mic by name.- The glow was eating a third of a CPU core. Cached the drawing and dropped the frame rate when nothing is moving. Way chiller now.

11: E.V. stopped talking to itself. lol

This one was actually wild. On speakers, E.V. would hear its own voice, think I was interrupting, transcribe itself and then reply to itself. Forever. Main character behaviour, but no.

Three fixes, stacked:

• Compare against itself, not the room. Interrupting used to mean “louder than background noise”, and speakers win that every time. Now you have to be louder than E.V.’s own voice coming back through the mic.

• It learns what I sound like. A small voiceprint learns my voice and E.V.‘s voice through the speakers, and keeps adapting the more you use it. If the audio sounds like E.V., it’s ignored. It’s not face ID for voices, it just tells “me” apart from “the speakers”.

• It checks the words. If what it just heard matches what it just said, it gets dropped. This is the one that makes the loop actually impossible instead of just less likely.All of it is pinned in tests so it can’t sneak back. The suite is at 866 tests now, all offline. That’s the update fr.

yk i gotta come clean. half of this major update was vibe coded. forgive me. but the other half was done by me.
ALSO THIS DEVLOG is not vibecoded. it took too long.

E.V. can drive my screen now

This update was all about letting E.V. actually take over the computer, not just open apps.

What’s new

• It can run a whole errand by itself. I said “find me a gaming mouse with an infinite scroll wheel and add it to my amazon cart” and it searched, picked one and added it.
• Web stuff goes through Playwright and reads the page instead of taking screenshots. Screenshots were eating my Groq rate limit in like 4 steps.
• There’s a red frame around the screen plus a little status panel whenever E.V. is in control, so you can tell it’s doing things and it’s not a virus lol
• Kill switch: ctrl+alt+q, or just say “stop everything”. It locks E.V. until you tell it to unlock.

Bugs I hit

• It added the same mouse to my cart 4 times. Now it remembers what it already clicked and won’t do it twice.
• It got stuck on Amazon because “Search” matched 4 things on the page and it clicked a hidden one. Now if a step fails it reads the page and figures out a different way.Next up: getting it to use Brave instead of its own Chromium.

I SPENT 6 Hours DEBUGGING

i dont even have a major new photo to add.
Please Understand

Gave My Assistant FULL SCREEN Control using Playwright.

IT can now do complex tasks autonomously. eg: find a Gaming mouse under 5k with an inifinte scroll wheel. it will do that.

The rest of the time was spent debugging

EV - THE AI ASSISTANT

Just Spent 8 Hours on EV, Creating And Refactoring. to finally achieve as close as i can get towards the real thing with the resources that i was allowed.

It is fully controlled via Voice ( BUT can be switched to text, if necessary). it can do advanced tasks like open Apps, Create Files, And Much more.

All you need is a Groq or Gemini API key. and it consumes a maximum of 200MB RAM. I couldn’t just use an LLM for this purpose as I’m working with a PC that has 4GB RAM.

AS OF NOW, this is a terminal program. but soon this will be much bigger. it will become a full scale app with enjoyable UI/UX Design and an avatar for EV