zWork Browser Control

I benchmarked zWork (desktop ai agent made by me) using zbctl (extension) to test its new browser use abilities. ive used the prompt in the picture below. the model answered all but one correct. the google form used to “benchmark” the model was: https://docs.google.com/forms/d/e/1FAIpQLSfHyJDMoOuDENsrrTmFsGtoDfe-9RRhlB2SDkKgYQlAAXOvhg/viewform

the form included a variety of answer types, including radio and dropdowns. this is by no means a final polished product, but is certainly a good proof of concept. I will continue working on zWork and improve the harness to also improve the models tool calling and accuracy. Thanks guys

i tested zbctl using zWork (the agent im working on) as well as using claude code. initially there wasnt an endpoint for claude code, but claude code made one by itself. to make things fair, both harnesses were wired up to use deepseek-v4-pro as the model, and after testing the same prompt and same task, claude code basically won. my harness zWork needs a lot of work, but thats a bit besides the point. The zbctl bridge extension works.

i made a desktop agent (still in development)
now im making a browser extension to let it attach onto browser sessions that users are already signed in on.

Cool stuff:

Push-based DOM snapshots (MutationObserver auto-detects page changes)
Stable element IDs via WeakMap (same element always has same ID)
Works with Gmail, Google Forms, real sessions
Tab management: switch, open, close specific tabs
File upload/download support

Technical stuff:

Python FastAPI daemon (port 8788)
Manifest V3 Chrome extension
Content script walks DOM → structured JSON
200ms debounce for settle detection, 3s hard cap
Biggest limitation: Cross-origin iframes blocked by browser security. Can’t interact with YouTube embeds, payment forms, etc. But handles ~80% of normal browsing fine.

right now it does work

magic