You are browsing as a guest. Sign up (or log in) to start making projects!

ImageTranslation

  • 4 Devlogs
  • 19 Total hours

Just the image translation, which runs 100% locally

Open comments for this post

4h 25m 13s logged

TL;DR

  • Messaging: Fixed a silent settings synchronization bug by introducing typed, namespaced messages.

  • Hover Translation: Added a hover-to-translate popup with language selection and race-condition-safe handling.

  • Default Language: Users can now configure a default translation language that syncs across tabs.

  • Error Feedback: OCR and translation failures now show toast notifications instead of silently failing.

  • Domain Rules: Fixed empty domain lists incorrectly allowing translation everywhere.

  • Auto-translate: Restored automatic translation behavior that was accidentally removed during the refactor.


Unified messaging

I discovered a pretty annoying bug in the settings synchronization system.

The popup was sending:

ts"notify-settings-changed"

while the background script was listening for:

ts"settings/notify-changed"

Because of this mismatch, settings changes were not propagated correctly and failed silently.

To avoid this kind of problem in the future, I introduced types/messages.ts with typed, namespaced message routing. Now the message names are centralized instead of being manually written across different parts of the extension.

Settings changes can now propagate correctly and update active tabs in real time.


Hover translation popup

The overlay can now translate individual OCR regions directly through a hover popup.

When hovering over an OCR box, the extension displays:

  • The original detected text

  • A target language selector

  • The translated result

The popup uses a small debounce before appearing and has a delayed close behavior when the mouse leaves. If the user moves back into the popup, the close action is cancelled.

I also had to handle race conditions. Translation requests can finish in a different order than they were created, so late responses are ignored when they no longer belong to the currently active popup.

The popup handles its own translation request and lifecycle, which makes it much more self-contained.


Default translation language (model error)

Users can now choose a default target language in Settings.

The current default is Vietnamese (vi), with support for 17 languages. The setting is stored using browser.storage.sync and changes are broadcast to active tabs.

The language selected inside a hover popup is temporary, while the configured default language remains the user’s persistent preference.

In the future we will switch from machine translate or deep learning translate

Error feedback instead of silent failures

Previously, OCR and translation errors could fail without giving the user any useful feedback.

I added utils/toast.ts to provide inline error notifications. Errors are debounced and deduplicated so the user does not get spammed with notifications when multiple failures happen at once.

I also extracted shared popup styling into utils/popup-base.ts to reduce duplicated UI logic.


Empty domains fix

There was another logic issue with enabledDomains.

An empty domain list should mean:

Translate nothing.

However, the previous implementation treated an empty list as if every domain was allowed.

This has now been fixed in both background.ts and isDomainAllowed(). An empty list correctly disables translation entirely.


Auto-translate restored

During the refactor, some of the automatic translation logic was accidentally removed.

This included:

  • autoTranslateIfAllowed()
  • URL polling
  • The live DOM observer
    As a result, the extension would remain inactive until the user manually triggered it.

These behaviors have now been restored, so automatic translation works again when the page loads and when new content appears dynamically.

0
0
9
Open comments for this post

5h 12m 10s logged

tl;dr

WXT.js: Refactored the extension into smaller, more maintainable modules.
Context Menu: Inject content scripts on pages to reliably detect clicked elements — a privacy/resource trade-off.
Overlay: Reworked image overlays to be more independent from the website’s DOM and avoid rendering on invisible images.
SPA Support: Added DOM-change detection for React-style navigation, at the cost of some extra resource usage.
PaddleOCR: Still fighting text-box grouping 😭. Current grouping is imperfect, and we’re considering a better OCR model.

WXT.js

This is really easy. Almost everything is guided on their introduction page, so I need to follow it. Also, I need to refactor code from 1 huge file into many smaller files, which helps me maintain this more easily

better context menu ( and first trade-off)

Uh, due to the main problem that extensions don’t know which one was clicked. So I need to trade off user privacy with convenience. From now on, extensions will inject a content script into every page that you open, including enabling and disabling auto-translation

better overlay box

Over time, the extension displays an image by adding an extension element to a div, which is the image tag’s parent. But now we switch the approach to create a separate div that does not depend ( or depends less) on how the page works. Also, we need to check whether the current image displays are visible, or we will not render an overlay box for them.

Add case for SPA web

For frameworks like React.js, navigation is often handled as a Single-Page Application (SPA) update rather than a traditional page reload. When the user navigates to a different page, the application may update the DOM and the URL in the browser without triggering a full page refresh.Because of this, the extension cannot rely only on events such as page reloads or tab switches to detect when it needs to update its state. We therefore need to listen for DOM changes so the extension can detect when the application has rendered new content and reload or reinitialize its functionality accordingly.

This will consume more resources, but we are unable to measure them

Paddle-OCR and grouping text box (I hate this; why yolo )

This is the very unsolved problem that may annoy the user and me. The problem is paddle-ocr working like this, meaning it will separate the text bubble into smaller parts, which will make translation crazy. Currently, we are adding grouping to solve the issue of text on the same line, but the solution is not ready yet. Also, we are considering switching models for a better experience

0
0
3
Open comments for this post

6h 8m 1s logged

tl;dr

I am trying to make the extension more convenient for the user and boost the performance of AI tasks. And switch to a new framework for a better development cycle
(https://crxjs.dev is really bad; don’t use that currently)

Why

Because I recently found a manga page that has the manga that I want to read, I tried to make the extension more useful by adding a domain system (URL matching system ) which will automatically translate pages that match the URL.

CORS problem

This is not the main section, but I want to tell you about why this extension has some limitations with strictly page.
Due to the original idea, focus on local first, but Chrome’s security design mainly blocks other processes from accessing image data of pages that have a config to block ( their CORS config ), so our extension will unable to run on that page, also i will work on a local backend to handle this problem (in a good situation)

#Domain system design

This design is quite weird because that firstly working on all page start will that domain. Then I used that, and all image include thumnail will be translated, and so i switched to a regex design, which will be more dynamic for users

Extension page

I am trying to make them easier to use, so I designed a simple ui ( tell AI to do that ). That will be more useful in the future ( currently, that just helps me in debug stage )

Background change

Because of real-time (auto-translation mode ), we need to switch from using single mode to batch mode of the OCR library. If we continue to use the legacy solution that will make the user’s computer overloaded and cause a huge delay ( and error rate ). So we are switching to this solution. Also, is the batching solution for the background ( manager process, orchestrator)

The background process will collect tasks from the content.js process in all tabs and then send them to the OCR and translate process ( batchingly, like 5 images per step, in about 80ms if no image is in the queue to keep offscreen work)

build problem

Vite builds to modular JavaScript code. But browsers don’t like that due to backward compatibility, so they reject it.

next step

I will migrating to mordern framework (https://wxt.dev/), which will help me in the build process (I have a build problem with the module webpack of Vite, sadly )

0
0
12
Open comments for this post

3h 33m 48s logged

After many efforts, I can OCR the image locally. (pic. 1 )

Tech choosing

Firstly, I need to find a way to help me use the npm ecosystem. So I know that I can use Vite ( because Chrome just cares about how you define config in manifest, other files are not important ). but their don’t support live-reload, which makes me feel very bad in development. I hope I can use https://crxjs.dev/guide/installation/create-crxjs/ for this.

UI and UX development

Then I think that I want to build up some menu in the extension page, which will make my extension very modern. But I think about that, if I try to translate just one image, how can the user translate just one image? Because of that, I decided to build up a context menu that allows users to choose what they want to translate and just leave the extension page there with hello text. (pic. 2 and pic. 3 )

CORS problem

Then I need to solve some problem with CORS in Chrome ( legendary web development problem ). They won’t allow me to just read the image tag, so I need to send the image URL to the background

Context problem

But background doesn’t have the context (like document.createElement or canvas, which will be able to boost my OCR speed ) which normal HTML page has, so I need to create an offscreen. Then the CORS problem comes back, and I need to build up some indexed storage solution to fix this. Thanks to Chrome.

TL;DR

And after their cors problem, isolation problem, chrome problem i here to present you the image that has the box showing you text that you see 😢😢

0
0
2

Delete project?

Are you sure you want to permanently delete this project? This action cannot be undone.

All devlogs, followers, and associated data will be removed.

Followers

Loading…