Devlog 2: it found the hidden message, then cut it off mid-sentence

You can hide text inside a photo by nudging the colour of individual pixels by one step. The image looks identical to any eye and to most software. The message just sits there, spread across a few thousand pixels, waiting for someone who knows how to look.

Trawl is the thing I am building that looks.

The good part

I hid a note inside a photo of the Eiffel Tower at night and dropped it in. Trawl found it:

If you can read this than Trawl was a success! If not then well.. lets just say.. prepare to los

Right message, right place, picked out of 42 different ways it could have been hidden. Then it stopped mid-word.

That was not a decoding failure. I had written a line months earlier that trimmed the preview to 96 characters, and my note was 155. The tool read every single word and showed me two thirds of one. Somehow more annoying than it failing outright.

Fixed, and there is now a button that pulls the whole thing out.

The message that was never there

An earlier test went the other way. Different image, and Trawl confidently announced it had found BM{GEBF}.

Nothing was hidden in that file. Those eight characters turned up by chance inside the image’s compressed data, and my scanner was hunting for that shape anywhere in the file. Give it half a megabyte of near-random bytes and the shape appears on its own eventually.

A tool that invents answers is worse than one that finds nothing. Matches now get judged by where they came from, and anything sitting in compressed data gets demoted with the reason shown rather than announced as a discovery.

Both bugs had the same cause

My test images were fake in a way that mattered.

I was generating smooth colour gradients to test against. Real photos are not smooth. They have grain, texture, tiny variations between neighbouring pixels. My detectors passed every test I wrote and then fell over on the first real photograph, three separate times, because the thing I was testing against did not behave like the thing I was testing for.

I now write a test that checks the test image before any test that uses it. That one took me embarrassingly long to arrive at.

The feature I deleted

I built automatic warnings into one of the new tools, then found it flagging a completely clean image. I could have nudged the threshold until the warning went away.

But then the number would exist because it made one test pass, not because it meant anything, and it would start guessing again on the next photo. So I deleted the feature. The tool shows you the measurements and makes no accusations.

The two tools that do make claims are based on published research from 1999 and 2001. Given an image with a quarter of its pixels carrying hidden data, they estimate 25.0% and 25.9%. Those I trust.

Next

Reading camera metadata, which is where people hide things when they are not hiding them in the pixels.