Devlog 3: the PNG half is finished

Here is my favourite way to hide a message in a picture, and Trawl can now catch it.

A PNG can store its colours as a numbered list. Pixel one uses colour seven, pixel two uses colour twelve, and so on. Nothing in the format stops that list from holding the same colour twice. So if entry 10 and entry 200 are both the same shade of blue, every blue pixel can point at either one. The image on screen is pixel for pixel identical. The file underneath is not. That choice is a hidden bit, and even a small 64 by 64 image gives you about a thousand bytes of them.

That was the last thing on my list. PNG support is done.

What done means

Trawl reads every valid PNG now, including the two kinds it used to refuse outright. It opens compressed comment blocks instead of shrugging at them. It finds text written the Windows way, where every character is followed by an invisible null byte that made my old scanner walk straight past it. And its brute-forcer tries reading hidden bits down the columns as well as along the rows, which doubled the number of combinations from 42 to 84.

Sixteen tools, all working on one format properly. That felt worth stopping to notice.

The bug where it found two flags and threw them both away

I planted a message in a compressed comment and another as wide text, then ran them through. Neither showed up.

Both had actually been found. The problem was one line where I merged results: I was taking either the results from the PNG-specific reader or the results from the general one, never both. So the general scanner would find the wide text, hand it over, and the merge step would quietly drop it on the floor. Two correct detectors, one bad if.

A related version of the same mistake: I inflate compressed comments now, but the flag scan had already run before that text existed, so nobody ever looked at it. It re-scans after decompressing.

The one I got wrong for twenty minutes

One of my clean test images started tripping a detector. My first guess was the new interlaced decoder, since that was the newest thing.

So I built the same picture again without interlacing and ran it. Same false alarm. Not the decoder.

The real cause was my test image itself. The detector works by checking whether pairs of colour values appear equally often, which a hidden message forces and a real photograph has no reason to do. My test image was a smooth gradient, and smooth gradients spread their values so evenly that the pairs come out equal on their own. I had built a control that genuinely looks guilty.

I gave it a tone curve, the way a real camera would, and it went quiet. That is the fifth time this project a test image has been the thing that was broken.

Next

Everything except PNG. GIF and BMP first, since they should light up all sixteen tools the moment they decode. Then extracting files that are hidden inside other files, which Trawl currently finds and then refuses to hand over.