The hard part was knowing when to stop

Here is what someone hands you in a competition.

NWE2ZDc4Njg1YTMzNzQzMDYxNDg0YTZjNWE1NjM5

Eighty-eight characters of nothing. No hint about what it is or how many times
it has been through the wringer. You paste it into Trawl and it comes back:

base64: 5a6d78685a33743061484a6c5a56397359586c6c636e4e665a47566c
hex: ZmxhZ3t0aHJlZV9sYXllcnNfZGVlcH0=
base64: flag{three_layers_deep}

Three layers, unwrapped one at a time, and the answer sitting at the bottom.
Nobody told it there were three. Nobody told it what any of them were.

The new half

This is Mantis, the code breaking side of the project, and the first thing I
have built that does not need a file at all. Paste a string, get an answer.

It is named after the mantis shrimp, which punches hard enough to crack a crab
shell and sees colours we do not have the eyes for. Force and perception, which
turns out to be the whole job.

Decoding was easy. Stopping was not.

I expected the tricky part to be writing the decoders. It was not. There are
thirteen of them now and most took twenty minutes.

The problem is that decoders always succeed.

Take the sentence “the quick brown fox jumps over the lazy dog”. Delete the
spaces and what you have left is, by pure coincidence, a perfectly legal base64
string. Feed it to a base64 decoder and it will not complain. It will hand you
back a paragraph of garbage with total confidence.

So a tool that decodes whatever it is given will happily take your answer, turn
it into noise, and report that as progress. Mine did exactly that, repeatedly,
while I was building it.

The fix was to stop asking “can this be decoded” and start asking “did decoding
it make things better”. Every layer now has to earn its place. It gets kept
because of what fell out of it, never because of how it looked going in.

Then I got it wrong the other way

Once it was cautious, it became too cautious.

Look at that chain again. The middle step decodes to a wall of hex digits. That
is not more readable than the base64 above it. It is arguably worse. So my
careful new version took one look, decided the second layer was not an
improvement, and stopped one step short of the answer every single time.

Which is the lesson, really. Getting closer to something does not always look
like getting closer. It had to learn to follow a chain to the bottom and judge
the destination, not each footstep.

Where the catch goes

When it does find something, it lands in a panel at the top of the page called
the Cod-end.

That is the closed end of a trawl net, the bit that drags along behind and where
everything you caught ends up. It felt like the right name for the box that
holds the answers.

What it still cannot do

It does not break real encryption, and it never will. If something is properly
encrypted, Trawl says so and stops, rather than producing a confident sentence
made of nothing.

It also cannot solve every classical cipher yet. Two of the harder ones need a
much better model of what English looks like than the short word list it
currently carries, and I would rather ship the gap than fake it.

Next up: hashes.