AI vs Debugging

I added some tests for Y2038, before, and after, because, of course, that could be an issue. It turned out that Darian timestamps on non-leap years (even-numbered years) were incorrect. But they were correct in leap years.

So… what gives?

Manual Looking

I thought it could be the result of the calendar epoch being incorrect (it did end up being slightly incorrect) - but I tried using other dates on non-leap years, and couldn’t find a consistent offset that all the dates were off by.

This was.. really weird. I also tried setting the year to calculate the year before - no dice.

Taking out the AI

Eventually, I decided, “screw it”, so I took GPT-5 mini and asked it to try to figure out what was happening.

Spoiler alert: It did eventually figure everything out.. but it was.. quite the process to watch it do so.

It detected that the sols-passed-at-the-start-of-month table was actually missing the value for the 8th month. Of course, adjusting this caused tests to fail, so I was hesitant to trust it. It also suggested subtracting 1 from the day count when multiplying by 86400 seconds per day. This was bogus.

Asking it to make a diff

I asked it to make a .patch or a .diff I could apply - and it didn’t work. Missing headers or invalid patch data, whatever it was, it can’t generate a .patch file. Which is kind of hilarious.

It also didn’t look back at my codebase when I applied some of it’s suggestions, so it would be making patches that refer to deleted lines that didn’t exist anymore. Could just be a skill issue.

Figure it out, man

I told it to figure out what was wrong. It spent the next ~16 mins, making a “debug mkmarstime” script, and a bunch of other small scripts like “probe_unix”, “probe_y2038” - in some cases, it was just going around in a circle - making changes that only resulted in the same prior status. I thought it was hallucinating at first, especially with the compiler errors it got. Eventually I’ll look back at the thread and it’s thinking, because like, in some ways, it was stupid, but in some ways it was smart.

I don’t know if the method to debug everything was the most efficient - but it eventually worked. Probably trying to fix the epoch, and also fix the leap years, and also fix the month - all at the same time - overwhelmed it, and my guess is there was no confidence in itself.

I’m talking too much about the AI here for code I didn’t do in a dev log. But anyway, it brute forced tested leap year handling, (it also mentioned “leap seconds” which I’m not dealing with.. so I’m not sure what the heck it’s doing), but eventually all the testcases did past.

Some helpful tidbits

It also did suggest using llround to fix some precision issues, like sometimes conversions being off by 1 second. This is actually not a terrible idea, and I may make a build-time config option for this. Since it was successful, I also asked it to import all its scripts in a test suite - which it wasn’t going anywhere for, so I decided to stop wasting energy and stop it there. Eventually I may come back to it.

Overall thoughts

So it spent at 187K tokens - and even though it did seem to go nowhere, it eventually kind of.. worked? And mkmarstime behaves properly now. I was worried the AI would just try to focus on “solution that solves testcases” instead of “solution that does actual behavior”, but the resulting diff is justified (I even had it check that the output produced was the same in its loop vs closed form intercalation).

Vibe coding, no. But vibe debugging? Well, maybe there is something to be said about it..