I made KANVAS, a failure predictor for GPU cluster jobs built to be auditable. I wanted to show a readable model can keep up with a black box on real data. The hardest part was the data. My first telemetry join kept only 2 failures in 996 rows, because the trace records telemetry mostly for jobs that succeed, so I swapped in neighbouring jobs’ telemetry. I’m proudest that I used the model to attack itself. Its best signal was one user’s habit, and on unseen users all three models fall to chance.
