The pipeline is up!
I built a research pipeline to evaluate whether ML model feature importance is trustworthy given a small, imbalanced medical dataset. What’s exciting is that it uses the relatively new 2026 PhysioNet Challenge sleep dataset, so it’s new stuff!
It was a huge pain getting the pipeline to work reliably. I had to reorganize the data, which kept messing stuff up, and it was pretty slow and uneventful at times. I had to deal with file mismatches, adapt to missing files, and figure out how to handle licensing between my code and the original Challenge code. However, the real challenge was validating results and catching a mistake along the way. Initially, I thought a feature like arousal frequency was the strongest predictor after two separate runs. At the time, it seemed promising, but I was unsure. When I tested it across 20 random seeds, it completely died. I’m pretty glad I kept digging deeper and running precautionary tests; I’m proud of realizing that mistake, which is a crucial skill in real research.
Anyways, if you want to try it, there’s no clickable button. Since it’s analysis code, you’ll have to clone the repo, follow the setup in the README, and run the scripts yourself. You should get the same numbers I did. If not, please let me know. Oh, and please let me know what you think, or if I made a mistake (you can never be too careful)!
