okay so this one’s been in my head for a while. every time I’m debugging something slow on linux I end up with like 3 terminals open — strace in one, perf in another, sometimes py-spy if it’s python — and I’m just manually lining up timestamps trying to figure out what actually caused the slowdown. none of these tools talk to each other. they just dump their own little slice of the truth and leave you to do the detective work.
so I built whyslow. it hooks into the kernel with eBPF and watches three things at once: scheduler switches, futex waits, and block I/O. instead of showing you three separate logs, it actually chains them together and tells you the story.
like instead of
thread stuck for 400ms (somewhere)
you get
thread A blocked on a futex
which was held by thread B
which was blocked on a disk read
that’s the whole point. not “something was slow” but “here’s the chain of custody for why.”
biggest pain building this was honestly not the eBPF part, it was figuring out how to correlate events that come from completely different sources without just guessing. had to build this whole interval-based matching thing to line up “thread went to sleep here” with “thread got woken up there” and make sure the causality actually holds up and isn’t just coincidence in timing.
kept scope small on purpose for v1. only 3 event types, no fancy stack traces yet, x86_64 + arm64 only. figured getting 3 things exactly right beats having 8 things that are kind of right and kind of not.
tested it by writing a little program that deliberately deadlocks itself with a futex and forces a disk read, so I’d have a known answer to check whyslow’s output against. felt really good when the chain it printed out actually matched what I knew was happening under the hood.
still needs root to run since eBPF just doesn’t let you do this stuff unprivileged, that’s a permanent tradeoff not a bug.
next up is probably symbol resolution so it can tell you actual function names instead of just thread ids, and getting it properly published so people can just cargo install it instead of pulling from git.
repo’s here if anyone wants to poke at it: github.com/kaorii-ako/whyslow

Shipping whyslow v1

The problem

Every Linux dev has hit “my program is slow” and reached for strace,
perf, or py-spy — but each tool sees one layer. None of them tell you
why across layers. You end up manually cross-referencing three terminal
windows and timestamps by hand.

What I built

whyslow — a CLI that watches sched_switch, futex, and block I/O events
via eBPF and stitches them into a single causal chain:

$ sudo whyslow run -- ./my-slow-program
 
14:32:07.001 — tid 4821 blocked 412ms on futex 0x7f2a3c001000
 ← woken by tid 4809
 ← tid 4809 blocked 380ms on block I/O (dev nvme0n1p2, sector 88213)

No manual correlation. It finds the root cause and walks you back to it.

Stack

• Rust, aya for pure-Rust eBPF (no libbpf/C toolchain needed)
• Ring buffer for low-overhead event collection — can’t perturb the thing
  you’re measuring
• Interval tree + happens-before edge inference for the causal graph
• Workspace: whyslow-cli, whyslow-ebpf, whyslow-common

Scope for v1

• x86_64 + aarch64 Linux, kernel 5.8+
• Three event sources: sched_switch, futex, block I/O
• No stack symbolication yet — tid/pid/comm only
• Needs root or CAP_BPF — no way around that, eBPF is privileged
  Kept scope narrow on purpose: three event sources done correctly beats
  eight done half right. Validated against a synthetic program that
  deliberately causes futex contention + a disk stall, so there’s ground
  truth to check the causal chain against, not just vibes.

Distribution

• curl -sSf .../install.sh | sh
• Homebrew tap, APT repo, pip wrapper — see DISTRIBUTION.md
• crates.io publish still pending — cargo install works via --git for now

What’s next

• Stack symbolication (native + interpreted languages)
• More event sources without losing causal-chain accuracy
• Actual crates.io publish so cargo install whyslow just works

Repo

github.com/kaorii-ako/whyslow