Benchmark

Before I started working on the bot ranking arena, I did a quick sidequest to figure out how many games per second I could simulate. So I wrote this script to execute a bunch of games sequentially, then print the elapsed time and calculate the games per second rate.

I ran the benchmark in Node 24.15.0 on my 4.40 GHz Intel i5-12450H using Bogo drivers:

Simulated 82157 turns across 1000 games
Took 109048ms (109.048s). Speed is 9.170273640965446 games per second

So that was surprising. I was expecting a speed in the low thousands, but instead it was less than 10.

I had a suspicion what was causing the slowdowns. To make sure, I swapped out the Bogo driver (which uses QuickJS) with a non-QuickJS driver that I made for testing, and reran the same benchmark:

Simulated 53854 turns across 1000 games.
Took 130ms (0.13s). Speed is 7692.307692307692 games per second

(the turn count is smaller because the drivers used a different strategy which made games end faster)

So, obviously, the slowdown is caused entirely by QuickJS. Which makes sense. Every time a QuickJS driver is invoked, it spins up a brand new QuickJS VM, configures it, executes the actual code, extracts the return value from the VM, disposes of the VM to avoid memory leaks, and then finally returns the value. Non-QuickJS drivers can just execute the code directly, and the code can be optimized very well by the JIT compiler that Node uses.

So the solution seems simple, right? Just don’t use QuickJS for the drivers. Whatever the advantages of using QuickJS are, surely they aren’t worth an 839x decrease in performance. Well, unfortunately that’s not an option. More specifically, dynamically running strings of JS (like the driver code that users write at runtime) as native code is not possible, so I have to use a JS VM. And QuickJS is by far the best embeddable JS VM that I know of. So I just have to accept the slowness.

This means that I’ll have to be clever about the bot arena, though. I can’t just brute-force a ranking with thousands of games if I can only run 10 games per second.

---

btw, if you’d like to see how your PC does on my benchmark, just run these commands:

git clone https://github.com/ethmarks/hadronize.git
cd hadronize
pnpm install
pnpm bench --count 1000 --driver prng

If your time is faster than mine, leave a comment to gloat.