Over some time, I made a few improvements. The service now no longer runs as root, but as a dynamic user, which allocates a UID and GID at random. I had a few issues in the meantime, because this also automatically enables a lot of hardening functionality, preventing Deno from downloading its dependencies. I overrode the DENO_DIR environment variable and had systemd create a cache directory, and now it works.
Also, the access token is now no longer provided as a string, but inside an encrypted .env file. The dotenv file, however, is not loaded by Deno, but by systemd, which then passes it to the Deno process as usual.
Thanks to @jak2k for a lot of pointers to these systemd features!