Previews

I have a really cool new feature (actually it is really old, the first version was already working on the 26. 6. (two months ago), but I kinda forgot to write about it):

Previews

The problem is pretty simple: Imagine someone is adding a new record like super-cool-site.hackclub.com.

The record is not merged yet, so the DNS record does not exist. Because of that you normally can’t just open the domain in your browser to check if the site is working.

So how do I solve this?

Well, to understand this you need to understand how the hostname actually works.
When you open a site in your browser like hackclub.com, it resolves to an IP address, in this case 216.150.1.1 (owned by Vercel).
But on Vercel many sites are hosted on the same IP, so how does Vercel know which site to serve.

The Host Header

The Host Header is a HTTP header which is automatically added by the browser and contains the original hostname that was requested in this case hackclub.com

So the server can look at this header and then serve the correct page.

With HTTPS it is a little bit more complicated, because the hostname is needed before the HTTP request even exists. This is where SNI comes in. I did some tricky and hacky stuff to make it work.

How the preview works

When a new proposal is detected, the Slack message now has a link next to the name of the proposal:

https://dns-tracker.fantamomo.com/preview/<host>

When the link is opened, my server looks in the database for the target IP or CNAME.
Then it makes a request to the target, but instead of using the target hostname as the Host header, it uses the hostname from the proposal.
So even tho super-cool-site.hackclub.com does not exist in DNS yet, the upstream server gets a request which looks like it came for super-cool-site.hackclub.com.
Nearly all headers and the full body are passed through to the upstream server.

When the response comes back, I check what it is.
If it is a redirect and it targets the upstream server, I rewrite it so that it goes through the preview server.
If it is anything besides HTML, I just pass it through.

But if it is HTML, I parse the HTML and do two things:
First I rewrite all links, relative or absolute, so that they point to the preview server instead of the real domain. I also inject the /preview/<host> path so my server knows what it is supposed to preview.
Second I inject a small JS script which overrides the fetch function.

This is needed because otherwise the website could load correctly, but JavaScript could still make requests directly to the real domain (which does not exist).

The script is here if you are interested: proxied_fetch.js

There is one problem

While this allows the client to basically pretend that the DNS record already exists, it doesn’t work with every hosting provider.

The problem is that the hosting provider itself may expect the domain to be configured before it serves the website.

My server can make the requests like if the DNS record exists, but I can’t make the hosting provider believe that the domain really exists.

More problems

Another problem I have is that the program uses to much RAM, so much that it gets all the time killed by the Out-of-Memory-Killer.
So I am going to investigate what exactly takes so much RAM.
If I can’t find out or it actually needs that much, then I am going to request more RAM for my nest server.

So until then, it is possible that the server gets killed, or change detection gets delayed. If you really need it, I recommend you to just wait 1-2 days until I fixed that, or ping me in Slack so that I can restart the server.