OrbitLens Devlog #8: Scaling the 3d globe and some other stuff

In the last update, i finally implemented the 2d map so this time i want to take this website a step further since the foundation is stable enough. I decided to go from tracking 5 hardcoded satellites to over 16,000 space objects. Here is what i changed, how i changed it and the challenges i faced:

Change 1:

Originally, i just added the starlink satellites to what i already had which was my initial goal. but the thing was, i was requesting the API for thousands of satellites, which did work once and all of them were stored in localStorage but for ‘testing purposes’ (i was just stupif), i cleared the localStorage and re-requested the TLE which it then instantly rejected cus of its ‘anti-bot firewalls’ and i got ip banned for like a couple of hours and it gave a 403 error aswell.

To fix this, i read some CelesTrak documentation and i realised that there is a way to ismply request all the satellite (every satellite that it had) data in one request. Apparently there was a special search query (‘active’) that i could’ve used but instead i individually fetched everything. So once i used that query, i got all of the satellitse (a bit more than 16000) rendered on the earth but it was laggy, which leads to my second change.

Change 2

To fix this massive lag caused by the 16000+ satellites, I completely replaced the standard 3D mesh pipeline with THREE.InstancedMesh so that instead of creating 16,000 individual sphere objects (which would like instantly crashes the browser), i can feed the GPU a single geometry and material, along with an array of 16,000 matrixes. That means now the gpu like renders everything in one go.

But, I hit a massive 5-second freeze whenever a user clicked the ISS (which makes sense cus this was a 4k extremely detailed 3d model). The thing was, WebGL delays uploading heavy 3D textures to the GPU until an object is actually rendered on screen. And because the ISS model was hidden on startup, the compiler skipped it, so the website was stuck trying to render it when i selected the ISS. I solved this by forcing the renderer to draw one “dummy frame” with all models visible during the loading screen, which, in a sense, ‘warmed up’ the GPU.

Change 3:

The lag was better but it was still there and i realised the problem was that each of the dots representing a satellite were smoothly moving in space. This smooth-moving physics applied to over 16000 dots was what slowed down the website.

To fix this, I rewrote the loop that propogates the satellite dots so physics calculations wouldn’t choke the main JavaScript thread. To do this i built a system to process the satellites in batches of 4,000 satellites per frame instead of all 16,000. Second, i scrapped geodetic trigonometry (latitude/longitude conversions; see one of the earlier devlogs) and replaced it with direct Earth-Centered, Earth-Fixed (ECEF) 3D vector rotation.

Initially, our batches were too small, causing satellites to visually ‘teleport’ instead of smoothly gliding, which looked really glitchy. But by upgrading to matrix rotation, i managed to get a huge increase in speed. This allowed me to crank the batch size up and get that smooth motion across the entire thing.

Change(s) 4:

There were still some small bugs in the visual aspects like the dots disappearing or the transition jump (see previous devlog(s) to understand) and stuff but i took care of them.