rasterminal
- 38 Devlogs
- 189 Total hours
A portable and fast 3D model viewer in the terminal from scratch https://github.com/PavolUlicny/rasterminal
A portable and fast 3D model viewer in the terminal from scratch https://github.com/PavolUlicny/rasterminal
Fixed a Windows bug that crashed the program on model sizes bigger than 2GB, improved the orbit usability (fixed mouse drag behavior when models are turned upside down), and did a few cleanups and fixes in the CLI parsing.
I’m planning on adding a few CLI flags that I think can be very useful for some people, specifically better spin settings, starting camera angle/zoom control and a few more.
The branch is finally merged. I’ve accumulated a bunch of usability and code quirks and issues that i can finally get to now that the branch is merged. So now I’ll be reworking all the CLI flags, camera and other components. And we also passed the 700 commit mark with this merge!
Until now, the conversion picked the closest color possible mathematically. But from testing on a bunch of different models, I’ve realized that it doesn’t actually look very close to the human eye. So I swapped it out for a different method, called CIELAB distance. Now it still doesn’t look perfect, but as a fallback for older terminals it’s close enough.
The color terminal compatibility thing I was working on has landed in the branch, now I’m adding a –color flag to override the automatic capability detection for situations like ssh, where we almost always degrade to the 256 color pallete for maximum compatibility. So now you’re gonna be able to use the right color mode even if the automatic detection is wrong for your situation.
Some older terminals can’t display full 24bit RGB colors and use a 256 color palette instead. I’m adding a fallback so the older terminals can still use rasterminal. It still calculates the full old RGB colors, but then just maps them to the nearest 256 color with LUTs.
On a side note, I switched my terminal emulator from Kitty to Ghostty.
Did a bunch of small fixes and improvements in the last ~week. Right now I’m on vacation for the next couple of days, so I won’t be active. I am currently working on broader terminal and platform support, so older systems and terminals will be able to run cleanly with some fallbacks or will fail loud with a clear message about what isn’t supported.
Before, rasterminal built a shadow map on model load and displayed it during runtime. I now completely removed it, because it was slower during load, slower during runtime, sometimes looked broken on curved surfaces and added essentially no value.
Here, the first picture is with the shadow map and the second is without.
I did a bunch of small things alongside it, but the main feature for this devlog is the pararelization of the wireframe shading mode. It was previously fully single threaded, but it wasn’t that hard to pararelize, i just had to make the function it used to write into the framebuffer (draw_line) CAS atomic, to avoid race conditions. Now wireframe is the fastest shading mode in basically all scenarios.
Below, you can see a performance benchmark (built into rasterminal using a flag) with wireframe mode on a super dense mesh and how the wireframe mode looks like (you can also cycle the colors).
Gouraud was previously the main shading mode, but now I’ve just completely removed it. It was by far the slowest, while not adding any actual value to the project. Blinn-phong lighting is faster AND it supports more features (normal maps, etc.), so there was just no point in keeping gouraud. Now that I’ve removed it, the project has less code to maintain and a faster default shading mode with more features.
OBJ bump maps are textures for simulating bumps and wrinkles on models. Blender used to author bump maps as normal maps for some reason, so I made it automatically detect if it’s a normal map disguised as a bump map and handle it well, since there are a lot of old Blender OBJ models in the wild. But it also handles bump maps as bump maps normally, so it’s basically the best of both worlds.
Not that hard, just read it, pass it to the rasterizer and then display it. I did have a very sneaky bug in the loader that made the whole feature render wrong, where I just had to negate one value to fully fix it, but now everything fully works.
Below you can see the Khronos test asset for texture transforms. (tick means fully working, clearcoat is not yet implemented so it doesn’t show as working)
glTF sometimes authors a second TEXCOORD set (first one is TEXCOORD_0, 0 indexed numbering). The official spec only requires support for the first two TEXCOORD sets, even though there can theoretically be any number of them in a model. That said, major model exporters don’t author any more than the two sets, so to avoid extra work, I only added support for 0 and 1.
(not merged into main yet, I want to polish it a little and maybe add KHR_texture_transform support on top of it first)
After rendering the model in the transparent path, we have to resolve the transparent pixels, which basically just pastes in the pixels behind it and sometimes tints it based on how dark and what color the transparent material is (tinted glass will make the things behind it darker, normal glass will be mostly completely untinted). Before, the resolve function looped over every single pixel in the frame, even non-transparent and background ones. This is obviously wasteful. So I made it find the smallest box, in which all the transparent pixels of the whole frame fit and just looped over that. It recovered the overhead of the transparent resolve on models with smaller transparent parts, but it also made fully transparent or mostly transparent models slightly slower (it still basically loops over almost the whole screen on these types of models, so keeping track of the smallest bounding box while also still doing the same work as before made it a little slower). But the performance benefits for the average model outweighed the losses on rare fully transparent models. In the image, you can see the benchmark results versus the baseline.
Did the same template if constexpr (compile time if branches) thing that we already do for opaque/transparent models with the shading mode (was a runtime per triangle if branch before), which improved performance. It basically just makes a new copy of the function for each shading mode x each transparency mode, so 4x2 which is 8 slightly different functions and uses the correct one on each call. That increases binary size and icache pressure, but drops per triangle or per pixel if branches that hurt performance, so it’s an overall performance win. Benchmarked across a ton of different resolutions, models, shading modes, etc. (the benchmark by itself took like 2 hours to run)
Still a work in progress and I might not merge it into main, because it’s a lot of work for not that much benefit.
I have to catch a lot of bad_allocs and things like that for super large SSAA sizes, which is really annoying and can affect performance.
SSAA works by rendering at a higher resolution and then downscaling for better edges (makes edges smooth and fade-in instead of jagged)
Picture with 2x SSAA and without SSAA:
After two unsuccessful and unmerged branches, I finally did something useful on main.
I used to deduplicate the vertices in stl_reader and then reduplicate them in the loader. But now, we just dedup in stl_reader and use those. That also allowed me to enable ambient occlusion, which was disabled for stl before. And now crease smoothing fully works too. And it’s faster (runtime wise, load time is slower because of ambient occlusion and crease smoothing that weren’t there before).
So an all-round very positive change, and also an end to my non-merging dry spell.
But after actually replacing it and integrating it into the source code, i realized it’s not worth it, because rapidobj just has too many downsides and isn’t very actively supported anymore.
For example, it doesn’t support multiple mtl files, which is against the spec. So if i want to keep it spec compliant, I need to find a workaround for it, which just isn’t worth the speed increase of rapidobj, even if it’s like 4x faster (from my testing on a large obj)
The graph is from this article: https://aras-p.info/blog/2022/05/14/comparing-obj-parse-libraries/
Transparency works correctly, but transmission, which is completely different, is simplified to transparency. So for spec-complete gltf, I would need to implement the whole transmission pipeline, which I’m way too lazy to do right now. So for now, we just simplify it to transparency and call it a day.
For that, I have to completely rework the rendering loop and the rasterizers, so basically the whole core of the project. It’s going very badly.
And since performance is a big factor here, I’m going for a template approach so the transparency overhead doesn’t affect non-transparent meshes.
I did a hot-loop performance cleanup run, because the hot loops were getting slower with new features, and I found a big optimization opportunity in clip-near call sites:
clip_near slices off the part of a triangle that pokes behind the camera’s near plane, but almost every triangle is fully in front of it with nothing to clip. The pipeline still called it for all of them, and in the compiled binary that common case just copied all three vertices to memory and back, millions of times a frame. The fix was to check inline whether the triangle is fully in front and skip the call entirely if so. Result on a 2.77M-triangle model: median frame time dropped from 21.1 ms to 19.2 ms, about 8.7% faster.
Because of all the libraries I added, the binary size increased significantly.
I don’t use most of the vendored code, so I added build flags to ignore unused code in the final binary.
Other than that, I added some other performance flags and also a CMakePresets.txt.
It took longer than it should’ve, because I tested and benchmarked a bunch of other configs and flags.
I put off PGO for now.
I wired in Webp image format support by vendoring in google’s libwebp decoding library.
It wasn’t that hard to wire into the source code, the hard part was the vendoring, dealing with licenses and professional vendoring practices.
Only thing that remains to add for full gltf extension support is avif image support, but I’ll do that later.
I just added unlit material support, which means materials that are supposed to stay intentionally unaffected by lighting render correctly.
Pretty straightforward change, just read it at load time and then check if that material is unlit during the lighting pass and skip it.
Reworked the whole readme with better instructions, screenshots and a gif.
The gif was super annoying to polish, because i used asciinema which records all the setup escape sequences so there were gaps at the start and end that i had to manually read and remove.
And I also had to perfectly close it exactly when it did one full rotation and that took like an hour to perfect.
At least now I don’t have to touch the readme for a while.
Well, I tried to refactor the main rendering loop for like the third time, because it’s horrifyingly indented, but it has defeated me once again.
At this point, I’m confident that it’s impossible to do without a noticeable drop in performance, or g++ is just ragebaiting me.
I installed a rainbow indentation neovim plugin just to show the absurdity of this.
I just implemented KTX2 and Basis texture support, which are special compressed texture formats.
So now any models with these textures are viewable in rasterminal.
rasterminal is a full software 3D rasterizer and model viewer that runs entirely in your terminal. The whole rendering pipeline is implemented from scratch. I only rely on third-party libraries for load-time things like 3D model parsing, image parsing, and decompression.
It doesn’t touch the GPU at all, running purely on the CPU. That makes it completely safe for SSH usage, though it does come with a performance cost. I use multithreading to recover some of what’s lost, and the result is still significantly faster than any other terminal 3D viewer I’ve come across.
Right now I’m focused on broader 3D format feature support and transparent material rendering. If you want to try it out, I’d love to hear what you think, feedback and issues are very welcome over on GitHub.
https://github.com/PavolUlicny/rasterminal
Thanks for any support! 🙏