DEVLOG 14 - I Rewrote The Memory Aliasing Algorithm

 
Intro
Unfortunately, I don’t have anything exciting to say. I’m forced to do this devlog since I’ve reached ten hours since my last devlog, so I have to publish one, otherwise my hours wouldn’t count in full. I’ll try to share what I did, the problems I encountered, and how I solved them.
 
The Problem
I might sound crazy, but after working on the Render Graph for twenty hours, I rewrote it entirely from scratch. You might ask, why? The reason I did this was Vulkan itself. I discovered that, since I’d previously relied on VMA (Vulkan Memory Allocator) to manage memory, now that I was managing memory without it, there was something I’d overlooked: a parameter called Memory Type Index. The Memory Type Index indicates where to allocate memory. There could be a Memory Type Index for the GPU’s local memory, one for the Resizable BAR, etc. So, what’s the problem if all the Render Graph resources must live exclusively in the GPU’s local memory anyway? Wouldn’t it be enough to simply add a few lines of code and everything would be solved? Unfortunately, not because this index indicates in which memory to allocate that resource, but there’s also another parameter called Memory Type Bits, which is a bitmask that indicates in which Memory Type Indexes a resource can or cannot be allocated. All resources are allocated in the same Memory Type Index, but they may have different Memory Type Bits. This means, for example, that a texture without multi-samples cannot fit in the same memory space as a texture with multi-samples. This can also happen with the aspect mask, so a color texture and a depth texture might not fit in the same place. It could also happen with the format or tiling. The same goes for buffers, but these are more flexible than textures.
 
The Solution
My Memory Aliasing algorithm was designed without taking all this into account, so I chose to rewrite a large part of the Render Graph, trying to recycle some parts. Therefore, the Aliasing algorithm now operates on “Buckets,” which are in-memory containers containing resources that can fit in the same space. Without this distinction, memory aliasing wouldn’t be possible because a resource attempting to reuse memory used by previous resources must only check among resources with the same requirements, not among all resources. Therefore, it checks per bucket, not all existing resources. This has also made the code much cleaner and more organized.

DEVLOG 13 - Memory Aliasing

 
Recap
In the last devlog, I explained what I did and why I was creating a Render Graph. I mentioned a concept called Memory Aliasing, but I explained it briefly. I’ve been working extensively on the Render Graph these past few hours to implement the Memory Aliasing algorithm.
 
What is Memory Aliasing?
Memory Aliasing involves having two or more resources use the same portions of memory, thus sharing that memory. For example, let’s take a texture we’ll call A. This texture is used as Depth in pass 1 and read in pass 2. Another texture, called B, is used in pass 3 and pass 4 (see photo). We could allocate both resources independently, but you can clearly see that the resources don’t overlap in the passes. Texture A begins its usage cycle in pass 1 and ends in pass 2, and immediately after, in pass 3, texture B is used for the first time. What we could do is have both textures occupy the same memory space to save VRAM. This is the concept of Memory Aliasing. Obviously, since the resources share the same memory space, barriers must be created to synchronize resource access, otherwise there would be a race condition (two threads operating in parallel on the same memory space, not just for read-only operations).
 
Resource Pooling
In addition to memory aliasing, I wanted to optimize another aspect that took me some time. In Vulkan, when you want to create resources, you use vkCreateImage and vkCreateBuffer. These functions create a handle without associating physical memory with these objects. Binding occurs later, after executing the memory aliasing algorithm, which determines which memory space to assign to a resource.
This means that I have to create and destroy resource handles every frame, which apparently isn’t a light operation to perform on multiple resources. So I thought that if there was a possibility, I could reuse handles from previous frames by simply comparing whether the settings of the requested handle were the same as the one to be reused and then binding the memory.
 
The Problem and the Solution
This was the idea I had, but unfortunately it’s not feasible because Vulkan doesn’t allow memory binding to a resource more than once, so this approach wouldn’t have worked. After thinking for a while about how to solve this, I decided to include in the requirements for using old handles, in addition to the texture or buffer settings, the offset and stride of how they were bound in memory. This way, if the pass structure and usage hadn’t changed significantly during the user’s setup phase, the Memory Aliasing algorithm would always produce the same results, given the deterministic nature, making this all feasible.
 
Finally, if the resources aren’t reused for an N number of consecutive frames, they are permanently destroyed.

DEVLOG 12 - Render Graph, Code Refactoring

 
A Little Recap
In the last devlog, I mentioned I watched the 2017 GDC conference, where they explained how the Frostbite development team had created a Render Graph. But what is a Render Graph, and what is it used for? To answer this question, we must first understand how the GPU works.
 
The Difference: CPU & GPU
First, a GPU, unlike a CPU, performs all its operations in parallel. When we write code for the CPU, we say: do this operation, then this, then that, and so on. The CPU performs its operations synchronously, in sequential order. The GPU, however, doesn’t work that way. If we tell the GPU that we want to draw three objects, we can send it three draw calls, which we’ll call A, B, and C. These draw calls are sent in order, so first A, then B, and then C. The order in which the commands are sent doesn’t correspond to the order in which they are executed, because the GPU could execute the commands in this order: C, B, A, or B, A, C, or A, C, B, and so on. This is because, while operation A might be in progress, operation B might start executing, so both commands will use the same resources and therefore run into a race condition if both operations are not read-only.
 
The Problem
From this perspective, the GPU is very powerful because it tries to stay constantly busy, eliminating dead time. However, this greatly complicates things on the synchronization side. Let’s take two operations: operation A that writes an image and operation B that reads the image. How can we be sure that the GPU executes operation A first and not B? Who can assure us that the GPU, while executing A, doesn’t have to execute B? The answer to all these questions is no one.
 
How Graphics APIs Handle All Of This
In graphics APIs like OpenGL, all of this is hidden from the programmer, so they don’t have to worry about it. But in graphics APIs like Vulkan and DirectX 12, this is the programmer’s responsibility. The programmer must write a barrier and tell the GPU not to touch the image from operation B before A has finished. For a few barriers, this is feasible, but as soon as you write more complex applications, making the code sufficiently organized, non-repetitive, and fast to develop becomes very challenging. This is why the concept of a Render Graph exists.
 
What A Render Graph Does
A Render Graph allows the programmer to specify which steps the GPU should execute, which operations resources like images and textures will perform, and in what order. The Frame Graph looks at this graph constructed by the programmer at each frame and internally understands where barriers and sometimes semaphores are needed (if we’re working with multiple queues, that’s not the case for me). Another responsibility of the Frame Graph is to internally allocate and manage resources that only exist within a single frame, called transient resources. Finally, I’m implementing Memory Aliasing. This reduces the memory used by resources: for example, if resource A is used for the last time in pass 3 and resource B for the first time in pass 4, they can share the same physical memory, thus reducing the overall memory requirement.
 
Bonus
I’ve also restructured the texture creation methods, making them simpler and allowing for fewer specifications. The before and after is a good result… The same goes for 1D and 3D textures.

DEVLOG 11 - BDA, Pipeline, Shaders

 
In the last devlog, I explained how I wanted to structure my bindless descriptor set. With that approach, I had a buffer for each data type. This approach works, but it’s a bit cumbersome, so I switched to using Buffer Device Addresses (BDAs). The idea behind it is: Create buffers, around VRAM, placing the buffers within VRAM memory pools. Instead of saving the physical data inside the descriptor buffer, we can only save pointers to the buffers, so I’ll have a buffer of pointers to the buffers scattered across VRAM. Since the pointers take up 8 bytes, managing the buffer is very simple, as it will be very light. So, essentially, I separate the physical data of the buffers from the shader binding logic.
 
Rather than using namespaces to hide the internal implementation, I chose to use Hadles, which are simply numbers that identify that resource. So I could have a BufferHandle, ImageHandle, SamplerHandle, PipelineHandle, and so on.
 
I chose not to call the structure that manages all this with names that might be reminiscent of the Vulkan API, but instead called this system Resource Mapper. At the moment, I think I’ve done something wrong with the code: Since the buffers are simply exposed via a 32-bit index, internally the buffer index corresponds to the index of the vector in which the Vulkan buffer handles and all the allocation information are allocated. I think I’ve erred in saving the buffers in vector indices that don’t match their exposed handle index outside of Eve. This causes access to incorrect buffers or buffers that don’t even exist, causing UBs or crashes. I still need to verify this issue and, if it occurs, I’ll have to fix it.
 
Furthermore, yesterday I added all the code responsible for creating the graphics pipeline and compiling the shaders. This code is still a bit rough and needs improvement.
 
Furthermore, since I don’t want the user to have to manage synchronization outside of the engine, I’ve been thinking about how to structure a Render Graph. I saw Frostbite’s presentation at GDC 2017 where they explained how their Render Graph works and that gave me some insights.

DEVLOG 10 - Working On The Vulkan Wrapper

 
For now, I’m focused on creating a wrapper for Vulkan to make it simpler and less repetitive to use. I don’t want to create a wrapper that abstracts too much, thus losing control; instead, I’m creating a fairly thin wrapper.
 
I’m currently restructured the Application class a bit and divided some of its parts into three “modules.” I’m structuring this using the Factory Pattern. I’ve created three Builders: one for the window and SDL initialization, one for creating a Vulkan context—VulkanInstace, PhysicalDevice, LogicalDevice, and the graphics queue—and the last one for building the swapchain. I chose this approach because I consider it flexible and I don’t want everything to fit into one huge monolith; I prefer having multiple modular classes.
 
In addition to these more or less complete builders, I’m focusing on creating a system that can allow shaders to see and manipulate the data stored in VRAM. This is usually done with DescriptorSets, which have been the standard for years. They’re more or less like pointers to objects in VRAM so shaders can access them. In reality, they’re fat pointers because they contain other information besides the address. However, this approach is good but not optimal because it introduces CPU overhead. Since the draw calls may not share the same materials and therefore require different textures, a new DescriptorSet must be bound before each draw call, which, with many draw calls, incurs CPU overhead.
 
What I’m doing to solve this problem is looking towards a more modern and efficient approach called Bindless. Basically, instead of having multiple small DescriptorSets, you have one huge global DescriptorSet that contains all the resources used by the application. So we could have a sort of array containing all the samplers, then another containing all the textures, another containing all the buffers of a specific type, etc.
Then we just need to bind this DescriptorSet at the start of rendering, which will be used for the entire rendering process, and pass indices to the shader that will tell it which resources to use from the DescriptorSet.
 
I’m a little slow in developing all this because I’ve never used Vulkan; in fact, I’m still learning. The second reason is that Vulkan is very complex and there’s a lot of boilerplate code.
 
I still have to finish the ECS system since I don’t yet have a centralized approach for obtaining the IDs and GenerationIds of new entities, which are verified and therefore safely managed internally by the engine. But I think I’ll do it later since I’m a little tired of fiddling with entities and want to do something different for a while.

DEVLOG 9 - 100K Entities

 
After the last devlog, in which I rendered 1K entities, I tried increasing the number of entities to a 15x15x15 cube, or 3375 entities, and I saw the frame rate drop unexpectedly, 25 FPS. I was wondering if this was normal or not… Apparently, it wasn’t… I tried to figure out the problem, but it wasn’t easy.
 
I tried using a tool called RenderDoc that lets you see what’s happening on the GPU, so all the calls, data passed to the GPU, buffers, textures, etc. That information also included the time, and the GPU indicated it was taking about 16 ms, which was quite strange.
 
I started to suspect Vulkan’s validation layers. By disabling them, I got to 30 FPS or a little higher, I don’t remember exactly. Then I disabled ASan and set the swapchain to Immediate Mode so as not to be constrained by the screen refresh rate, and I got to about 100 FPS. Then I made several CPU-level optimizations at the code level by changing the way components are obtained from batches in my ECS system, specifically limiting those operations that can be avoided in the internal component iteration loop, and I got up to 140 fps. Finally, the thing that made me open my eyes and say WOW was compiling in Release and not Debug, the fps skyrocketed to around 2300… I couldn’t believe it either…
 
After doing all this, I rendered 100,000 entities and my fps were stable at 90… Incredible! Finally, just for the sake of curiosity, I tested 1,000,000 entities and got 9 fps, which is good because it means the system scales linearly.
 
This is what I’ve been doing for the last two hours. Before that, I spent some time revisiting my ECS code to make it more secure. For example, many of the Table and EntityManager methods now return references and not pointers. I’ve also made the ComponentRegistry class much more efficient by storing component information in contiguous arrays instead of lists, which therefore contain non-contiguous data.
 
Next Steps:
I need to implement a way to get verified IDs from the EntityManager, since currently the IDs are only manual and non-unique. And then I’ll need to refactor the rendering code again, making it more modular, since it’s currently all in a single class.

DEVLOG 8 - 1K Entities

 
I implemented several things:

• One of these is a manager that allows systems to register their functions so they can be called, for example, when Update, Start, LateUpdate, etc.
   
• I implemented Address Sanitizer in the project to detect leaks, overflows, or out-of-bounds reads, thus uncovering some bugs. I didn’t understand why, but ASan was having trouble integrating with the project; apparently, the shaderc library was causing problems because it had a different connotation than the project. I finally solved the problem by telling CMake to use the Release version of shaderc, not Debug.
   
• I created a test system called S_Test, which I used to verify that I could create and manage multiple entities (apparently, yes). After having several memory issues, which I resolved with the help of ASan, I also started creating a camera component so I could move freely in 3D space.
   
• I’ve improved the security of the code; Entity Command Pools now no longer need to be created by the programmer, but by the Entity Manager itself, which manages them. This minimizes the risk of freeing memory too early, resulting in cleaner code for the game. Furthermore, the EntityCommandInfo and QueryInfo classes can now be allocated on the stack rather than the heap, resulting in a more secure program.
   
  Finally, for fun, I wanted to try creating a thousand entities, also to see if the system was capable of it and if there were any problems. I have to say, I’m quite happy with the result.
   
  Next Step:
  I still have to improve several parts of the code to make it more secure and faster, such as the Component Registry.

DEVLOG 7 - Code Reorganization

 
I tried to reorganize the project so that the headers needed by users of the engine are visible, while the internal, and therefore unnecessary, ones are not.
 
What do I mean?
Let’s say we have a class called A. This class is used by the engine itself, but it can also be used by the developer, for example, to record commands or something similar.
So the developer will include class A in their header or source files so it can be used. But there’s a problem: Header A could include other headers that aren’t necessarily needed by the developer using the engine, thus polluting the headers.
 
How do I solve this problem?
Since I haven’t used C++ for years, I’ve been researching the problem online for months and found the following solutions:
 

• Forward Declaration - This consists of declaring the required class in the header, in this case A, without using the include directive, which literally copies the headers included in the file containing the directive. This seems like a good method, but unfortunately the data can’t be variable by value, only by reference or pointer, since the compiler must know at compile time how much memory that specific object occupies.
   
• Pimpl Idiom - This involves enclosing or encapsulating the information needed for the internal functioning of A in a class or struct that is hidden from the developer using the engine. This method could work and is almost standard, but personally, I don’t really like the idea of ​​having to create another structure to achieve this goal. It’s more of a workaround, since the language has significant limitations in this regard.
   
• C++20 Modules - Modules are a new way of programming in C++ because, among other things, they solve this problem. They also reduce compilation times and avoid separating the code into two separate files (.hpp and .cpp). I tried using them, and from a compiler perspective, they work great (I’m using Clang), but from a Clangd perspective, the situation is quite different:
  While the compiler support is fairly mature, Clangd still has several issues; in fact, it reported errors everywhere. I even tried modifying the .clangd file by adding various flags, but the situation didn’t improve.
   
• There are other methods, such as Fast Pimpl, but they aren’t as flexible.
   
  What I’m looking for is something flexible that generally works in all conditions and doesn’t introduce overhead, which Pimpl Idiom doesn’t.
   
  The final solution was to use namespaces.
  All the public headers that must be used by the developer using the engine are in the Eve/ namespace, while the internal headers that shouldn’t be used by the developer are in a namespace called Eve/Internal. If the developer uses these files, it’s their fault; they’re marked as internal to the engine and therefore shouldn’t be used.
   
  I also changed build system and switched to CMake as it is more widespread in the C++ community and this can also give an advantage in the engine configuration section for other users

DEVLOG 6 - Crash, Crash, Crash and…. Crash

 
What I did
After spending several hours designing my ECS architecture, I realized I could do better, so I basically started from scratch and, as I mentioned in the previous devlog, I chose to use Archtypes so everything would be extremely efficient. Maybe I’m a bit too obsessed with performance…
 
The Problem
Anyway, after finishing the entire system, I started the app and it crashed. After several hours of debugging with simple print(); I thought the problem was in the memory, but I couldn’t figure out where… I finally decided to use Address Sanitizer, a tool that alerts you to out-of-bounds writes or reads, which allowed me to pinpoint the problem. I had performed a nonsensical multiplication, so the memory pointer was pointing to the wrong place, specifically to the end of the allocated memory portion, causing the metadata to be overwritten. (Apparently, when allocating memory, the program itself allocates additional memory slices that store information about the allocated memory portion.) The problem was that the app didn’t always crash, only sometimes, and I didn’t understand why. But I finally managed to get it working…
 
The Result
At the moment, this is a single entity, and since I haven’t tested the rest of the code, such as the destruction logic and entity transitions from one archtype to another, there are likely other bugs waiting for me.

DEVLOG 5 - The ECS Architecture
 
Since I can now draw something on the screen, I’d say it’s time to start building the actual graphics engine. The first system I need to create for Eve is the management of game objects. There are two options: the first is to use the OOP programming paradigm and therefore manage game objects just like Unity does; the other, which is the one I chose, is to use the ECS (Entity, Component, System) architecture.
 
To understand why I chose ECS, you need to understand how this architecture works. As I mentioned, the word ECS is an acronym for Entity, Component, and System, which represent the three pillars on which this architecture is based. In short, entities represent objects, the essence of the object; that object simply exists, and they are usually represented by a simple numeric ID. Components, on the other hand, represent pure data; therefore, each entity can have its own components and therefore its own data. Systems, on the other hand, are the part of the code that executes the logic. Let’s take the rendering system as an example. This will iterate over all entities that have mesh components, and a transform will take their data and send it to the GPU for rendering.
 
This division of responsibilities ensures incredible efficiency, given how the CPU works, since the component data is stored contiguously in memory. This facilitates the work of a CPU component called the Hardware Prefetcher, which tries to predict which portion of RAM the CPU will use next, thus reducing cache misses.
 
There are many ready-to-use libraries for this architecture, such as Flecs, EnTT, and others. I chose to try and create my own so I could learn and understand more. Currently, I’m using, broadly speaking, the same approach Unity uses with their ECS architecture, which is part of their Unity DOTS ecosystem.
 
All entities in the game can be classified based on an ID called an Archtype, which identifies the type of entity being viewed based on its components. This way, you can have contiguous arrays and minimize cache misses, thus maximizing CPU efficiency.
 
I’m still working on it, but I have to say it’s extremely interesting…

DEVLOG 4 - Textures!
 
I finally managed to use textures; Vulkan has a bit of boilerplate in their configuration, especially with the descriptor sets. This was the part that slowed me down a bit.
 
The code isn’t very tidy at the moment… For now, I’ve focused more on learning, so I haven’t written anything extremely tidy and readable. As I said, I’ve only learned the basics so far.
 
Now that I feel more confident about the subject, I think I’ll start writing something. I’m thinking of a framework or something simple that allows you to draw geometry on the screen without having to write hundreds of lines of code directly with Vulkan. So, more generally, I think I’ll create a layer of abstraction while still trying to maintain excellent flexibility.

DEVLOG 3 - First Cube!!!
 
How satisfying, my first 3D object!

Since previously all vertices were written manually in the shader, I opted to learn how to create buffers to load more complex 3D geometry onto the screen.
 
I’m loading the resources directly into a specific portion of memory called Host Visible, where the CPU can also write, and the GPU’s reading isn’t as fast as it could be. If I instead allocate the memory in a portion of VRAM called Device Local, the GPU will be very fast at accessing the data, but the CPU, on the other hand, won’t be able to do anything with that memory. To modify it, you need to first allocate the data in an intermediate buffer called Staging Buffer, and then from there the data can go to Device Local memory (the GPU’s fast memory).
 
I also added some transformations and rotations with GLM to rotate the cube, and this is the result, which is nice.
 
I think I’ll try using Device Local memory by loading data to it, as discussed before, with a Staging Buffer and after that I think I’ll start experimenting with textures.

DEVLOG 2 - Consolidation of things learned
 
Well,
What’s new? Just a square? It might not seem like much, but yes. I spent all these hours rewriting the entire code from scratch so I could understand where I had the most doubts and therefore review some concepts.
 
Synchronization was and still is the most difficult concept to grasp; understanding how the semaphores and barriers are linked together wasn’t easy for me, and I must admit I still have some doubts about it, but certainly much less than before. Given the asynchronous nature of the graphics card, you can’t program operations that will be executed in order, unlike the CPU, and this is one of the graphics card’s strengths and weaknesses. You can parallelize many operations, but you have to be careful how they are linked together and synchronize them as correctly as possible.
 
My next step, I think, will be to use buffers to give more complex geometry to the shaders, primarily for drawing 3D geometry. I also think I still have to delve deeper and better understand things like semaphores, timeline semaphores, and barriers, which are topics I feel I haven’t internalized as well as others.

DEVLOG 1 - Hello Triangle!!!

 
Finally,
after several hours trying to understand how Vulkan works, watching and reading several tutorials, I got it, my first triangle, isn’t it cute?🫠🫠
 
I’ve figured out most of the things, but I think I need to delve a little deeper into the process of creating a pipeline with dynamic rendering and synchronization. These are the two topics that have been giving me the most trouble.
 
I think the next step will be to re-explore the code and try to create something on my own, without a guided tutorial, to see if I’ve really understood the concepts.