Zinn
- 6 Devlogs
- 21 Total hours
A high-performance, perceptual media-to-ASCII engine written in pure C23.
A high-performance, perceptual media-to-ASCII engine written in pure C23.
18 changed files with 1153 additions and 547 deletions
I wasn’t happy with the old error handling. A flat enum doesn’t tell you what went wrong or how bad it is. So I tore it out and replaced it. I’m proud of this module because it uses modern C23 features everywhere you look, is fast, secure, and gives me the important debugging info in case something goes wrong.
This commit contains the initial implementation of the module, although with a few bugs I addressed in one of the two later commits.
Warning! This commit DOES NOT compile. I haven’t refactored the codebase to use the new error handling implemented in this commit.
The new zinn_err_t is a uint32_t with three fields packed into it:
[8 bits domain | 4 bits severity | 20 bits code]
A thread_local fixed-depth LIFO stack (max 16 frames, 128 message chars each). Every frame records file, function, line, the packed error code, a human-readable message, and a unix timestamp.
Pushing a frame manually is tedious, so I wrote macros that inject them for you.
Higher-level macros build on this:
ZINN_RETURN_IF(condition, error): if true, push a frame and return the errorZINN_RETURN_IF_ERR(expression): if the expression fails, push and returnZINN_TRY(expression): same, but evaluates to the expression’s value on successZINN_CHECK_ERROR(call, message, ...): push a frame if error, but keep goingzinn_diagnostics_dump_frames() dumps the whole stack to stderr with colored formatting, newest first.
Check the attachment to see the stack dump in a terminal. Contains all the info you’d ever need.
C doesn’t have RAII. But GCC has __attribute__((cleanup)). The scope guard module wraps it:
ZINN_DEFER_CLEANUP(function, pointer)
This declares a local guard that calls function(pointer) when the scope exits, whether by normal return or early return. You can disarm it with ZINN_SCOPE_GUARD_DISARM(name) if you transfer ownership.
The logger now uses this for mutex cleanup.
The old code scattered manual pthread_mutex_unlock() calls across every return path. One missed unlock would deadlock the whole process.
The codebase was refactored to use the new error handling and diagnostic functions, macros and types.
A few examples:
zinn_err_t everywhere; precondition checks use ZINN_RETURN_IF; mutex handling uses scope guardszinn_arena_create changed from returning a pointer to returning an error code with an output parameter. Every error site now pushes a diagnostic frame with detailsfclose() calls that leaked file descriptors and a renamed internal function amongst a few othersThe test count also dropped from 50 to 49 Logger tests because I removed a redundant test.
With the error handler now functional, it will be finally time for the (probably) final core mod-… oh wait. Nevermind that. Next I have to write a ton of unit tests for this new module. Lucky me!
8 changed files with 1638 additions and 44 deletions
This devlog covers another core module, the Arena Allocator, as well as improvements to the test framework. Implementing this took quite some time, due to the fact that I want to make sure the codebase stays secure and fast.
The next core module I implemented is a linear arena allocator, the polar opposite of a general-purpose malloc. The whole purpose is simple: no free individual allocations. Just bump a pointer and reset the whole thing when you’re done. It’s also quite fast.
Every pointer returned by the arena is aligned to max_align_t (typically 16 bytes on x86-64).
When the CPU loads an 8-byte value from an aligned address, the memory controller fetches a single 64-byte cache line in one transaction. The bytes land in the CPU’s load buffer in order, and execution continues the next cycle.
When the same load is unaligned (say, address 0x1003 instead of 0x1000), the 8 bytes straddle two cache lines: 0x1000 and 0x1040. The memory controller now issues two separate cache line fills, the load buffer stitches the result together from the two halves, and the pipeline stalls waiting for the second request to arrive. On modern x86-64 this penalty is roughly 2-3x the latency of an aligned load.
For an allocator focused on speed, every allocation must be aligned. The arena guarantees this in two branchless instructions:
uintptr_t raw = (uintptr_t)arena->memory + arena->offset;
uintptr_t aligned = (raw + 15) & ~15;
The snapshot API (zinn_arena_snapshot_begin / zinn_arena_snapshot_end) lets you checkpoint the arena offset and roll back to it later. In debug builds the rolled-back region is poisoned with 0xAA so dangling reads are immediately obvious.
47 thorough tests covering just about everything about the arena.
Every test function’s runtime is now measured. The runtime is printed next to the PASS/FAIL label with microsecond precision.
Check out the second attachment.
This commit also fixes a subtle C23 compatibility issue: nullptr has type nullptr_t under C23, which can’t be directly cast to uintptr_t. I added a _Generic dispatching macro (ZINN_TO_PTR) that maps nullptr_t to (void*)0 while leaving everything else untouched.
The failure messages also got an upgrade: actual and expected values are now printed numerically instead of a bare expression string.
The original test runner supported one suite at a time. You’d call zinn_test_summary() and get a single block. Now the architecture is multi-suite.
The final output is a detailed breakdown:
Check out the first attachment if you are curious how it looks.
With memory management settled and testing infrastructure solid, the next milestone is an error handler.
8 changed files with 1264 additions and 30 deletions
The next priority wasn’t to build more features, but to ensure that the ones I have are unbreakable. Thus, I’ve implemented a custom, zero-dependency Unit Testing Framework and a comprehensive suite of 50 tests for the Logger module.
Because of my ego in the way, I didn’t want to rely on heavy external libraries like GoogleTest or Check (or any external library, for that matter).
The result?
_zinn_log_reset_internal) to Logger that allows the test suite to scrub the global state of the Logger between runs. This ensures that every test starts with a clean state, which prevents state-leakage bugs.EXPECT and ASSERT checks that capture the exact file and line number of any failure.I also updated the Makefile to add a
testtarget, which compiles all source files EXCLUDINGmain.c(as I have a customtest_main.c), test files, and runs the tests.
Instead of being lazy, I decided to write a test for everything imaginable for the Logger. This may seem pointless, even redundant at times, but as I say, better be safe than sorry.
Proving that Init/Shutdown/Reset cycles don’t leak memory or leave the mutex in a deadlocked state. I verified that the Logger returns to a “Zero State” after shutdown, allowing it to be safely re-initialized later.
This was the most critical part. I wrote a “ruler” test to verify exactly what happens when the 1024-character message buffer is exceeded. By analyzing the memory buffer line-by-line, I proved that the logic correctly truncates and appends ... at the exact 1023rd byte without overflowing the stack.
Every public function was tested with nullptr arguments.
Where applicable, of course.
Using fmemopen() to capture the logger’s output in memory, I verified the exact formatting of timestamps, ANSI colors, and source-location metadata.
Ajajaj, yet another POSIX-only function. Surely this won’t backfire…
Now that the Test Suite is done, and Logger unit tests are written, it’s finally time to tackle yet another core component, this time it being a custom (minimalist) memory allocator.
7 changed files with 89 additions and 16 deletions
This is a very short devlog, as it was a very short integration. I simply added a Doxyfile, which controls how the documentation will be generated. I set it according to my project, e.g. a C project, the project name, what directories it should traverse etc.
To make the documentation generation and opening as seamless as possible, I added a docs target to the Makefile, which checks if you have doxygen installed. If not, it throws a simple error message and doesn’t proceed. Once it’s verified that doxygen is available, it generates the documentation and opens it in your default browser via xdg-open.
While I was going through the codebase for a final check of the comments, I noticed a few functions didn’t have any Doxygen comments whatsoever (especially the static ones) and a few things were incorrectly documented. I added and fixed comments for clean doxygen documentation.
In the attachment you can see the generated HTML documentation for a few functions located in logger.c. Notice the @note being displayed neatly with a yellow background on the website.
6 changed files with 594 additions and 9 deletions
The first core module I implemented is the Logger. I feel like I spent a huge amount of time on this, but it was well worth it, as it’s a marvel. Below you can read about the engineering of the logger.
This is a quick overview, for the full context you can check out the source code.
The logger uses a two-tier filtering system.
__VA_OPT__, the logger identifies the build type. In a release build, TRACE and DEBUG calls are completely stripped from the binary. They don’t just “not print”; they don’t exist in the final machine code.While zinn doesn’t use multithreading yet, I felt like it’s best to “future-proof” the Logger by guarding everything with pthread. This ensures, that whenever (or if at all) I add multithreading, the stream output will be clean and not a garbled mess of characters.
Again, this isn’t used currently, so this is another “future-proofing” measure. The logger can seamlessly start writing into any stream desired, as long as it exists. This will be used in the future, but no spoilers as to what will use it ;)
timespec_get and localtime_r for nanosecond-precision timestamps with thread-safe formatting.[[gnu::format(printf, x, y)]], allowing the compiler to catch type-mismatches in log messages at build-time.malloc. All formatting happens in a fixed 1024-byte stack buffer. This also prevents nasty, difficult-to-catch memory bugs, which could be fatal in a core module such as the logger.This screenshot perfectly demonstrates the preprocessor filtering. Notice the difference between the release and debug runs. In release, the TRACE log is vanished. In debug, the logger is fully verbose. This confirms that the compile-time filtering is working.
Showcase of multiple severities, as well as the FATAL one. Notice the Return code: 2 at the bottom. This proves that the logger successfully captured the error, flushed the stream, and triggered a clean abort() to prevent further corruption.
Ignore the makefile error, I suspect it occurs when I run
make run, and the binary exits abnormally (e.g. return code != 0), it thinks it’s an error.
Compared the the previous screenshot (right above), you can see the TRACE log level appears. This didn’t show in the previous screenshot as the minimum log level was higher than TRACE, thus showing the minimum log level works.
Here you can see how the code looked for the previous two screenshots. In the init function (zinn_log_init), you pass the minimum log level as the parameter, so when it was ZINN_LOG_TRACE, it logged all the severities. When it was something higher, it didn’t log TRACE, for example.
If you were to take a look at the code on the repository, you might notice that it is a bit different. That is due to this screenshot being a bit old, and I’ve went through like 2 hours of improving the Logger:)
With the first core module done, the only correct next step is to implement another core feature.
10 changed files with 1056 additions and 0 deletions
For zinn, I decided that the foundation had to be as robust as the final output. Before writing a single line of code, I spent the first hour or so building a thorough foundation from scratch.
For this project I have decided to use the latest C23 standard. Why? The answer is simple: modern features.
nullptr: Replaces the macro NULL. It’s a real type, which prevents type-confusion bugs and makes the code’s intent explicit.constexpr: This allows us to define constants with full type safety. More importantly, its value is known at compile-time, thus the compiler can catch math errors or type mismatches before the code even runs.And many others…
I have outlined the code style that will be followed in this codebase. The whole file can be found here.
The directory structure is simple:
.
├── bin <-- contains executable binaries
├── build <-- contains intermediate files
├── include <-- contains header files, allows sub-dirs
└── src <-- contains source files, allows sub-dirs
I have developed a recursive Makefile that supports multiple build confiruations (Debug, Release, Sanitize, Coverage). It features:
clang-tidy, cppcheck and GCC’s -fanalyzer.-MMD -MP flags to automatically generate header dependencies. This means the build system is “smart”; it only recompiles what is absolutely necessary.|) to handle folder creation. This prevents “ghost recompilation” where the binary is rebuilt just because a folder’s timestamp changed.The project is licensed under GPLv3, which ensures it stays as FOSS (Free and Open Source Software) forever.
Next, I plan to implement bulletproof core components, such as a logger and CLI argument parser.