Devlog #12 - Solid, Boring Progress

I’ve spent the last few days making regular feature updates to PSim.

Here are the highlights:

1. Unload Modules

Until now, if a user loaded a module and didn’t want it anymore, he would have to reopen the app. Additionally, there was no way to unload the built-in modules. I added an options tooltip in the module order GUI that allows the user to unload a module and everything it has registered.

2. Confirm Action Menu

If a user is about to do an action which might lead to loss of data (close the window, override a save file), we want to confirm they know about the potential consequences and accept it. For this, I created an interface for creating a blocking modal that forces the user to either confirm an action or cancel it.

3. Graphics Handlers Behavior Improved

The default should obviously be for graphics callbacks to have what they draw to the screen be projected into the actual intended buffer - that is, not anywhere in the top menu bar or the left-hand control panel. This is also already where particles are drawn, so a particle at 0, 0 is drawn in the top-left corner where the top menu bar and control panel intersect. This math had to be applied to every single thing drawn by a graphics callback, and I decided to fix it by calling graphics callbacks while in a raylib 2D mode that offsets everything so that 0, 0 is where a particle at 0,0 is drawn.

4. DPI Scaling

I switched to a new laptop this week, and went from a 14 Inch 1080p screen to a 14 Inch 1440p screen. Suddenly, the menu became much smaller, because the program was drawing by pixel instead of by the system’s default scaling (which, on my new laptop, is 200%). I fixed this by making Raylib scale everything by the user’s system DPI.

5. Module Internal Serialization

If a module has any internal state that should be preserved when a save file is written, it can now register a callback that returns a std::string and a callback that accepts one, and those can be used to serialize and deserialize its internal state.

6. Extended Module API Documentation

I’ve now written documentation for everything but the feature discussed in point 5.

7. New Particle Target

I’ve added a cross target placed in the position where a new particle would be placed according to the Position buffer, to ease on visualizing the new particle. This cross is customizeable in the settings menu and you can change width, length and color.

Devlog #11 - Adding A Settings Page, A Config System, And More Module API

Yesterday I wrote about my commitment to making PSim an actual, usable desktop application that doesn’t feel limiting to the user. What I did yesterday was create an error notification system, so if something goes wrong the user can understand why and code has a standard philosophy to handle issues.

Today, I put my focus on improving the user experience by adding a settings page to PSim.

The Layout

Personally, when I think of a settings page in a desktop application, I usually imagine a blocking pop-up (as in, a subwindow inside the application window that while opened, does not let you touch anything outside of it) with a few tabs for different categories of settings.
ImGui actually has great support for this, with OpenPopup to create one. the smaller, left-side portion of the settings page is a tab-selection area, and the right-side portion is the tab which is selected.

There are 4 tabs:

1. General
2. Simulation
3. Modules - modules can register custom settings
4. About - basically just the full text of the MIT License + a link to the project’s GitHub page

The Config System

Personally, when I change a setting on a desktop application, I generally expect that change to still be in effect when I close and reopen it. To do this, PSim needs to get to the target platform’s app data folder, create a folder for itself if one doesn’t already exist, and then save whatever information it needs there. Now, this is obviously annoying to have to re-write every time you want to save something as simple as what FPS the user wants, so I wrote a template system to save and access binary settings files on the fly.

I now had a way to make it so arbitrary settings stay the same every time I open the app. Now, I need to make it so modules have the same privilege. I created 3 new functions in the module API, writeConfig, readConfig, and registerSwitch. Each of these does as the name implies, and it allows you to add on-off switches (support for more will come later) and read/write configs to disk.

Conclusion

It may have taken 6 hours of work, but I managed to create a fully-functional settings UI with module support to my first desktop application. This is an important piece of architechture and makes it much easier to add more features in the future and expand the app into something I can be very proud of. Thank you very much for reading! Please enjoy this proof-of-concept video of me changing settings and reopening the app.

Devlog #10 - Don’t Hate The User

One of the most important unwritten (though I’m sure it is written somewhere) rules of writing good software is responsible error handling. I’m writing a desktop application, and personally, when I’m using a desktop application, I don’t like it when it suddenly crashes with no useful error message. This is especially possible in a desktop application written in C++, where irresponsible coding practices can lead to segfaults and other crashes without so much as a Java-style “NullPointerException”. It would be incredibly frustrating for the user to lose progress and not even know why.

Another potentially frustrutaing situation is when the program fails silently. It doesn’t quite crash, but when you ask a program to do something and it just doesn’t do it, that can be frustrating. When a program can’t satisfy an action the user wants it to make, it should make it as clear as possible that it’s aware of its failure, and attempt to clarify what went wrong.

What I’ve Been Doing Until Now

Up until now I have handled runtime errors by… well… not handling them. I wrote my application under the assumption that everything works smoothly, every module loaded by the user exports the correct signature, every file opened by the user is well-formed, and every file the user wants to write to has the necessary permissions. If any of this went wrong, the program entered undefined behavior. Most likely, it will crash.

My Solution

Whenever the user requests an action by the program, for example, trying to open a saved simulation state JSON, the program goes through the following steps:

1. Make sure the file is actually opened
2. Make sure the file is correctly-formed JSON syntax
3. Make sure the actual data in the file aligns with PSimUltimate’s requirements and constraints
4. Actually deserialize the file

If any of these goes wrong, the deserialization process immidiately ends and an error notification is pushed to the bottom-right corner of the screen. If multiple things are wrong about the data in the file, only the first thing in the deserialization order is written. There can be up to 3 error notifications displayed at once, with any more notifications being queued for when those are closed by the user.

This solves the issue of responsible, user-friendly error handling, and is an important piece of infrastructure for a usable desktop application.

Devlog #9 - Expanding The Module Developer API

Before I begin this devlog, I would like to apologize for being inactive over the past week. I have had quite a busy week, and I did not have much of an opportunity to work on PSimUltimate unfortunately.

What I Have Been Working On

I’ve been putting more attention to the module API, modules are the most important part of PSim, and their API should be more advanced than the very basic one that currently exists.

I’ve split the API into 5 different sections:

1. Physics Behavior - callbacks into force, velocity, and positions of particles
2. User Input - adding new keyboard shortcuts, reading mouse positions, reading which keys are being pressed
3. Graphical Behavior - being able to draw stuff on the screen
4. GUI - adding buttons, settings pages, descriptions, and more for specific modules.
5. Other Modules - modules should be able to register dependencies and force certain modules to be either lower or higher than them on the callback order.

I have been hard at work (when I can) on designing and implementing these needs. So far I have made some progress, but there is still much to do and research, and I’m sorry it’s not so juicy for a week’s worth of progress. Either way, thank you for reading, attached is an example of custom buttons that can be registered by modules in the top bar, and please keep following the project for more updates.

Devlog #8 - I Need Your Help

This devlog is not so focused on what I’ve been doing since the last one, but what I have been doing is pretty cool, so I’ll go over it:

PSim uses Dear ImGui as the backend for its GUI. Up until now, the font size in the GUI has been way too small (13px) and the GUI sidebar could not be resized in any way. I had to make it so resizing is possible and the font is more readable, because I want the app to be properly useable.

The first thing I did switch fonts from Dear ImGui’s default non-scalable font to Dear ImGui’s default scalable version, ProggyForever. This took a few minutes but the big challenge was getting resizing to do exactly what I wanted. Despite Dear ImGui usually having exactly the correct amount of features, there is no option to make resizing only possible from one side or disable the completely free bottom-right resize grip. I had to go directly into Dear ImGui’s source code and manually disable those things myself, which was a first for me.

What I Need Help With

Right now whenever I try to do anything I realize that I don’t actually have a good feel for what’s missing and what needs improvement. If anyone reading this could spare some time of their day to install my application and play around with it, maybe write a module. I have added proper documentation for modules to make the whole process much easier, so hopefully someone will be able to help me find issues and missing features. You can get the release on GitHub.

Thank you for reading and thank you for helping out!

Devlog #7 - Steady Progress Is Not Interesting

Sorry for the low logged time, I’ve actually been working for several hours but I forgot to install wakatime after switching to CLion.

What I’ve Been Working On

One of the most important things when working with multiple modules that work on a callback-based system is the ability to reorder modules so some callbacks get called before others.

Imagine this:

You have a module foo which clamps all particles to the window border so they can never escape the boundaries of the window.

You also have module bar which increments the X position of each particle by 3, each frame.

Say foo gets registered after bar, particles will get clamped to the border AND THEN move 3 pixels, breaking the behavior both module authors likely intended.

The Solution

Create a GUI which allows the user to reorder modules by, so a module placed after another module gets called last.

Implementation Challenges

I had to learn how std::list works (a linked list was the obvious choice for this use case), and let me tell you, it is by far one of the worst STL containers I’ve had to work with, and anyone who writes C++ code knows that MEANS somethimg.

Conclusion

Again, not much to write about today, but steady progress just isn’t all that interesting. Thanks for reading!

Mini Devlog - How Writing Unit Tests Made Me Fight C++ For An Hour

Whenever I spend some time fighting with C++ instead of actually being productive, I feel like I at least need to get a devlog out of it, so here goes.

Background

I’ve been writing unit tests for my particle simulator, and one thing I’ve noticed is that I have no code to protect global state when a single module is registered more than once, which leads to weird and unintended behavior.

My solution is lazy: silent failure. If a module already exists when addModule(const std::string& name) is called through the module interface function registerModule(const std::string& name) (You can read about why these are separated in an earlier blog post, it’s very cool!), the function just doesn’t register it again. Simple enough, but I wanted to implement it in “correct” C++.

The Implementation (Where Things Went Wrong)

What I did was create a function in module.cpp with internal linkage called getModuleByName which returns a std::optional<Module&>.

Well, apparently for the C++ standard that amounts to heresy, since up until C++26 the committee never managed to agree on how to handle std::optionals that contain references.

Well, I thought, I am already compiling with -std=c++26 in my project, so this should work. But it didn’t. I needed an updated libc++ implementation, which I managed to get through MinGW. I spent 30 more minutes fighting with CMake as it insisted to link against MSVC’s headers, but eventually I did manage to get it to compile.

A normal programmer would have just used std::optional<Module*> but I am definitely not normal.

Was it worth it? probably not. Especially when it’s all for a function which only has internal linkage and is only used in one place. But I like to think that writing code I’m comfortable with will help maintain the codebase in the future.

Devlog #6 - Adding Testing To My Particle Simulator

I’ve been working on adding unit testing to my project.

It was important for me to add unit tests for several reasons:

1. I have never written unit tests in C++, so it’s a good learning experience
2. I genuinely want this project to end up as a polished desktop application, and that’s not really possible without following best practices. Tests are important.
3. Parts of my application are already extremely vulnerable to bugs
4. It’s easier to catch bugs through unit tests than through actual use of the application

Catch2 as the framework

I chose Catch2 as my framework because it seemed easy to work with and not too verbose.

Current sensitive parts of the application

Position, Velocity, and Force are all independent wrappers around Vec2<float>. It is extremely important to me that these have proper arithmetic, but are not implicitly convertible, since that is almost definitely not intended behavior.

This is why I wrote almost 200 lines of code’s worth of unit tests to make sure that the typing system is strong and acts the way I want it to.

Catching bugs easily through unit testing

While a user will most likely eventually stumble upon any bug worth fixing, I don’t really have any users yet! I would like to ship an app that isn’t obviously broken, and to do that, I need unit tests that stress the logic of my program to make sure it can survive users messing with it.

Current progress on writing tests

I’ve written fairly extensive tests to enforce the Vec2<float> variants type system, and to enforce particles working and ticking correctly and interacting with modules correctly. I still have many more tests to write, and I expect to spend the next couple of hours doing that. Thank you for reading!

Mini Devlog - Just Spent An Hour Fighting With The C++ Standard Library + Live Showcase!

I just finished spending 8 hours on a massive refactor and new features, so I thought it would be nice to sit for a little while and play around with the program and fix some errors, and generally try to make the code a little safer.

I’ve spent the past hour looking at various crashes i’ve had and tryint to pinpoint and debug them, and pretty much every single one of them came down to the fact that I keep forgetting that std::vector<T>::resize(n) does not have the same functionality as std::vector<T>::clear(); std::vector<T>::reserve(n).

This should be a very simple concept, and yet it’s apperently riddled my program with bugs. That’s all solved now, and you can try release 0.1.4 here with an example boxing module! Happy programming!

Also, here’s a live demo of the program in action. The recording doesn’t capture the file dialog opening up, but you can tell it’s there before the particles spawn and when I load in the module.

Devlog #5 - I spent 8 hours overengineering my desktop application to avoid spaghetti code

A few days ago I wrote about how my small desktop app was already suffering from spaghetti code due to interaction between its modules. I had originally set out to create an application with just 3 modes: Gravity, Electrostatic movement, and chemical bonds. But this has a big problem: The scope is small, making development uninteresting and the actual application pointless to users.

The Change

In order to solve my spaghetti code problem, I came up with a solution: Instead of implementing 3 different modes, implement an interface that allows you to add as many modes as you want.

The Process

My first step was to decide what the public API should be for creating a new module. I settled for this:

What the module author needs

The module author may want to intercept a few different things:

1. Change Force, Velocity and Position values every frame when they are being evaluated
2. Add an attribute for a particle (like how a particle needs a charge in electrostatics)
3. Add a universal physical constant (like Coulomb’s constant or the gravitational constant)
4. Render something on top of the application

What I need to do

I built a module system to register all of the above, plus a psim_module_api.hpp header for the module author. It defines a C-ABI entry point that calls a C++ init function the author provides. I was almost done with the entire process when I discovered that Windows apparently doesn’t properly support the ability of DLLs to access functions in the main program by linking against them at compile time, and had to rewrite a large chunk from scratch.

All the user has to do is include the header and define a void initializePSimModule() which is forward-declared by the header and called by the entry point after populating the function pointer table.

Then we just use user-friendly functions to register anything the module might need. An example module is shown in the screenshots, along with its source code. This module creates constants for boundaries that particles cannot cross, effectively boxing them in a rectangle.

I rewrote the Gravity and Electric modules in the module system. They are part of the core application, so I’m not loading them dynamically every time, but they use the same API as dynamic extensions. This makes it essentially impossible to create spaghetti code around the different modes, and now all I have to do to load a new module into the program is call loadModule(const std::string& path).

The final flow is something like this: The module entry point is a function that receives a function pointer table to all the C-ABI functions that the module uses under the hood. It populates the module’s function pointer table (the one I added because of Windows) and then calls the main initialization function implemented by the module author.

To put things in perspective, here is what happens when you register a constant:

You call registerConstant(...) with a std::string and a std::function => they’re destructured into C-friendly structs => the module finds _regConstant(const char* module, size_t moduleLen, ..., PSIM_Constant_Change_Callback) in the pointer table and calls it => the main program restructures the C parameters back into C++ types => the main program calls the real registration function.

And this is precisely the hard part: The module author can believe that he’s dealing with the same exact std::span<Particle> that is passed to him by the main program, but in reality that span was destructured into a Particle* and a size_t and was then restructured, all before his callback was even called.

This took me 8 hours, and out of everything I’ve done with PSimUltimate for now, this has definitely been the most valuable learning experience. I learned how to expose C++ APIs while maintaining a stable C ABI backend. Very cool!

Devlog - 04 - Fast JSON Serialization And Deserialization
I need a way to save the simulation’s state. My options were few:

1. Create a custom binary format (pretty easy, but not readable)
2. Create a custom text format (hard but readable)
3. Use an existing JSON library and format state into a JSON (very easy, very readable)
   Option 3 is naturally the best one, so that’s the one I went with.

I used nlohmann/json as my JSON library, and for now I have just gone with the native approach of manually looping through the particles to build an object tree and walking the object tree to deserialize files.

Now, I know that technically this part of the application doesn’t really require any form of optimization, since even with 1,000 particles and O3 compiler optimizations this native approach takes a small enough time that there isn’t even one frame of overhead. But I still didn’t want to leave performance on the table for no reason.

I started with a 10000 particle serialization benchmark. It originally took 100ms to serialize and write to disk with O3 optimizations. After a few changes, mainly first formatting the json object to a std::string and then piping it to a std::ofstream to reduce OS overhead, I quickly reached about 40ms for building an object tree and writing to disk.

I got similar results without really adding any optimizations to the deserialization process.

Now, this is definitely fast enough for the kind of application that PSimUltimate is, but it still seems slow. People manage to write gigabytes of JSON per second, why is my process so slow?

Well, turns out the main reason is the fact that nlohmann/json is, well, pretty slow to build an object tree with. Every single item in the tree is a heap-allocated object, which means both malloc and free overhead and cache-unfriendliness. You can see in the benchmark that serializeState and deserializeState both take a couple milliseconds despite not doing anything but set up the work for the other functions, and this is because of the time it takes to malloc and free the object trees.

Theoretically, there are ways to write faster nlohmann/json code that doesn’t rely on as many heap allocations, but it’s not really worth it for me to do spend time looking into it and designing it right now, so this is good enough for me for now.

(Advice Welcome) Devlog - 03 - How to design extensible architechture for my particle simulator desktop application
I tried making a simple refactor - when I was first experimenting with the code, Whenever electrostatics were disabled the vector of charges for each particle was reset. This is obviously not great behavior for when you want to be able to quickly toggle modes on and off, so I tried making it so toggling electrostatics simply switches whether the simulation notices them or not, and not whether the data exists. This took 4 different attempts at compilation and playing around before I arrived at the expected behavior - and all I was doing was just deleting some code!

It became clear that even though my application is still tiny (under 1k lines of code) and even though I have only 2 modes (gravity and electrostatics), I already have spaghetti code.

Now I need to do one thing: design a safe and extensible architechture for adding new types of forces.

The first thing that needs to happen is deciding how each mode interacts with the particles. This is the easiest part: Each mode is told to calculate the force for which it is responsible for every particle, and those forces get added to an array of forces, and then the sum force for each particle is used to determine its new position. This alone means there will never be any meaningful spaghetti code.

Now comes the hard part: deciding how modes should be declared and added to the application. There are two approaches:

1. Create an interface for modes which is known at compile time. This will lead to better performance due to optimizations, faster startup times, and more control for me over how the application works.
2. Create an interface for modes which is registered dynamically at startup, this approach allows for 3rd-party extensions in the form of dll files that are dropped into some folder in the install directory. This leads to slower startup times, less optimizations due to the need of a stable ABI, and an overall increase in architechture size.

I don’t know what I should choose, but I would love to have someone’s input on which would be better for my project.

Devlog - 02 - Before Shabbos
Since the last blogpost I have been working on cleaning up the existing codebase, improving some APIs and laying down the groundwork for electrostatic movement.

I spent over an hour splitting the GUI rendering process into multiple independent files that are each much smaller than the monolithic “graphical.cpp” from before, which allowed for a cleaner structure which enabled me to think about the code more cleanly and refactor it further.

I created a generic Holder struct which runs some function when constructed (and can receive arguments for that function when constructed) and calls another function when it goes out of scope. This is useful in ImGui because very often you have stuff like ImGui::BeginGroup() which must be followed with ImGui::EndGroup() which can easily be forgotten and also leads to unclean and spaghetti code.
Instead, All that needs to be done now is a declaration of a GroupHolder variable, and then ImGui::EndGroup() is automatically called when that variable goes out of scope.

I moved a bunch of constants from graphical.cpp to a PSimImpl namespace which allows them to be accessed from anywhere without polluting the global namespace.

Additionally, I spent some time laying down the groundwork for electrostatic movement, which includes a Force type, which is a strongly typed wrapper around Vec2. I need these to be strongly typed so Force cannot be accidentally converted to Position or Velocity, but I also wanted to be able to explicitly (or implicitly) convert a Vec2 to one of its wrappers. This took a while to get right and eventually lead to the line of code in the screenshot.

One more thing: I noticed that currently this website has no way to actually find the GitHub page for other people’s projects, so here’s the link to mine: https://etaiami09-cmd/PSimUltimate
Enjoy!

Devlog - 01
This project is a remake of my first C++ project, which was an electrostatic particle simulator. This time, I’m looking to do a bit more - the simulator will have multiple modes which can be switched on or off, and instead of trying to reinvent the wheel by making a GUI with raw SFML, I’m using ImGui and binding it to raylib for the particles.

So far, I’ve spent multiple hours (I started before linking the project) setting up the project with CMake, an install wizard, and external libraries. Additionally, so far I have implemented the ability to toggle gravity and the simulation, and there is a button to toggle electrostatics, even though electrostatics are not implemented yet.

I also have a menu for spawning new particles, an about page, and a menu for editing constants (like gravity).