3GB RAM milestone!!

How I managed to achieve 3GB RAM usage in the live environment (but at the same time - break a couple things)

Stage 1 - package optimization

Probably not a surprise to you that Gnome desktop environment is a heavy beast. Well, as it turns out, it bundles a bunch of stuff that’s not technically required to run the system.

Alpine package gnome adds many apps (like calendar, weather and more) that aren’t relevant anymore. However, even though unused, they take up space, and in Alpine, the live environment installs every package onto ram to provide a diskless environment.

At first I tried making a filesystem capable of loading packages from mounted iso as it’s necessary for the install either way, but optimizing package selection turned out to be easier (besides - who has a 64bit system with less than 6GB ram these days?)

Stage 2 - package loading

The fact that package selection is optimized to reduce Gnome’s bloat wasn’t enough to make the install environment lighter.

Alpine has a file /etc/apk/world in which every package that should be installed is defined. Utilizing it I could control what packages must appear within the live system, and which - not (because you probably don’t need a heavy browser before installing the system).

Stage 3 - things break if you’re too brave

And they broke for me too. Certain admin functionality (like some polkit integrations) don’t work anymore due to the limited selection (and probably some hidden dependencies) of packages.

I think I have diagnozed the issue though - testing suggests that it could be the missing linux-pam package or some dbus initialization race conditions.

Other important updates

Besides implementing optimizations to the image (consequences of which I can hopefully fix), I also made sure to update extensions, as well as the matcha-calamares package that now has a broader support for systems and reduces crashes. The building pipeline was also slightly tweaked.

Either way, I’m off to fixing the issue, so see you in the next devlog!

Tested and working!

How (and why) I implemented tests in RedeeMOD

The reason behind “why”

My dad is a high profile developer, and many new things I try are influenced by him. While I always liked setting up CI/CD pipelines and other automations, one thing my projects lacked was testing.

Turns out, many tech jobs require knowledge about tests, because they guarantee several things:

1. They assure that your application works as it should
2. They document your working concept
3. They let you make refactors that don’t hurt the functionality

So, after considering these benefits, I started reading how tests are implemented.

How did I implement tests for RedeeMOD?

There are 3 main types of tests:

• Unit (where one small component is tested)
• Integration (where several combined components and their behaviour is tested)
• End to End (where full system flow from start to finish is tested) (they take a long time, so I didn’t even consider them for RedeeMOD)

First thing any testable software should have is clear dependency injection.

Dependency injection is a pattern where you don’t call dependencies directly, but pass them externally. I opted for implementing method argument injection that I could later use in tests.

Why is DI required? Well, during testing, the external environments must be mocked and created virtually to test different use cases (things like sys.* calls and more). Luckily, RedeeMOD already had a nice and semi-testable architecture where very few things needed to be changed.

Writing tests is a tedious and rather boring process, because all the edge cases must be defined. Don’t hate me for this, but I handed over writing tests to AI. My arguments for why AI is appropriate for this are:

1. You can’t test software that hasn’t been previously written (by people)
2. Repetitive work takes away precious time, and AI just performs checks

While one could argue that Test-Driven Development exists, after trying it out I can confidently say that I’m not a fan of TDD.

However, giving very broad instructions, like “Make me tests for this project” usually doesn’t go well, because AI can’t retain project philosophy and tests what doesn’t need to be tested (things that don’t contain logic, thus can’t really fail).

RedeeMOD currently has a test suite based on pytest with integration and unit tests written.

How tests could benefit you?

With tests, instead of clicking around (and often forgetting) every edge case manually, you can offload it to a machine that does hundreds of calculations extremely fast. Not only is it a well-regarded pattern within the developer community, but also helps with coding efficiency and guards from failed refactors.

Quake fans - I got you!

Thought RedeeMOD would only be available for niche titles? Not exactly!

Which Quake?

Most people would probably agree with me that Quake 3 Arena was the best entry of the series for multiplayer games.

After seeing just how many Quake 3 mods there are (some even being standalone games) I had to implement an adapter for it with support for both standalone games and mods!

I resorted to the IOQuake 3 engine support due to its extra capabilities and being the modern choice for most, so original Id Tech 3 might not work.

How does it work?

IOQuake 3 inherits cvar support (and adds more!) from Id Tech 3 engine. Utilizing com_basegame, i could bypass Quake 3 Arena file presence for standalone games.

What about Quake 3 MODS that are not standalone?

The new adapter follows an interesting edge-case detection logic:

1. If a mod has default.cfg file somewhere within its .pk3 files, it’s considered standalone. Other games are required to have genuine Quake 3 Arena present, about what RedeeMOD will warn.
2. IOQuake 3 requires Quake 3 Arena or Team Arena files to be in baseq3 and missionpack directories. This adapter handles warning users against such restriction.
3. Quake 3 Arena can’t be launched through RedeeMOD as a mod. Everything else works perfectly fine (based on testing).

All mods I tested worked fine. If you find one that does not, kindly report it on GitHub issues 😄

The most challenging adapter of all

No doubt, the amount of testing that was required to get this working is absurd, and definitely it was the most challenging adapter of all.

Pre-ship devlog!

Maitenance pre-ship revealed 3 more bugs that I couldn’t live with…

Bugs? What bugs?

Actually, very counter-intuitive!

1. UT99-specific: Turns out, when Unreal Tournament ’99 reads configuration and finds duplicate filenames in different directories, FIRST ENTRY DEFINED WINS, which, from coding perspective, is very unusual! Usually, if you define a variable, it can be redefined later. UT99 does not work like that, so I made the mods override any system files if they do override any.
   I noticed this by trying to load one of my favorite mods ever - the HD pack, which overrides system textures. Glad this bug got noticed!
2. UT2004 skins not working: Some skin mods would not work due to the correct files not being imported on game runtime. By adding an extra file extension to the list of import files, I was able to fix this!
3. Label truncation fails: Previously I had implemented an ElidedLabel that truncates a string and replaces the ending with ... if it is too long. In some rare instances however it failed and set a boundary on window size instead of truncating. However, it’s not an issue anymore!

Error handling is better now

I ran into an issue that got me confused - I couldn’t launch a game. I panicked, thought that my launcher was broken again, and only then did I realise that the game wasn’t installed!

Launcher did not notify me and I got frustrated. Since I want the UX to be flawless, the best decision was to make a pop-up in case of a critical error.

I gotta say - the ship is finally truly 100% ready!

Code, break, fix, repeat!

Here’s what the final 1.0.0 release of RedeeMOD brings:

Fully tested and documented!

I built RedeeMOD not only as an app to manage Unreal Tournament games exclusively (though at first, that was the plan), but also as a framework to create more games in the future compatible with RedeeMOD!

I recently wrote a Contribution guide that lets anyone create adapters and expand RedeeMOD by following a detailed guide!

Also, as a standard practice, GitHub actions were added to help with releases. Both Windows and Linux builds work perfectly, but are unsigned, so Windows might throw Smart Screen or Smart App Control warnings. These are safe to ignore, and if they bother you too much, you can always resort to launching RedeeMOD directly through Python!

Changes in code

The code is formatted with Black formatter. It’s way easier to format the code using a tool than doing so manually.

Also, I found some bugs that weren’t immediately noticeable. Things like:

• An uninitialized variable being still required in code (produces a TypeError)
• Image painting events overriding each other
• Much more

They are now fully fixed. Some inconsistencies between files have also been resolved.

This latest build now features logos to feel more professional. User interface has been updated with more detailed card subtitles, so the app is easier to navigate.

Overall, RedeeMOD is in its stable form for the first time ever. Now it’s just the matter of whether more adapters will be created for other games by the community!

Does RedeeMOD lack something?

Well, the mod management is very simple - adding, removing and enabling/disabling a mod probably doesn’t sound like the most sophisticated mod manager on the earth, but I am planning to later add a pop up that lets users customize the mods even further, like adding custom launch options, paths, even image customization, but that will require more time and I feel confident enough that the current feature set is good for the first ship.

For more info on RedeeMOD, visit its GitHub repository!

Finally, it feels complete!

What I’ve been through and why do I feel relieved

So, after a LOT of trial and error, and MANY wrong paths taken, Unreal Tournament 2004 adapter is functioning!

Why it took so much effort for such a simple thing

UT2004 differs from UT99 in many different areas, even its modding system:

• It now has a dedicated -mod flag instead of using an overriden .ini file
• Every mod has its own mandatory structure, and UT2004 doesn’t like multiple mods loaded
• Mods can load diffs of system .ini files

My plan was to replicate what I was doing with UT99 - building a custom mod at runtime that would bundle every enabled mod together.

Since this game is quite old, so are its development sources. I resorted to the best source there is - proven to work mods.

My top 3 problems with UT2004, ranked:

1. Using empty files or ones that are with LF line endings instead of CRLF produces either a segfault or a Bad Mod error
2. UT2004 creates a LOT of mod-specific runtime files, and they need to be cleared to be regenerated (I hate caching, as going through them all is difficult)
3. I couldn’t get mods with heavy custom GUI usage to work, unless they’re being launched standalone, using UT2004 Community tab (which defeats the point of my launcher)

Since I wanted to use the modern -mod flag, I had to make my own mod, which needs such files to function:

• ModDir/UT2K4Mod.ini (Name and description)
• ModDir/System/Default (Diffs of what to add/remove)
• ModDir/System/DefUser (Can be left empty, but requires at least one \r\n line)

I’m thankful to whatever or whoever made me as patient as I am, because DefUser being absolutely empty causing a segfault or files requiring CRLF even in Linux was very unexpected, and required hours of investigation.

On top of said implementation, backend framework also needed changes to ensure stability and flexibility. New methods returning more data to the game adapters were added, as well as improvements in attribute names and path handling.

Bugs are always present…

After testing with some edge cases, I found bugs that got resolved shortly after.

Windows, as of now, works flawlessly, but I’ll be performing more tests when I distribute a full build.

What’s that, some new GUI?

Yes! I wanted to implement logos since the day 1 of starting this project. Surprisingly, that was one of the easiest modifications to date.

Screen state tracking was also improved to provide no visual bugs after adding, removing, selecting and deselecting added material.

Finally shipping, finally stable?

Again, yes. All the intended functionality is verified to be working, and you can already try it out for yourself!

Running python -m frontend.app from the project’s root will open up this GUI for you to play around! Over the next few days I’ll finish making all the necessary branding, README and ensure licensing is correct. Then I’ll push a final devlog and make a ship.

Thank you for everyone following RedeeMOD’s journey. Keep rocking, and make the world truly Unreal!

Last devlog before shipping!

Final changes

Before pushing any major update I tend to test all the things myself to ensure that user-facing compatibility is pristine. However, when I went on and tried engaging one of the main features of my installer - parallel installs, I began to face a problem - the extension wasn’t installing!

My guess is that Google family of browsers (Chromium, Chrome and Chrome Canary) have some protection or sandboxing, and trying to initiate an install across multiple browsers is disallowed deep inside Chrome’s code.

Another possibility is that my code is broken or the server that runs locally to install the extension isn’t configured properly. This is probably more likely, as I could’ve overlooked things.

Either way, the easiest fix for that was to disallow concurrent installs and only let users select one browser at a time to install the extension for. This works surprisingly consistently, as I wasn’t able to encounter any issues.

Thanks for anyone who keeps an eye on this project. It was more of a pain than actually something useful, but I don’t regret my efforts.

Stay creative, stay open source!

UI tweaks!

This looks better in my opinion

After taking a look at the previous design I thought it looked quite ugly and inconsistent when some cards have center-aligned elements and others don’t. This update fixes these issues!

You really spent so many hours on simple UI tweaks?

Yes and no. While the primary purpose of this devlog IS to describe subtle interface change, under the hood many quality of development improvements hide as I’m trying to implement and fiddle with Unreal Tournament 2004 game adapter, which required a more streamlined API of adapter creation rather than what was previously developed.

So, this devlog has hours logged from both a fully developed interface and half-developed new game adapter implementation. Stardance was warning me about getting close to 10h limit, so I must post this before diving deeper into Unreal Tournament 2004, which has way more twisted stuff within its modding system.

Stay tuned and wish me luck!

A fresh look

Expanding on the design I like, I decided to add some improvements.

What improvements exactly?

All these things:

1. The launcher now supports adding multiple games (even though currently the only game supported is Unreal Tournament ’99 (more to come))
2. A sidebar was introduced for better UX
3. A more user-friendly card design was expanded to become the main component interface relies on
4. Backend experienced a bit of an overhaul too to support game/mod branching
5. Safeguards were implemented to disallow adding multiple copies of the same mod OR directories that do NOT contain Unreal mod content
6. Config is now stored in local paths for saving/loading added material
7. Library tab disappears if no games are detected

Wow! This is really awesome! Is it ready to be shipped?

I could ship the current version, but there are exactly 3 things stopping me from doing so.
I need to:

1. Verify functionality on Windows (so it doesn’t become another Linux-only project of mine)
2. Add verification for game paths and prompt user to enter location for both executable and config (if applicable/not found)
3. Add Unreal Tournament 2004 functionality to leverage multiple game selection and see if it works (as well as possibly other games of the franchise, like Unreal and maybe even Unreal Tournament 2014)

Interface update

A functional UI demo has just been made!

Following a design language similar to Unreal Engine 5, I was able to make a functional UI, that can launch the game with selected mods!

The UI is written with PySide6, which allows for Qt6 API to be used on Python, giving a modern development workflow, with customizable stylesheets and cross-platform capabilities.

What’s next?

I want to rewrite the launcher a bit to support more games in the future and be more architecturally correct, at the same time feeling more premium both from the user-facing app, and the source code too.

Discovery - done!

With the installation discovery done, I can finally focus on the real provisioning and frontend.

How does it work?

Utilizing common installation paths of Unreal Tournament, I could build a simple config.json file that has all this information for Windows and Linux.

Then, I wrote a simple json parser, and methods to use as a simple interface for the frontend.

What else currently exists?

The current source tree has this functionality implemented:

• Filesystem utils for finding and sanitizing paths (required by Unreal Engine to load paths relative to the executable)
• .ini file editor (because Unreal Engine operates on .ini files for configuration, one of which allows for custom file discovery. Editing it allows injecting custom mod files)

Next plans?

I will try and build a user-facing application that does exactly 3 things:

1. It discovers the installation directory (so the frontend can also act as a launcher)
2. It lets you add folders with mod files
3. It lets you load one or more mods (and the base game) by constructing a .ini file on the go

Maybe something more will come to my mind as time passes.

Never more excited!

What am I building now?

As a kid, I used to play Unreal Tournament ’99 with my dad. I always thought that game is genius - how it encompasses the soul of classic arena shooters yet still being fun to play!

What is this Unreal Tournament?

You’ve most likely heard of Unreal Engine. The reason it got this name is due to Epic Games’ first major game - Unreal!
While at first it acted as a competitor to Quake I and II style games with its campaign against various creatures, in a year turned to compete with upcoming genre of Arena Shooters, specifically Quake 3, focusing entirely on multiplayer.

Never heard of it. Why?

While the game itself was awesome not only the way it was written (with an interpreted UScript language, which lets pure UScript mods run on every platform, as well as GPU utilization), and Epic even released the last installment’s source code available to read with promises of development based on what the community wants, quickly realized the harsh reality of Arena Shooters losing market share and turned to developing Fortnite, finally shutting down official server support on 24th of January, 2023.

So can I still play it anyhow?

Despite cancellation of the whole series, community has stepped in and is still keeping up with modern games!

Epic Games even allowed the Unreal and Unreal Tournament (original as well as 2004 version) ecosystem be maintained by a group called OldUnreal, while the latest game is available on UT4Ever, a self-hosted group of volunteers, who have rewritten Epic’s servers. And, guess what, all of this is available for free!

What RedeeMOD aims to provide

I am always looking at ways to modularize the software I use for maximum customization. Currently the mods follow a drag-over logic, which mixes mod files with original game’s files, rendering the original, pure and unmodified version unplayable if you ever want it back.

After researching how modding architecture of Unreal Tournament ’99 works, I was able to prepare a plan on how to implement a mod loader, and started to work on it! So stay tuned, and join the Unreal Tournament community!

Linux it is…

How this project turned out to be Linux-only

While testing how builds worked and preparing this project for a release, I came to the realization that if I’m promising a cross-platform product, I should test it on all platforms first. I’m a Linux person, so when it all worked on my local machine, I didn’t bother much.

Turns out - that was a huge mistake! Chrome-based browsers on Windows and macOS tend to block custom policy installed extensions due to security reasons. For that to work, a machine must be enrolled to a real management framework - Active Directory or similar. Linux is excluded due to no standard tool existing and the fact that not everyone using a Linux machine might have root access.

What did I do to resolve this?

First of all, I panicked, as any normal human being would. My first thought was to ensure a manual installation flow on Windows/macOS by letting users install the extension through development mode. However, the thing that disallowed me from implementing such approach was incompatibility between browser Preferences file schemas (at least when I went to check, it seemed this way). Not knowing how to ensure a working build quickly, my solution was to only support Linux systems.

Can I still install the extension on my non-Linux machine?

Yes! If your browser is built on Chromium (the architecture that powers Chrome, Brave and more), you can, and there are still instructions on how to get the builds working on all platforms for the extension itself. However, you should be aware of such things:

1. You will get the default Alt+Q shortcut provisioned instead of Ctrl+Tab override (unless you decided to edit the BrowserInstall/Profile/Preferences file after installation (replace BrowserInstall with your browser’s data directory and Profile with your profile name (Default by default)).
2. You are taking management into your own hands without any official support from me.

Are there more updates planned?

Not really. I set up this project for shipping with custom branding and subtle tweaks within extension code, as well as a fully working provisioning framework for Linux users, and I’ll go create something new now!

This was never intended to be production-ready, but instead serve as a tool for making Chrome more usable than it currently is.

Even if you see a struggle, try to do everything in your will to overcome it, even if it means cutting corners is necessary!

Suit up!

A graphical way of installation was just implemented!

After quite some time of research and rest from constant development, I finally found out how the extension should be installed, removed, and provisioned for human usage.

The installer is still in very early stages of development, and I believe that some bugs still exist, despite my best efforts of testing various scenarios of installation (for now, it’s still fully proven to be working on Linux, with no guarantee of Windows or macOS functionality). So let me tell you what I achieved.

The offline installation framework

This installer fully works offline, provided you have a built and signed extension ready.

About its inner mechanisms

Chromium-based browsers (like Chrome, Edge, Brave and more) expect the extensions to be installed through one of these ways:

1. An extension store, provided in browser’s configs (for Chrome the default is Chrome Web Store, other browsers often implement their own)
2. Enterprise policies with force-install capabilities
3. Through default startup install list

I chose the second option. It’s the only one that reliably lets me achieve one thing - locally hosted extension installation.

Why policies?

Well, Chromium-based browsers expect an update.xml file hosted through http, alongside the extension.crx file, which is signed using some private RSA key. I wanted local hosting for security reasons (so the user has more control of what happens on their pc rather than downloading something from the internet (more on that later)). Only way I can get true local installation is through those policies, as limitations apply using other methods.

How the installer’s lifetime goes

First of all, after user chooses desired browser profiles, it closes selected browsers, if any are open, hosts the installation server, deploys the policies, opens the browsers up briefly (installation can’t be performed in a closed browser), closes them again and provisions the shortcut (as one of this project’s goals was to integrate natively with Chromium’s Ctrl+Tab shortcut for switching tabs). After all that, all selected compatible browsers should have the extension installed. Uninstalling simply makes sure to remove any provisioned files (extension itself and the policies).

What limitations it has?

1. Enterprise policies prevent the user disabling or uninstalling the extension through browser’s UI tools. It’s treated as “managed by organization”
2. A “Managed by organization” label appears. It does not limit the functionality of the browser, but can be irritating
3. If you don’t trust this software and the things it does (system file and registry changes, root access requirements and more) can feel a bit unsafe. On the other hand, the source is open and can be inspected by anyone at any time, altered or forked to fit your needs!

Next steps?

Well, the extension will still need some sort of way getting on your PC, so I’m thinking about these 2 methods:

1. Making installer download it from the latest GitHub release
2. Adding a –local-path= flag for custom, local builds and providing a script for auto-packaging the extension yourself

Also, I need to think about how the extension should be updated, but that’s for the future.

Thanks for reading, I’ll update you with the newest updates soon!

Storytime!

I feel like every project I make I face a similar set of limitations. So, let’s make a recap of the first stage of this project!

Very lucky unlucky day

As usual, I’m facing API issues. I enjoy making app extensions less and less due to restrictive rules they enforce. To make it clear, initially I had a specific set of plans at which my extension would excel compared to others:

• No timeout logic, instead handling tab switching based on specific key inputs (this will be important later)
• Save/load tab history from storage for access after restoring tabs
• Unlimited tab history
• Indication of the tab that will be switched
• Ctrl+Tab override

However, only one of those I was able to accomplish, with another one planned for the second stage of this project:

• Unlimited tab history
• Ctrl+Tab (planned for later)

Spending a lot of time, I at least made sure the extension functions well, with no memory termination and with each window having its own tab access history.

Why other things failed?

Well at first, Manifest V3 greatly reduced the amount of stuff that can be accessed by Chrome’s API. On top of that, chrome:// pages can’t be anyhow monitored or modified due to essential safety. This posed many issues, one of which was the inability to detect the press of a modifier key, so I had to go one of two ways:

1. Build an OS-level keylogger which communicates with the extension through an open bridge, but I turned it down because:
   • It’s a threat to security
   • Python on the host computer is required (in order for me to retain a single codebase for each platform)
2. Or just resort back to carefully adjusted keypress timeouts

I chose the second option.
I spent a whole day trying to save and load tab history, but the problem is that Chrome:

1. Doesn’t expose any method to etch the data to a saved session
2. Figuring out when history should be loaded, when saved to disk, and how to conect fast RAM with slow disk I/O is a true pain due to asynchronous nature Chrome is built on

And, any tab indication falls short because Chrome doesn’t have any API to reliably highlight tabs without selecting them.

Next steps?

Next thing I will most likely do is make the provisioning system that overrides Ctrl+Tab shortcut and installs the extension automatically.

The extension itself is memory safe and very useful as is. I don’t plan to update it anyhow besides changing the current branding.

The bitter truth

What I wasn’t able to accomplish

Due to Chrome’s very restrictive API, there are things that will not be implemented as promised. Things such as:

• Holding modifier key freezing the queue (instead, I resorted to using timeouts)
• Indication of what will be the tab switched

Not all hope is lost!

Having said the limitations of Chrome’s API, I was able to fulfill these things:

• Unlimited tab queue (or at least enough for any normal usage, I haven’t pushed the extension to the limits of RAM usage)
• Intuitive usage (timeout seems appropriate, and the extension works)
• Clean and modern development flow

What’s left to implement before moving to provisioning?

Before I start coding the Python applet that would provision this extension, I want to also solve these problems:

• Implement restoring tab order upon closing and reopening the browser
• Adding any newly added tab to the top of the queue
• Fiddle around with timings to find what’s the best for intuitive usage that doesn’t get in the way

A new project begins!

What’s it about?

I’m sick of not being able to switch recent tabs in most of Chrome-based browsers! And the problem with the existing extensions is that they don’t memorize more than 5 tabs, and use a timing-based system, which requires very fast shortcut tapping. I’m on a mission to change this.

How will this extension differ from others?

On top of my goals of making an unlimited tab queue and fix the timing issues, the extension will be 100% free and open source, also I don’t plan to release it on any extension store, because I have a different install method in mind.

How will I install it then?

I want to make a beautiful Python CLI or GUI wrapper that installs and provisions the extension. Things like:

1. Ctrl+Tab shortcut setup
2. Installation
3. Information on what will be changed inside the system

Will all be shown and handled by the installation app

Stay tuned! If I succeed it will be a very nice extension for everyone and every machine!