As requested, README has been updated and rewritten by a human.
There are also now three ways to install - directly through PyPI by doing pip install waifmark, or downloading waifmark-macos.zip and unzipping.

there’s not a lot otherwise to say about this devlog

it was found that I accidentally committed using personal details and a local email.

The gh repo has been wiped and recreated; here’s proof of the original repo and work.
Thank you reviewers for your understanding.

Waifmark 2 has been updated with a new UI!

After criticisms of the previous UI being too AI, I’ve refactored it with a cleaner, pastel aesthetic.

I kinda like it!
tell me what i should change next -»>

(readme has been humanified too…)

after ~18 hours over two days of benchmarking, 17 models were evaluated on the waifmark 2 benchmark.

(💢💢💢 benchmarking time got recorded to root ~/ and so I can’t include in this project time)

Final findings:
1: higher param models like Qwen3.5 9B and Gemma 4 12B are able to score high despite their low Q3 quants, with only their speed dragging them down
2: Nanbeige 4.2 3B and Granite 4.2 8B at Q4 are not considered to be mainstream open models (i.e. competing with Qwen or Gemma), but scored incredibly well given their small size. The only downside came with unoptimised runs and thinking traces inflating the time/response metric.
3: every AI lab except for Liquid (LFM) saw a direct proportionality trend, that for a higher score a higher time/response is required.
4: As a larger portion of the benchmark is now tied to Agentic tasks in a sandbox, there is a much larger variance in the distribution of scores with more recent models scoring comparatively higher, with less models scoring in-between compared to Waifmark 1.

The Pareto Frontier for Waifmark is:
Qwen3.5 9B (89%) -> LFM2.5 2.6B (55%) -> Ling 3.0 Tiny (35%) ->Qwen3 4B Instruct 2507 (21%).

Now, the issues faced:

• UI issues from the refactor to FastAPI + frontend; fixed with page indexing.
• During benchmarking, many models timed out due to unstable mlx weights, a corrupted test bank during Day 4 with 6 times the number of actual roleplay scenarios caused a benchmark time of over 12 hours; once I realised the mistake at the end (due to ~600+ tasks that had to be reviewed) I reran the benchmark with the correct test bank (which had also been recently changed to 1/3 agentic + 2/3 roleplay, and consists of 4 times more questions than Waifmark 1.
• over 260+ out of ~600 total tasks had to be re-reviewed by a human, as the LLM-as-a-Judge flagged the answer. Most were agentic tasks that the benchmarked LLM did not provide an answer to but had demonstrated steps in its reasoning, with toolcall accuracies above 0%. This took a significant portion of my time to audit all scores.

See images for results, or navigate to:
https://qubixal.github.io/waifmark/chart.html

finally, benchmarking has begun.
The benchmark was re-evaluated meticulously based on Qwen3.5-4B as a ~50% guideline for benchmarking.
Then, I set up my now dead macbook pro (dead display, dead battery) to a portable monitor to use as basically a cloud server and started benchmarking. This is the fourth night of trying (after benchmarking failed over the night due to the computer not plugging in and losing all charge (day 1), automation logic not working (day 2), LFM weights getting stuck (day 3) so hopefully it works today with the 17 consecutive models to be benchmarked.

using 17 models, including:
MiniCPM5-1B (0.57GB)
LFM2.5-2.6B GGUF (1.59GB)
Granite3B (2.09G8)
G9V3 (1.77GB)
Nanbeige (2.50GB)
Granite8B (4.98GB)
Ling(4.58GB)
Falcon (4.28GB)
LFM8B (4.51GB)
Gemma E2B(4.04GB)
Gemma E4B (4.79GB)
Gemma12B (5.30GB)
Qwen3.5-9B (4.35GB)
Qwen3-4B-2507 (2.1GB)
Qwen3.5-2B (1.60GB)
Qwen3.5-4B (2.83GB)
DeepSeek-R1-Qwen3-8B (4.29GB)
Note, all models were selected to fit within 6-8GB vram total so it could be evaluated on my local hardware.

devlog 4 - UI refactor
streamlit was replaced with FastAPI + a frontend and i think it looks much better than previously now.

issues + features:

• hugging face’s api (i.e. HfApi.list_models() was not filtered correctly during search)
• direct model import based on local dir is now possible
• openrouter judge was outdated and replaced, now user-configurable
• install now possible in a oneliner pip install -r requirements.txt && waifmark
• if a benchmark was cancelled, previously a bg server process vLLM workers would leak; instead process groups (start_new_session=True) and os.killpg gets rid of all processes when a session is ended.
• streamlit ran CORS and XSRF disabled; rebound to localhost

devlog #2: UI prototyping

i began working on what the UI might look like using photoshop.
this is in particular for the main screen, where the user will be looking out a window into an ambient background.

The UI will be split into layers, the sky in layer 0, skyline in layer 1, sun/moon in layer 2 (typo on document), background + windows in layers 3 and rest of UI on top, so that each layer can be independent and give a better depth of field.

devlog #1

Idea: we all work from desks (at least i hope we all do), and so what if a webOS could take that familiar look?

deskOS: a comfy desk overlooking scenary from a window. You work on a desk where the windows become books, tablets, stationary…

i hope this can be cool.
Right now, I’m using React Three Fiber to create a 2.5D scene using 2 combined 2D scenes. This is done as a test by setting up a 3D cartesian system and mapping two rectangular planes which will become the viewport/workspace, and now i’m prototyping the transitions. Once this is polished, I’ll move to actually creating the webOS content such as apps, the welcome screen, etc…

notdev log #2
basically finished every major hardware component.
worked on:

1. fan exhaust system + mountable radiator
2. adjusted sff->pci splitter board dimensions to fit
3. added PSU adapter/expansion boards (-> 8pin pwr/24pin mobo)

next step is to finalise DRAM attachment system on the motherboard.

DRAM prices so bad that I’m building a server rack (in fusion)

The goal is to build a full 4U server rack, scale it down and 3D print it. Then, print a full shelf of racks. Keep going and you get infinite compute.

What I’ve done:

1. Fully modelled GPU (design adapted from 5090/RTX Pro 6000), M.2 SSD, SFF and Network Adapter Cards, Power Supply
2. Partially modelled Dual-CPU EATX Motherboard.

Every part has been modelled by hand to reference materials. As the end result will be scaled down, many parts have been widened to accomodate that.

Thank you for your support!

qubixal memory technologies (qxmt) prints its first 3x downscaled DDR5 ECC RDIMM at 32GB capacity using 32Gb modules

after some time thinking about it, i decided at this scale it was better to take the two piece locking mechanism design of a typical dimm slot to one piece (no reference intended).
this was because of:

1. easier printing, 2. easier to produce consistently 3. easier designing
   and it works! after about ~5 prototypes (i think?) the dram slot works. now i think the only thing left is to create the CPU latching mechanism.

image: a rough diagram showing flaws (oh yeah first design also bent because of unstable base because the ram stick would push the unevenly extruded latching arms out)

qubixal memory technologies (qxmt) prints its first 3x downscaled DDR5 ECC RDIMM at 32GB capacity using 32Gb modules

after some time thinking about it, i decided at this scale it was better to take the two piece locking mechanism design of a typical dimm slot to one piece (no reference intended).
this was because of:

1. easier printing, 2. easier to produce consistently 3. easier designing
   and it works! after about ~5 prototypes (i think?) the dram slot works. now i think the only thing left is to create the CPU latching mechanism.

image: a rough diagram showing flaws (oh yeah first design also bent because of unstable base because the ram stick would push the unevenly extruded latching arms out)

i swear this is the last devlog even tho i shipped already…

v0.2 release

Bug Fixes:
issues #1 #2 on github
#1: wallpaper capture only works for sometimes (due to beta feature)
wallpaper capture is currently a beta; it uses NSWorkspace.shared.desktopImageURL(for:) to get the URL for the wallpaper, then prompts access to your folder where the wallpaper is actually stored even with sandbox disabled.

This only works for static wallpapers. Apple built-in dynamic / slideshow / video wallpapers will NOT work.the API has also been depreciated since macOS 14.
(This also looks incredibly suspicious when it’s just retrieving an image).

Fix 1: use CGWindowListCreateImage to take a screenshot of the wallpaper/desktop and then theme from that, but that requires screen capture permissions + couldn’t get it working
Fix 2: feature removed in favour of using desktop system accent + light/dark theme.

#2: window layouts switch sometimes.
fix: profile editor puts row 0 at bottom (based on +/- grid system). but absoluteFrame (LayoutProfile.swift:25) applies row 0 to the top of the screen. switched indexing to be correct way.

Theming is now determined by system set accent colour + light/dark theme to avoid requiring any additional permissions.
some redundancies removed.
New dmg installation background.

Finished up unisnap and the proper README docs (formatting issues previously). You should now be able to follow through on that README and install it for yourself with no problem!

A few things worth mentioning that I didn’t get into in my previous devlogs:

unisnap just utilises the AXUIElement API, reads the window position/size/title and writes them to match the set layout. it is also that coordinate math that caused most of the pain for me.

unisnap was designed to be ultra-compact, taking up the least possible footprint while maximising your productivity. That’s why I decided to put it in your toolbar, so that you can access it anytime without occupying a large window to configure all the settings/profiles etc.

The drag-to-create custom profile editor is where the 3727+4 (custom+default) possible layouts number comes from. It is max 1-4 columns × max 1-3 rows, and accounting for every combination + merged cell you get 3731.

Wallpaper theming is a beta. It’s not really that big of a deal either as this project has limited UI.

Known limitations/bugs include:

• no multi-monitor support
• unsigned app (so macOS will quarantine it); there’s a one-line terminal fix in the README.

It is available as proper .dmg release (or you can download v0.1 zip version).

unisnap - a lightweight macOS toolbar utility for window management.

now available on github for v0.1 (macOS only…)
here -> https://github.com/qubixal/unisnap

I wanted to solve a long time problem I had. In Windows 11, there was a set option where dragging the window above the screen could reorganise them into halves/quarters of the screen. This is doable with raycast, but it takes long (manually config each window). So, I made:

unisnap!

1. No need to manually config desktop layout!
2. No more manual reconfig required when you suddenly want another app in a different place!
3. Basically fully customisable everything!
4. No AI slop integration / cloud / account / selling your data, 100% open source!

There are:
3731 possible custom layouts for you to try!

hope this helped anyone? or just me lol

notdev log #2
basically finished every major hardware component.
worked on:

1. fan exhaust system + mountable radiator
2. adjusted sff->pci splitter board dimensions to fit
3. added PSU adapter/expansion boards (-> 8pin pwr/24pin mobo)

next step is to finalise DRAM attachment system on the motherboard.

devlog 3:
it’s been even longer but I’ve finally done enough to call this a “devlog”.

basically everything has been fully rewritten in python. A few (quite detrimental) bug fixes are:

• nonexistent error recovery system (i.e. pausing benchmark + invalid responses)
• scoring rubric weights ignored in prompt sent to judge
• save files mislabelling
• A/B scoring only having one slider for better model
• several UI patches (floating navbar -> broken margins)
• added leaderboard from the original index.html currently visible on the github under “Leaderboard”, displaying top scores.

To work on:

• largely inconsistent and seemingly inflated scoring
• check if benchmark can actually be completed, finalise benchmarking criteria + process
• benchmark the rest of the models
  i hope this can be done within 2 more devlogs.

Image 1:
much more simplified run system (compared to before).
Image 2:
the og graph, now in v2!