@nosrep

i’ve reentered the nightmare

when i last left off this project in ~september, I basically hit a wall with optimizations. I had a crapload of pruning and whatnot, but the elo of the engine didn’t really reflect it. I then did a search rewrite but got stuck on RFP (reverse futility pruning), which is supposed to gain a lot of strength but did basically nothing for me.

brief (??) aside: reverse futility pruning, in simple terms, checks if the static evaluation (evaluation without searching moves) is above beta (if a move scores above beta, it’s “too good,” and the opponent won’t allow that position to happen in the first place) by some margin. if it is, it’ll return without searching any moves, because it’s not likely that the player can lose that much material/positional advantage in whatever depth is left to search.

anyway, i’m still stuck on this. I’ve re-reimplemented half the search to recheck its correctness, because correctness issues can sometimes stuff new optimizations, but it seems to be all good. RFP still doesn’t do jack.

i don’t really know where to go from here. I could just ignore it, but the fact that RFP doesn’t do anything (for reference, another engine dev i talked to got like 80 elo out of RFP, which is insane) is a troubling sign and might mean something else is wrong.

hello! trout again. i’m gonna try to get this to #1 haskell engine, but it’s been such a pain…

after summer of making I did some more work on this, but hit a wall where all my changes either did nothing or gained like 10elo (which is not enough for the hundreds i need). I tried rewriting part of search but hit a wall again with the same problem, on features that should have made improvements!

i’ve been fighting with the search some more trying to get reverse futility pruning to do something meaningful. in someone else’s engine, it gained i think 80+ elo in total (!!!), which is bonkers compared to the 20 or something i get on a good day. it’s not a good day, so RFP actually manages to lose elo here.

i might have to rewrite search again. i don’t know how much of this is just wrong.

(here’s the commented out old search)

I made some duct modifications and now cooling seems to be much better. I think the new duct is choking the airflow a bit too much, but it’s more focused and can actually reach the spot right under the nozzle (which the old one seemed to struggle with?).

I also did a LOT of testing, didn’t track most of it. Probably could benefit from a little more tuning, but we can do the pin support sphere on fullcontrol.xyz now!

fixed some of the lingering issues, like the bltouch interfering with the duct and the bltouch interfering with the endstop (it’s always the bltouch…). we should be done now

it has been built! it is so loud!

I finished designing the ducting and finished printing out everything, immediately a couple problems: ducts bump into bltouch (this was also in the cad but i didn’t notice) and the ducts are really flimsy. The ducts themselves are fine but the front plate is too thin to keep it in place properly. They work but it’s kinda sketchy. To fix the bltouch thing I just melted the ducts with my soldering iron, already remodeled a fixed version but have yet to print.

The extruder seems to be not tuned very well. I set the manufacturer-recommended esteps, but it could be off. The cooling also seems to be sketchy? I can feel an insane amount of airflow but i don’t know if it’s going to the part yet. It aims a little low and I reprinted a bridging test I made earlier and the results seem to be worse.

I also need to do some fixes later if i want to use this long-term. I couldn’t make a new extruder cable because my crimping tool couldn’t handle jst-ph, so i’m gonna have to deal with the stock extruder cable which severely limits the amount of vertical travel I can handle.

i think i finally finished the ducts…
these were nightmarish to make. onshape conveniently doesn’t have 3d sketches, so I did a horribly janky system involving planes and a routing curve. The path the duct had to take ended up being kinda strange, so the loft would constantly crash out because the profiles turned too sharply. i did eventually get the loft to cooperate, and the duct looks okay? it’s still very lumpy, but it should transition relatively smoothly, which is what’s important for airflow.
i’ve also started printing the parts now. as i’m typing this, the main mount is finished and the fan mount is almost done. after that i should theoretically be good to assemble once i print the ducts

(this is the video i used for duct design principles: https://www.youtube.com/watch?v=YijwkQCOBEA)

I have no idea how this devlog ended up so long
Anyway, I cleaned up some potential crashes/undefined behavior, fixed up the README, and finally tagged a release + published to AUR!
Spent a lot of time wrestling with static linking. I remember using old versions of linux before and getting glibc problems so I wanted to link it statically. I first tried using musl-gcc, but I ended up needing too many static versions of libraries so I just made a Dockerfile with alpine (which has static versions of basically everything) and made the binary there.
Did you know PKGBUILDs are just bash scripts? I didn’t.

the clipboard-as-files thing is added!
I originally wanted to use FUSE/libfuse, but i realized i could just change the storage format to have the data be in a single file. now i’m just using symlinks instead, no dependencies
this was relatively easy, ran into a couple of bugs but nothing too crazy. Biggest thing I got hung up on was which directory to put it in ($XDG_STATE_HOME? $XDG_DATA_HOME? ~/clipboard?). I eventually decided on $XDG_RUNTIME_DIR which seems mostly appropriate - the only thing is that it’s kinda annoying to access, but you could just make your own symlink, change the path with $ZZZCLIP_SYMLINK_PATH, or add a bookmark like in the screenshot.

Another thing was to add extensions (e.g. .png) to the file, because apps like discord won’t recognize them if they don’t have extensions. I was initially gonna take and process one of the giant lists online, but then i found out most systems have a /etc/mime.types so i parsed that instead.

I added some more options to the CLI, which ended up being a lot more annoying than i expected. Here are the ones I added:

added to zzzclip list:

• -v: verbose mode. Verbose mode also lists the mimetype for a clipboard item.

added to zzzclip get:

• -f: runs in foreground. Here’s something you might not have known: wl-copy starts a background process when you run it. The clipboard contents aren’t owned by the compositor, so a process has to be running to supply the data. I made zzzclip get do the same thing, but like wl-copy I added an option to make it run in the foreground.
• -s: output summary. It’s basically zzzclip list -v but only for one item.
• -l: list stored mime types. It lists the mime types for an item.
• -m : directly prints the data for an item.

big ticket is overhauling the config format. Previously, the index looked like this:

4a93d8d7 6b5d69df
2143c202

Each random string of 8 characters was a label for data under a particular mime type, and each line was a selection (a selection can have multiple mimetypes). Each label had its own file to store the actual clipboard data, with the mimetype included in that file.

this worked fine, but I’ve been wanting to implement the feature where the current clipboard contents are accessible via file. If I moved the mimetype out of the data files, I could just use symlinks because each file has exactly the data I need; using the old format, i’d need to use libfuse to make “fake” files without the mimetypes instead.

this was a lot more annoying than i expected (thank god for address sanitizer, caught many mistakes), but I eventually got it working. I think i’m nearing the finish line

lots of minor-ish stuff for this batch of changes.
Biggest change is actually adding proper cli parsing and help messages! previously, the cli was extremely bare-bones and just good enough to actually work but didn’t have any decent error handling or help messages.

I’ve also ditched the option that runs the selector program (e.g. rofi, fuzzel). I didn’t know about he -nth those programs have, which let you hide data from the selector, so that functionality was kinda redundant.
(also the sizes are human-readable now, i.e. in kb/mb/gb instead of just raw bytes)

the tests were bad. I fixed them and now everything passes. hooray!

Anyway, welcome to macro hell!
i’ve been working through the todo list and one of the items I wanted to hit was to support the ext-data-control protocol on top of the wlr-data-control protocol. Wayland compositors work on “protocols,” and not every compositor supports every protocol; the *-data-control protocols are what allow clipboard managers to mess with the clipboard. wlr-data-control was first (i think) and is only supported by wlroots-based compositors, but ext-data-control is the standard now. (except the compositor I use, river, doesn’t support it yet…)

The problem with supporting both is that C is not a very expressive language. I had to generate wrapper functions for every function supplied by the protocol, which was bad enough, but I also had to write wrapper functions for my own functions that were used in listeners. This part could have been so much worse, but I was able to hide some function pointers in the user data and cut down even more boilerplate.

Not fun. 3/10.

there’s always the boring janitorial stuff (e.g. i renamed a few files, moved some stuff around, fixed a mem leak and a potential crash, etc.)
the cooler change is that I finally added testing! testing an app like this is infinitely more annoying than something like a chess engine (shameless plug) because it’s built around IO. I followed in fuzzel’s footsteps and did integration testing in a shell script, which isn’t something i realized is practical until seeing it used there. the very sparse tests actually already did find a bug, but i’ll fix it later……
the testing is to set up more disruptive features i’ll add later, like implementing the ext-data-control protocol on top of wlr-data-control, and I want to make sure they won’t break anything without having to manually test

two things:

1. finally got rid of libpcre2. For the people that didn’t read the previous devlog, I needed a regex implementation as part of the user-provided mimetype selection, but I didn’t realize POSIX had one in regex.h and pcre seemed like the standard. POSIX regexes should be more than enough, though, so that’s a dependency killed
2. added locking and index refreshing. nothing was locked previously which is Not Good™️, but more importantly, the daemon never refreshed the index of clipboard data, so if i deleted one, the index would get rewritten anyway with the deleted file and corrupt it. Solution was to stop persisting the index and only read it when it’s used.

minor changes for this devlog. spent way more time than is reasonable thinking of the best way to deal with allocation failures… it seems like everyone online has a different opinion on it. I tried doing it “properly” before seeing how many lines of checks that would require for a codepath that would realistically never get hit, so I gave up and just did xmalloc (basically, a wrapper around malloc that exits with an error message when it fails).
Also in the theme of error handling, I did a sweep of the code and wrapped (most) of the io functions I missed to check for failure. it seems like overkill… there has to be a better way but maybe this is how it is.
Finally, I switched from libpcre2 to posix regex.h. I didn’t realize POSIX had a regex.h and it’s way more than is required for this use-case (matching mimetypes).

I started this project much earlier than stardance (it’s years old by now), but it’s only recently become usable. think of this devlog as motivation/background
the main goal was to make a clipboard manager that could

1. persist clipboard selections after applications closed
2. have configurable mimetype selection

Here’s something interesting you might not have known about linux clipboards (both wayland and x11): the data doesn’t actually exist until a client requests it. This is how applications like firefox offer 20 morbillion image types; it’s converting the image on-the-fly. Given this, it’s impractical to request every offered type, so most clipboard managers I’ve found solve this by giving up and only doing text or text+hardcoded image mimetypes. zzzclip has a probably-overcomplicated system that lets you configure which mimetypes it’ll remember and priority between mimetypes (plus regexes).
Another feature I want to add eventually is a FUSE clipboard system where it can make a “fake” (meaning it’s backed by the program, not something on disk) file holding the current clipboard contents.

(as for what these 17 hours are actually for: it’s just adding config parsing and a load of cleanups.)

made ducting setup and fine tuned fan mounts
actual duct geometry still gonna need a lot of work and maybe prototypes though

got the bltouch probing after much pain and suffering. needed to make new wires for this, duponts crimped okay but the crimper i borrowed was too wide to crimp the jst-xh properly. did it horribly and it seems sturdyish for now but it is incredibly sketchy and probably a fire hazard (i would be more worried if the bltouch drew more current). the plan was originally to remake the extruder wire, but after this i’m probably gonna have to hold off until i can get a proper crimper

reverted bltouch position. blower on the side looked too dumb so they’re both going on the front now. Blower mount still needs some refinement but should be basically done, then need ducts

switched out the bltouch mount position for the blower. struggled with making sane supports, still need to fix that