well that was a while debugging. I unfortunately spent more hours than I’d care on this.

this is a fork of a textmate-wakatime, a wakatime plugin for textmate, a macos editor.

Compared to the previous release, the modified v2.0.0 release builds on the 1.0.2 release (since the 1.0.3 release doesn’t seem to exist). It adds cursor position, the current time coded today under the File menu, adds OSX 10.9 support, removes legacy references to back when wakatime-cli was a Python project, fixes issues where Textmate autosaves within 2 seconds of a modification, destroying performance by spamming heartbeats.

that’s a lot of things. the weird thing about this project is that

1. it was very old. it doesn’t have configuration for stuff like a getter-setter-auto-synthesizer, but was compiled with the osx 10.10 sdk.
2. The rate limiting must have been different, since a write heartbeat happens everytime the user touches a key, and write heartbeats bypass rate limiting, causing a ton of heartbeats to be generated in a 3 second period
3. adding line number functionality wasn’t as fun as it was for xcode. Xcode’s editor is directly exposed, but you have to dig into the view hierachy in the window to search for a custom TextView and find the index of the cursor in there. It doesn’t just appear in the FirstResponder, which was annoying.

never the less, i got this working. The only reason I’m using Textmate is because it was on the mavericksforever.com website, and Webstorm and VS Code (at least the working ones for mavericks) are too old to work with Svelte 5, and Sublime text 3 is too ugly to use and doesn’t have a working LSP anyway. textmate isn’t even that bad of an option, considering there’s no way I’ll have LSP available.

if you still use TextMate on a version of OSX that doesn’t make a fuss about signed code injections, you should be able to use this! But it is a wakatime extension, so you probably don’t need to.

so it’s been a while, just barely 6h7m (676767676767676767). again… coding just gets me in the feels you know? you feel the breeze in the wind, the sun in the sky, the butterflies that release and change eddy currents to cause bit flips in a disc platter… just me? alright.

i added children traversal to the vfs. this was because I wanted my vfs to print out the output of all the items that got pushed through it. this also adds another feature that i took from Linux, which is that a given directory is connected to its children via a doubly linked list.

I actually took a feature from the Linux kernel: struct list_item. It’s a structure that has prev and next, which point to the addresses of struct list_item inside existing structures. So, for example

struct my_item {
struct list_item children;
struct list_item siblings;
}

would hold a prev and next pointer for each item. children would point to the first sibling, while the beginning of siblings would point to the original parent directory.

obviously though, my_item->children.next would hold a pointer to struct list_item, not struct my_item. this is where Linux’s container_of macro comes in (which i shamelessly copied word for word). it calculates the offset to subtract from different structures, given the structure and the name of the struct list_item.

this provides efficiency, since siblings and children are connected in a linked list that allows O(1) insertion and removal in a list where there’s likely to be a lot of removals.

my hashmap is also working pretty well!!! HASHMAP I WOULD USE A HASHMAP (if you get the reference LMFAO).

how much fun we shall have

so you may be wondering… where the hell was I for the past 6 hours. I was putting together the foundation for my VFS, with a ton of struggles.

what

so in my last devlog, I came up with all the storage structures and measurements for how the filesystem works on Astatine. But I was yet to implement a single function. So i started basic, and that still somehow screwed up too bad

1. Splitting paths by forward slashes

this seems really simple, but I messed it up somewhat bad. Since I don’t have a C library or strtok, I implemented my own string splitting thing using dynamic arrays (that I implemented) and a buffer. The unfortunate thing is that my dynamic arrays return a pointer to the value you save to it. Since I’m saving a pointer to a string, which in turn is a pointer of characters, I have to cast everything to a char**, while the default return types are all void*, so when I freed the strings, I forgot to dereference them (and the compiler didn’t complain), resulting in a kernel panic screen.

2. macro hell (in expansion of hashmap)

next, I need to implement a VFS global cache. when you access a path, it’s better to save the data in ram so that the next time you visit that path we aren’t traversing the directory again. unfortunately, i’m dumb and don’t know how to implement a hashmap, so I found this interesting tutorial from a decade ago and used that.

I also thought, at this point in time, that it would make sense to add thread safety to the vfs code so that it doesn’t completely crash due to race conditions, and I introduced macros for these. However, I realised that non-atomic writes are still not thread safe, and had to dig in with compiler builtins (since I’m using freestanding code, I can’t use <stdatomic.h>).

While putting together these macros with atomic, I had some weird behaviour where I kept hanging and hanging and hanging, and turns out it was because my macros were made improperly, and the formatting of the code meant that the while statements I was using to loop over code didn’t actually cover the entire loop body, resulting in a permanent hang.

3. malloc vs calloc

this is my fault, I forgot to use calloc to get zeroed memory, since my code seems to have failed to initialise all the required sections, it resulted in a page fault.

4. well this is embarassing

about at least 25 times, i kept a debug statement that dereferenced a null pointer, causing a crash. this was so sad because it taught me to remember: i’m stupid.

so does it work?

yea. thank my 2012 macbook air’s horribly patched version of qemu for the slowdown in the next clip. (i also included the crash so you can see the cool ntos-like crash screen)

after some deliberation i have some specifics as to the implementation of the virtual file system.

1. ive decided that i’m not smarter than engineers in the 50s, and am adopting a very similar idea to what the Unix kernel has: the i-node system (although i’m not calling them inodes that’s such an ugly name).

2. the kernel keeps track of 3 structures: fs_entry (the central identifier of the file, source of truth), fs_entid (what keeps track of the file name, parent directory, and what filesystem it belongs to), and fs_file (which is what processes interact with to read, write, mmap, etc. to disk).

3. we’ll then have a huge list of filesystem operations that use these 3 structures (as well as f_props to get properties of a file, dir_props to get directory information, sec_props, etc. etc.) to completely interact with the disk. This will allow us to make our own filesystem drivers and have interaction with physical media beyond fat32, something my operating system DESPERATELY needs.

ps. Maybe I’ll create my own read/write file system later.

well this was embarassing. turns out I forgot to clean the project, and stale files with the wrong idea of data had corrupt structures that made the kernel randomly crash. This was confusing for a couple reasons.

1. from the below screenshot, you can see that the kernel panic occurred inconsistently. the PCI device would register and memory would be allocated, but in other cases size was set to zero and stuff wasn’t very well allocated.
2. It turns out that my kmalloc function is bugged, and panics if you call it with zero (since it doesn’t do a precondition check and the loop always compares length with a min valid of 1)
3. Page faults were happening as well, which made me think a deeper memory corruption issue was happening. On top of changing the device structure to use an attributes pointer struct rather than an extension of the device itself, i allocated memory to copy that structure over, so I thought that early allocations were frying existing code.

all in all there’s a lot to learn from forgetting to clean stale files. now we can get back to the disk structure on hand!!!

ok so i added a bunch of metadata (at least whatever is supported under wakatime compatibility with the hackatime api, i would like to keep this compatible with all wakatime accessories.)

you can see that in the video below. courtesy of quicktime player.

I am utterly lost in space right now. So I wrote the syscall structure and now realised that I don’t really have a robust way of getting files. I’m using a stub and some random fat32 embedded driver online, which is certainly no “operating system”-like way to load files. I started working on my disk abstraction.

So now, I’m horribly lost. Well first, I made a couple of changes, all of which broke something random. I disabled optimisation for debugging purposes, and that changed the location of the entrypoint for some reason. I changed the way the bootloader is loaded up, and that broke the main partition for some reason. Something or the other keeps breaking.

After adding the abstraction code, I cycle between 3 errors. Either the code pagefaults while registering device, the malloc runs out of memory, or (a more random one) the disk driver fails to run. i’m taking a break.

ts is an extremely simple basic app to show the current time spent hacking on hackatime in the menu bar of your macbook air, pro, mac pro and mini, imac (pro), especially for 10.9.

clion 2019 doesn’t have a item on the status bar for wakatime and i got sick of not seeing how much time i code (to stroke my ego). i’ll add more features but its 12am and i just needed this done. I got 4 hours today, but then it became 12, and it reset to zero.

this is it for now.

it may not seem like much, but i’ve gotten to completely rewriting my syscall structure. how you may ask?

when it comes to system design, I want to separate Astatine from Unix’s own malpractices. Rather than going with the “everything is a file” paradigm, a concept that just squishes everything rather than make a decent api, I separated device and file nodes. You open with file_open and device_open.

On top of this, I added some better handling of flags, switches, and made some decisions that I thought would be better. Of course, I’m not going to throw out the 75-year-old works of Unix engineers, but I do want some creativity involved and Astatine-Is-Not-Unix (ainu?).

System design is probably the worst part out of any OS design: it feels so much better to start implementing, but sometimes, you need both DIRECTION and MAGNITUDE vector reference to work on code you’ll actually use. The almost 20 hours of research isn’t lost on me though. I hope I don’t have to take a Unix class in college.

oh my god the last 2 days have been like living in another hell. i wouldn’t wish this debugging session on my worst enemy.

so i wanted to implement an option on boot to choose how you want to display graphics on the screen (upcoming feature to give applications direct access to the framebuffer?). i made the smallest mistake of putting my strings in between the code, rather than all at the end. Because of this, nasm started doing random shenanigans that I still don’t know what happened and asked for help on the OSDev forums.

After like 5 hours on GDB and Hex Fiend and nasm and trying to get Gemini to do anything but hallucinate (hilariously so may I add), I figured out the issue was not putting all the program strings at the bottom of the code.

Now, I also have a configuration struct that allows stuff like boot-args, graphics config, and etc. to be configured as below. It looks decent as well

so i finally achieved the entire point of adding this second teletype driver. an important consideration to make in ANY computer is that you’re going to be working with different screens, different sizes, and different types of interfaces.

My old teletype code was only really optimised for VGA bios (even now, it can only render 16-color bios colors). But now, to change displays, all I need is to:

1. change the loaded driver from textmode.adv to bitmaptext.adv
2. change the loaded display device to VGA_BITMAP… rather than VGA_TEXT
3. change the boot program to initialise the display as 320x200

I plan on making only the 3rd option required through the boot program, the 1st and 2nd steps (should) hopefully happen automatically. And since we’re only dealing with ISA(80s)-compatible VGA cards, keeping the display device as an ISA device shouldn’t be that big of a problem (at least until we implement PCI discovery)

so i did try to hook up the bitmap into the new code but 1. i did hardcode 80x25 into the code so I have to fix that and 2. i did not implement a couple of functions and a couple of areas where it relies on quirks only available on text mode. you can see the result of relying on those quirks below in the poorly rendered and hardcoded error dialogue… more work will happen tomorrow.

i got all of the characters + a little bit of color handling code to complete the drawing. this will be part of the second teletype driver I write, although I still am not the biggest fan of this. my next step will be to write a bitmap driver arrangement that the teletype will then be bolted onto, since the current teletype ordeal with this will be just ad-hoc writes to memory without mounting to a specific display, it could possibly corrupt memory if loaded incorrectly (like the vga text mode driver). anyway ill git commit here so you can see

previously i was using a 7x7 font that worked, but it looks really ugly, looks bolded, and doesn’t really have a good lower-case variant. of course I could try to make one but I’m a god awful designer.

instead i stole this 5x7 character set from online specifically meant for bitmapped fonts on low resolution displays. 5x7 means that it won’t quite align perfectly, we do have a 53.6x25 display now (since its 6x8 with padding) but i just ignored the excess pixels and went along anyway. this looks much better, and I really do plan to use 640x480 since 320x200 is only usable for dos games and such.

i spent most of this 1h38m just putting in the character set into the character array, which I must say is quite relaxing.

continuing to work on the bitmapped character list, so far I have an A and a B imported from some random thing online. Unfortunately I can’t find an image online for lowercase 7x7 test, so I might have to design that myself, along with the rest of the special characters. At the very least, this foray is just to experiment with how my OS would interact with a bitmapped screen, I plan to use 640x480 basically as soon as possible (only reason I didn’t now is because its planar when set up via VGA, so I have to learn VESA)

Right now, I’m working on a bitmap display. I decided to go with a really basic starting point, just a 320x200 256-colors, and I’m programming the 256 colors as a 3-3-2 colormap. It took like an hour just to understand how this stuff works, I don’t like VGA a ton now.

holy crap there was so much debugging that had to happen. so the memory allocator pretty much fell apart. I realised that (with the help of gemini) that having a buddy allocator for physical frames was irrelevant, considering that physical memory could be remapped however necessary. as far as I know, there is no need for continuous physical memory except for legacy hardware, which i guess can be its own function later then. so I just made a bitmap allocator. physical memory is extremely irrelevant, we don’t care about anything other than the virtual memory it’s paired to. now I can finally start work on a bitmap display somehow.

made a piano in html5, basis of the app so far!