@TheShovel

Hi! so, i am making this devlog with 15 hours on record because i was honestly not sure if this would work. In my last stardance project, i tried making an agentic chatbot utilizing LLMs, but trying to make it as light and as compatible as possible so it can run even on my poopy 7gb of ram laptop.

That whole thing got me thinking. Do we even need LLMs? Do we even need generative AI for this?

So this project started. This is an attempt at making agentic AI without LLMs, or weights or anything similar. The entire idea is that this can be fast and easily modified without having to use an entire datacenter to retrain the model, because there is no model.

Right now, i have worked on a conversational “engine”, that basically puts words together algorithmically by detecting certain properties from the users prompt. Its not very impressive right now, but i think im onto something that could be quite a big deal if i get right. I dont even know if this project will be shippable but ill still document its development because i think its interesting. Alright smooches! See you later! <3

just a small patch related to memory usage, nothing more :)

though, the video is ready! im just waiting for it to get approved for the hackclub video thing and then im gonna post it! Heres the thumbnail

i think i am finally done! I have optimized, refined stuff and tested a bunch of random things on the lowest model possible, that still has some kind of a intelligence. Nothing really too interesting to talk about, but i DID get it working on my 7gb of ram old laptop! This is the first local chat app that actually works on that thing, so thats pretty damn cool me thinks. Working on a youtube video right now, im proud of the project and i want people to find it and contribute to it. In the meantime, until i finish that video, here is icanhelp writing a document about the importance of vitamin C!

finally a devlog where not everything was a colossal failure! As i said in my previous dev log, whats left to do is tools and refinement, and thats exactly what i did. The search tool has been updated to let the llm browse websites and their contents, which improved the quality a lot. also, i have implemented an addon system that lets you install external smaller llms to do tasks that are then outputed to the bigger llm. llmception. decided to get rid of the rag since with the improved web search its no longer needed. Besides this, all i did was little tweaks, refinement refinement testing refinement. As one smart video once said: “solution problem, problem solution problem.”

well well well. This project has been the most frustrating back and forth of my life. It felt like 3 steps forward, 2.5 steps back, CONSTANTLY. Well, after hours and hours of testing, refinement and research, i have found the solution…

I have decided to change my approach in how i handle responses. Now, instead of having 1 model that does everything, we have 2 super tiny ones.

The flow is kinda like: user types prompt -> small llm analyzes prompt -> small llm returns the instructions for how to solve it using the tools it has -> if it decides that the complexity is too high, then it lets the larger llm continue -> system does knowladgebase + web search -> the chosen llm presents the data to you.

This is REALLY fast as you can see, and quite reliable as well. Right now the answer quality is not exceptional, but thats just about refinement and gradual improvements. So hopefully in the next devlog i have some really cool stuff to show you, like graphs, visual explanations, letting the user choose a website to summarize and so on.

Along side working on the app, i worked on the website for it! Its still a work in progress but im proud of how it turned out :D
https://theshovel.rocks/icanhelp/

ok well, anyway, thats it for now. see you next time in “im going crazy because lobotomy ai doesnt want to follow my instructions”

well, today was a disaster. 8 ours that didnt really account for much to be honest :(

I have tried to move from qwen to minicpm 1b, a model that ALLEGEDLY is a lot better at using tools to gather data rather than using its internal baked in weights. Well, apparently thats not true. I have spent hours making the tools, adapting the outputs and everything for that model, just to then realized that “wow people are wrong and im so tired of ai bros hyping stuff up for nothing!”. So, in the end, i reverted to the qwen models, since they are decent enough for this. Even the 0.8b model gives better results than minicpm…

Well, at least thats not all i did. After reverting to a older commit, i have re implemented all the improvements that dont have to do with minicpm. You can now see the sources of where the llm fetched data from under the response bubble, and you can click them to open up the pages and double check the information and stuff. Also, i have worked on large file handling. Previously, for large files it would try to read the entire thing at once. That is insanely inefficient for large files, so i have added a new tool that lets it read from line x to line y. Like that it can only retrieve relevant stuff and not flood its context. I have ALSO worked on compatibility. Now you can change between cpu and gpu inference. If you have under 6gb of vram, the gpu option is grayed out. You might say “thats a little overkill”, but you have to take into account that context also eats up vram, and quite a lot as well. So now you can use the cpu, which uses your system ram, to reliably have long conversations. Sadly in the demo it ended up crashing the app and freezing the process with a node api error, but after i killed the process and reopened the app, the cpu inference worked fine. Thats a small oversight i didnt take into account where llamacpp is still doing stuff while im trying to kill the node process.

Well, thats it! See you next time in “trying to make lobotomy ai useful so people stop using giant datacenters to ask about how long to boil and egg for”

well, i am going to sleep for a few days. The last 24 hours have been tough.

First of all, i started getting harassed for making an ai project in my friend groups, which just broke my heart. I got actively attacked and called names for just wanting to make something cool. The point of this project is to make a completely local, light and fast ai assistant for your pc. I dont like the big corporations that do harm economically and environmentally. That is exactly why im making this.

With that out of the way, i have spent way longer than 6 hours and a half since the last devlog, bug a lot of that time was not coding, it was data stuff. I have started to build a knowledge base by asking local larger ai that run slower, to write information and documentation for different topics (linux, programming, science, health etc). Right now i have 10 categories with a total of 463 files. Then, i run an embedding model to turn those into searchable vectors that the ai can use during conversation. In the video bellow, i asked it how bash commands work in linux, and it found all the knowladge for that, and started telling me about it. Normally, the quality of the response is pretty short, vague and not very precise, since the model is so small, but now, it actually gives some usable output.

I have also worked a lot on making this as easy to install as possible. I have made a install script that covers all dependencies you need, registers the app on your system, and also lets you choose what knowledge categories you want to be included in the embedding process, which also happens locally during install. Yes that means all that knowledge if free and open source in the repository. If you want, you could even make your own categories, or improve existing ones!

To make this easier to use, i have also worked on the UI. Now theres a built in light and dark mode that you can use, along side whatever themes you ask the assistant to make for you. You can also now open all of the apps data folders. You can change whatever you want in there, and even customize your buddy! Would be cool to see some user made designs with like miku or something. That would be so cool :3

Sadly, in the process, i also broked the vision model stuff and i just realized that when i recorded the video for this devlog. I am too tired after doing data training for an entire day to debug and fix that right now. Also pretty demotivated after the stuff i had to endure lately for making projects based on ai.

I want to remind you that most of this project is made by me, and like my other project for stardance, Irongingot, i will make a graph with all of the files that were touched by ai, how many lines it wrote and what % is ai in the readme of the repository, using my opencode history on this project. I am a human being making stuff for human beings.

Anyway, pretty grim devlog i know. Hopefully, the next one will be happier and i can show some more cool stuff i have planned, like real time interaction with your desktop and select text and ask and so on.

Thank you to everyone who has been kind to me on here <3

Since i made the first ship, i have gotten A LOT of bug reports, from the same guy. I want to thank this guy for actually using this and reporting all of these bugs!

I still have not fixed all of them. there are a few open ones, but ill get to it eventually. The feedback for this project has been phenomenal and it has made me very happy !!

I will keep working on it, adding features from time to time and fixing bugs. It wont be my main priority but ill keep working on it as much as i can :)

Thank you!

well well well…
turns out cloud AIs are expensive and dont work offline. Also not very accessible for everyone to use!!!

So, i have decided to make all of this run LOCALLY!!! Thats right, instead of making you put in an api key to whatever provider you wanna use, you instead get to choose from a list of qwen 3.5 models. It auto detects your hardware, and displays all of the models that work on your system.

Now, obviously, this is less high quality than the cloud models. In the previous devlogs i used deepseek v4 flash from opencode, and it was way more competent. Here, i used the 2B version of qwen 3.5. With that said, even the 0.8B model is still fairly capable, considering that deepseek v4 flash is over 200b parameters.

I worked a lot on optimizing this. I added chat history compression, so instead of it remembering everything in the conversation precisely, it makes a summary within a token budget instead, to keep system requirements low even for the bigger models. More context tokens = more ram, and who the hell has ram nowadays, am i right gamers.

Along with that i made a bunch of small tool and ui improvements. Still a work in progress, but at least the UI is not vibe coded slop for the most part anymore.

Next up im going to try and work on a theory that i have. What if i give the small 0.8B model, access to a dataset, and a search engine… Could i possibly make this super tiny model really smart? I dont know. Saw a youtube video about it but it was just a theory, not applied practically. But since im here might as well try it out.

Oh also i renamed the little guy to Icanhelpy because i think its very cute

I got a lot done.

Added OCR with tesseract and then i realized it sucked so i switched to BLIP. Runs in a child process so it doesnt freeze the app.

The buddy icon animates now based on whats happening, pulses tilts shakes spins depending on state. Also i drew some nice little assets for it :3

Moved the effort dropdown and model selector to the top bar. Settings has sub menus, theme panel lets you browse saved themes and apply or delete them.

The AI can change css now through a set_theme tool. Colors gradients images whatever. It can save named themes and they persist across restarts in the encrypted config. Had to consolidate the theme tools cuz the llm request got too big.

Screenshots took forever. desktopCapturer asked for permission, imagemagick import has broken syntax, mss gave black screens on wayland, grim doesnt work on kde. Finally spectacle works. Falls back through a chain of tools and tries to auto install imagemagick with a sudo popup if nothing is found.

The attach button shows a file and screenshot popup, hides after picking, comes back when you send or remove. Attachment label is a clickable pill that opens files.

Fixed the llm re analyzing already attached files. Fixed bubble overflow on long lines.

Added chat history, cancel stream, sudo password popup, markdown, web search, file tools, right click menu, window position persistence, burger menu navigation blah blah blah this is getting soooo long bro i dont wanna type no more and a readme.

Still need to add proper security beyond the sudo prompt and encrypted config. Gonna work on that next maybe, but thats pretty boring stuff so idk…

also sorry for the video quality i just realized my settings were wrong right as i uploaded this. woops

I have started work on this thing!
So far, I have implemented some placeholder ui, a way for you to register your LLM provider, cloud or local, and some basic tools!

Copy pasted a lot of code from some of my other projects, so this didnt take too long to do.

The thing that took me the longest was to find a method to detect when a console command that the llm runs would return a sudo prompt, so i can prompt the user through the UI to enter their password. So far, thats also the only security thing i have in place. Will work on more of that.

The UI is very ugly and clunky right now because its just some css i vibe coded. Trust me i know its ugly ill work on making it better myself once i get everything backend related done.

Hiiii Hope your day is going well and stuff :)

Today i have added widgets! You can now right click on the desktop, and generate widgets that stay where you left them even when you restart! They have access to the theme and system apis just like any other app, but they are made to sit comfortably on your desktop.

I have also been working on some quality of life and introduction features! Now there is a user manual, that you can access through the welcome screen. It tells you everything you need to know to use the OS!

I have also added some stuff to the settings, like organizing your right click menu, factory reset and other misc stuff that is minor and more to polish things up. Oki thank you byeeeee :D

sup gamers, its me, theshovel, back at it again with a devlog for the worst operating system ever made

i have worked on the speed and quality of the app generating. I have also implemented more system apis like more customization stuff like light and dark mode, also some sound and storage apis so the apps can use local storage.

I also worked on the visuals a bit, making it more consistent. The apps now have a design that they follow. This allows for apps to change theme when you change your accent color or between dark and light mode.

The major issue right now is that im using a free google api for the app generating. Due to that, i have to lower the maximum output tokens so the user doesnt get rate limited. Due to that, apps might end up being simpler than you expect. The model behind it is still robust enough to give some pretty good output for that tiny amount of token allowence, but even then it might be simpler than youd expect.

I have started the project!
So far, i have gotten the bare minimum working. You can click the menu icon, you can type in the name of the app, and a free api endpoint running qwen coder will build the app for you.

Because I didnt want to be lazy and just use iFrames, i made it create a div inside the window with the generated code. This works, but it could be a security issue. Do i care though? No lmao

I have also started implementing some apis, like the notification api, which an app can call and it will show you a notification.

Im going to continue working on this by adding more custom apis and making the operating system more fleshed out. Probably will also have to write some more examples for the llm to use, right now there are a few but they are very simple.

and so… with this last devlog… it ends.

I will be honest, I am burnt out! So, i worked on getting a last stable
build that makes survival viable on this server. Tons of testing and bug
fixes went into this, and i worked constantly on it until burnout. It’s still not perfect. There’s some random bugs. You might get kicked randomly, but you should be able to get right back in every time. Hopefully. Gulp.

Since the last devlog, i implemented so much stuff, i will just link
the release here so you can read all of it!!

https://github.com/TheShovel/irongingot/releases/tag/V1.6.1

I still am missing some features i wish i would have added. stuff like
security improvements and a basic anti cheat but sadly, i gotta move on
to something else.

Thank you for being with me on this journey though! Was really fun!

Heres a video to showcase how far
i have come since i forked the original project on github :D

hiiii!! basic functionality is done!
you can now prompt it, and it will run a free small llm to decide the complexity of the task, and then it will make the actual llm do the task!

saves tons of money and stuff ypeeeee

gonna work some more on it cuz i actually want to use this. im making something i need right now lmao

heyyyy!!! lots since last time.
got the recipe book working, implemented item durability, underwater breathing, improved trading by making villagers have professions and levels and actually opening up the trading ui, but most importantly, a big change in the packet code that makes multiplayer incredibly smooth now. i am honestly so tired i cant be bothered to type every single thing i had to go through to get here, but oh my is this project getting more and more complex ish. oh also added critical hits forgot to mention that