icanhelp

i think i am finally done! I have optimized, refined stuff and tested a bunch of random things on the lowest model possible, that still has some kind of a intelligence. Nothing really too interesting to talk about, but i DID get it working on my 7gb of ram old laptop! This is the first local chat app that actually works on that thing, so thats pretty damn cool me thinks. Working on a youtube video right now, im proud of the project and i want people to find it and contribute to it. In the meantime, until i finish that video, here is icanhelp writing a document about the importance of vitamin C!

finally a devlog where not everything was a colossal failure! As i said in my previous dev log, whats left to do is tools and refinement, and thats exactly what i did. The search tool has been updated to let the llm browse websites and their contents, which improved the quality a lot. also, i have implemented an addon system that lets you install external smaller llms to do tasks that are then outputed to the bigger llm. llmception. decided to get rid of the rag since with the improved web search its no longer needed. Besides this, all i did was little tweaks, refinement refinement testing refinement. As one smart video once said: “solution problem, problem solution problem.”

well well well. This project has been the most frustrating back and forth of my life. It felt like 3 steps forward, 2.5 steps back, CONSTANTLY. Well, after hours and hours of testing, refinement and research, i have found the solution…

I have decided to change my approach in how i handle responses. Now, instead of having 1 model that does everything, we have 2 super tiny ones.

The flow is kinda like: user types prompt -> small llm analyzes prompt -> small llm returns the instructions for how to solve it using the tools it has -> if it decides that the complexity is too high, then it lets the larger llm continue -> system does knowladgebase + web search -> the chosen llm presents the data to you.

This is REALLY fast as you can see, and quite reliable as well. Right now the answer quality is not exceptional, but thats just about refinement and gradual improvements. So hopefully in the next devlog i have some really cool stuff to show you, like graphs, visual explanations, letting the user choose a website to summarize and so on.

Along side working on the app, i worked on the website for it! Its still a work in progress but im proud of how it turned out :D
https://theshovel.rocks/icanhelp/

ok well, anyway, thats it for now. see you next time in “im going crazy because lobotomy ai doesnt want to follow my instructions”

well, today was a disaster. 8 ours that didnt really account for much to be honest :(

I have tried to move from qwen to minicpm 1b, a model that ALLEGEDLY is a lot better at using tools to gather data rather than using its internal baked in weights. Well, apparently thats not true. I have spent hours making the tools, adapting the outputs and everything for that model, just to then realized that “wow people are wrong and im so tired of ai bros hyping stuff up for nothing!”. So, in the end, i reverted to the qwen models, since they are decent enough for this. Even the 0.8b model gives better results than minicpm…

Well, at least thats not all i did. After reverting to a older commit, i have re implemented all the improvements that dont have to do with minicpm. You can now see the sources of where the llm fetched data from under the response bubble, and you can click them to open up the pages and double check the information and stuff. Also, i have worked on large file handling. Previously, for large files it would try to read the entire thing at once. That is insanely inefficient for large files, so i have added a new tool that lets it read from line x to line y. Like that it can only retrieve relevant stuff and not flood its context. I have ALSO worked on compatibility. Now you can change between cpu and gpu inference. If you have under 6gb of vram, the gpu option is grayed out. You might say “thats a little overkill”, but you have to take into account that context also eats up vram, and quite a lot as well. So now you can use the cpu, which uses your system ram, to reliably have long conversations. Sadly in the demo it ended up crashing the app and freezing the process with a node api error, but after i killed the process and reopened the app, the cpu inference worked fine. Thats a small oversight i didnt take into account where llamacpp is still doing stuff while im trying to kill the node process.

Well, thats it! See you next time in “trying to make lobotomy ai useful so people stop using giant datacenters to ask about how long to boil and egg for”

well, i am going to sleep for a few days. The last 24 hours have been tough.

First of all, i started getting harassed for making an ai project in my friend groups, which just broke my heart. I got actively attacked and called names for just wanting to make something cool. The point of this project is to make a completely local, light and fast ai assistant for your pc. I dont like the big corporations that do harm economically and environmentally. That is exactly why im making this.

With that out of the way, i have spent way longer than 6 hours and a half since the last devlog, bug a lot of that time was not coding, it was data stuff. I have started to build a knowledge base by asking local larger ai that run slower, to write information and documentation for different topics (linux, programming, science, health etc). Right now i have 10 categories with a total of 463 files. Then, i run an embedding model to turn those into searchable vectors that the ai can use during conversation. In the video bellow, i asked it how bash commands work in linux, and it found all the knowladge for that, and started telling me about it. Normally, the quality of the response is pretty short, vague and not very precise, since the model is so small, but now, it actually gives some usable output.

I have also worked a lot on making this as easy to install as possible. I have made a install script that covers all dependencies you need, registers the app on your system, and also lets you choose what knowledge categories you want to be included in the embedding process, which also happens locally during install. Yes that means all that knowledge if free and open source in the repository. If you want, you could even make your own categories, or improve existing ones!

To make this easier to use, i have also worked on the UI. Now theres a built in light and dark mode that you can use, along side whatever themes you ask the assistant to make for you. You can also now open all of the apps data folders. You can change whatever you want in there, and even customize your buddy! Would be cool to see some user made designs with like miku or something. That would be so cool :3

Sadly, in the process, i also broked the vision model stuff and i just realized that when i recorded the video for this devlog. I am too tired after doing data training for an entire day to debug and fix that right now. Also pretty demotivated after the stuff i had to endure lately for making projects based on ai.

I want to remind you that most of this project is made by me, and like my other project for stardance, Irongingot, i will make a graph with all of the files that were touched by ai, how many lines it wrote and what % is ai in the readme of the repository, using my opencode history on this project. I am a human being making stuff for human beings.

Anyway, pretty grim devlog i know. Hopefully, the next one will be happier and i can show some more cool stuff i have planned, like real time interaction with your desktop and select text and ask and so on.

Thank you to everyone who has been kind to me on here <3

well well well…
turns out cloud AIs are expensive and dont work offline. Also not very accessible for everyone to use!!!

So, i have decided to make all of this run LOCALLY!!! Thats right, instead of making you put in an api key to whatever provider you wanna use, you instead get to choose from a list of qwen 3.5 models. It auto detects your hardware, and displays all of the models that work on your system.

Now, obviously, this is less high quality than the cloud models. In the previous devlogs i used deepseek v4 flash from opencode, and it was way more competent. Here, i used the 2B version of qwen 3.5. With that said, even the 0.8B model is still fairly capable, considering that deepseek v4 flash is over 200b parameters.

I worked a lot on optimizing this. I added chat history compression, so instead of it remembering everything in the conversation precisely, it makes a summary within a token budget instead, to keep system requirements low even for the bigger models. More context tokens = more ram, and who the hell has ram nowadays, am i right gamers.

Along with that i made a bunch of small tool and ui improvements. Still a work in progress, but at least the UI is not vibe coded slop for the most part anymore.

Next up im going to try and work on a theory that i have. What if i give the small 0.8B model, access to a dataset, and a search engine… Could i possibly make this super tiny model really smart? I dont know. Saw a youtube video about it but it was just a theory, not applied practically. But since im here might as well try it out.

Oh also i renamed the little guy to Icanhelpy because i think its very cute

I got a lot done.

Added OCR with tesseract and then i realized it sucked so i switched to BLIP. Runs in a child process so it doesnt freeze the app.

The buddy icon animates now based on whats happening, pulses tilts shakes spins depending on state. Also i drew some nice little assets for it :3

Moved the effort dropdown and model selector to the top bar. Settings has sub menus, theme panel lets you browse saved themes and apply or delete them.

The AI can change css now through a set_theme tool. Colors gradients images whatever. It can save named themes and they persist across restarts in the encrypted config. Had to consolidate the theme tools cuz the llm request got too big.

Screenshots took forever. desktopCapturer asked for permission, imagemagick import has broken syntax, mss gave black screens on wayland, grim doesnt work on kde. Finally spectacle works. Falls back through a chain of tools and tries to auto install imagemagick with a sudo popup if nothing is found.

The attach button shows a file and screenshot popup, hides after picking, comes back when you send or remove. Attachment label is a clickable pill that opens files.

Fixed the llm re analyzing already attached files. Fixed bubble overflow on long lines.

Added chat history, cancel stream, sudo password popup, markdown, web search, file tools, right click menu, window position persistence, burger menu navigation blah blah blah this is getting soooo long bro i dont wanna type no more and a readme.

Still need to add proper security beyond the sudo prompt and encrypted config. Gonna work on that next maybe, but thats pretty boring stuff so idk…

also sorry for the video quality i just realized my settings were wrong right as i uploaded this. woops

I have started work on this thing!
So far, I have implemented some placeholder ui, a way for you to register your LLM provider, cloud or local, and some basic tools!

Copy pasted a lot of code from some of my other projects, so this didnt take too long to do.

The thing that took me the longest was to find a method to detect when a console command that the llm runs would return a sudo prompt, so i can prompt the user through the UI to enter their password. So far, thats also the only security thing i have in place. Will work on more of that.

The UI is very ugly and clunky right now because its just some css i vibe coded. Trust me i know its ugly ill work on making it better myself once i get everything backend related done.