The grand settings, backend and UX update of ’26!

I added a bunch of backup logic behind the custom route fetching!
It’s working great now! I then also had to fix a bunch of things with the guest route, because that was all messed up from the settings stuff.

I added a new tool! The AI can now run endChat() to stop the user and delete the chat if they ask/say something harmful. I would like to stop it before it reaches my API provider, but I don’t think HC would appreciate me putting a “slurs.txt” file into my repo
When the AI calls the function, it fetches the current chat and simply deletes it. It also notifies the user through a simple error system I’ve made. From there, the user can currently create a new chat. I’m prepared to keep it this way for now, since I don’t want to limit other people from using their own local models for whatever they might want. I’m also considering providing the option to rely solely on the model’s guardrails.

I’ve just gotta hope nobody wants to test that feature upon shipping…
Speaking of which, I’ve massively changed my mind on how I’ll handle that! It’s NOT going to be hosted on-device for the ship, which I know sounds ironic (and frankly, is, given it’s a local AI client), but I don’t want any users saying “this local 0.5B AI is terrible, it must be a bad project!”, and will thus use Ollama’s official cloud API.
Also, if my Pi were to receive multiple prompts at once, it would likely… Die. This would A: suck, because I’d need to restart it frequently and B: prevent any users from rating it while it’s off. If it dies at midnight, I won’t know until I wake up!

Ollama’s Cloud API has seemingly good uptime (I couldn’t find any stats online, but it seems decent!) and provides me with much larger models, up to 120B (and likely higher) without having to pay. That brings me to one of the largest things I had to figure out!

If a user selects an unavailable model (Kimi K2.6, for instance), the server returns an error (which was surprisingly difficult to actually parse/decode!) and fails.
I created a system to return that the prompt failed, then check that the fallback model is different from the current model. If so, it warns the user, then goes to the fallback model and regenerates by itself, automatically. The default fallback is Gemma4:31B, since it’s a free Ollama model. I’m providing my own key, so users don’t need to enter their own.
If the fallback is the SAME as the current, it’ll show an unexpected error.

I also added auto scrolling, although I couldn’t figure it out, so I debugged using Gemini and managed to fix it.
It’s a ChatGPT-style scrolling system! It binds you to the bottom while it’s generating, but if you scroll up, it’ll sever that binding, letting you scroll freely. If you want to continue following the AI’s text stream, simply scroll back down!

I am considering adding a button to do so as well, since that could be a very easy feature to add and improve overall UX.

Oh right, I also fixed the displaying of when the AI uses tools! I found out that the Vercel AI SDK has a new system for it in the newest version and used that instead.

I think I added some more stuff, but I can’t really remember it all right now! Most of the time spent was debugging the things I broke while adding new features. I’ll add anything I remember as a comment on this devlog!

Also check the changelog if you wanna see more stuff (and specifics!) :)

I acknowledge that this is my longest devlog so far, but I just wanted to be thorough! If you have any questions, I’d love to answer them in the comments!!!