By
@imrxne1
A chat, coding, image and diagram assistant running on one free Oracle VM — two ARM cores, 11.9 GB RAM, no GPU. Routes between Groq, OpenRouter and local Ollama so one channel being rate-limited is a degradation, not an outage. Includes accounts with two-factor auth, a sandboxed Python runner, web search, file and image reading, and an installable PWA.
A chat, coding, image and diagram assistant running on one free Oracle VM — two ARM cores, 11.9 GB RAM, no GPU. Routes between Groq, OpenRouter and local Ollama so one channel being rate-limited is a degradation, not an outage. Includes accounts with two-factor auth, a sandboxed Python runner, web search, file and image reading, and an installable PWA.
2h 56m 50s logged
need a free ai to help ya? gotchu checck random generals AI free and a good companion and also it was a pain in the abs to make, still fixing some bugs but pliz make sure to check it out :)
6h 53m 2s logged
groq rate limit almost killed my whole app
so i built randomgenerals.com. its an ai assistant, chat + code + image
gen + diagrams. runs on a free oracle vm, 2 arm cores no gpu. flask app
that does the website, the pwa and the backend for the desktop version.
accounts, credits, emails, dashboard, all of it.
ok so the bug.
groq free tier is 8000 tokens per minute. not per request. per minute.
for the entire key. every single user on my site sharing that. and it
counts your prompt AND the reply you asked for as one number together.
i wasnt checking that anywhere lol
so if a chat got long enough (like 5400 tokens) it would just stop
working. forever. you hit retry and it builds the same too big request
and gets refused again. same error every time. only way out was starting
a new chat and nobody knows to do that.
my first idea was just make the replies shorter. bad idea. thats the
thing people actually came for. so now it drops old messages first and
only shortens the reply if that wasnt enough. forgetting what you said 6
messages ago is annoying. not getting an answer is worse.
then i found another one. i was never checking finish_reason so when a
reply hit the token limit it just stopped mid sentence and looked
completely normal. and the api returned 200 every time so my logs were
clean. no clue how long that had been happening ngl
126 commits so far. randomgenerals.com
