Ship Changes requested

@Oleksandr on BobAI · about 4 hours ago

BobAI — a chatbot on my site. A GitHub Pages frontend (with a click-to-reveal welcome screen) talks to a FastAPI backend hosted on Hack Club Nest. The backend calls Hack Club’s HCAI API (routed through OpenRouter, currently running Gemini 2.5 Flash) instead of running a model locally, and I added tool-calling so BobAI can search the web for things like current weather instead of only working off static knowledge.

I originally fine-tuned my own Llama 3.1 8B with DPO on Argilla’s UltraFeedback dataset, planning to self-host it. My RTX 4070 Super was more than capable of that part — the real problem was hosting a 5GB+ model for free. Hosting fell through repeatedly: Hugging Face Spaces dropped free Docker containers, Oracle Cloud registration failed three separate times, and Nest’s default 2GB RAM wasn’t enough — my request for 6GB got declined for server resource exhaustion.

So I switched to a smaller model that only needed about 1GB of RAM to run on Nest. That solved the memory problem, but the server only gave me 2 CPU cores under heavy load, and replies were unusable — minutes per response, even for a tiny test model. I had to dig into actual cgroup CPU pressure stats to prove it wasn’t just “the server being slow” — the hardware genuinely couldn’t keep up.

I’m proud that I shipped a working site through three full architecture pivots, and that I diagnosed the performance problem properly instead of just assuming “it’s probably fine” — I checked real metrics before concluding anything. I actually fine-tuned a model with DPO, even though it didn’t make it into the final version, and I picked up backend and frontend basics from zero along the way.

It’s live at oleksandr1dovbenko.github.io/BobAI — click anywhere on the welcome screen to jump into the chat. It runs on Hack Club’s shared HCAI API, which had some downtime earlier, but it’s working properly now. Try something time-sensitive, like asking for the weather, to see the web-search tool-calling in action.

5 devlogs
10h

Try project → See source code →

Open comments for this post

@Oleksandr on BobAI · about 4 hours ago

23m 1s logged

Final part

I edited the README.md, added a demo link to my BobAI repository, and have now prepared everything to ship the project!

Open comments for this post

@Oleksandr on BobAI · 2 days ago

3h 51m 43s logged

New project architecture again

I edited the project architecture. My request for 6GB of RAM was declined, primarily due to resource exhaustion on their servers :(. Because of that, the model that I fine-tuned myself is now useless for this setup. Now, my chatbot runs off the HCAI API. It’s a more boring way to do this project than making and hosting the model myself, but at least I can host it on Hack Club Nest easily!

The full website is done and working perfectly. There’s a small intro to the project, a link to the Git repository, and even a cool animation. I want to be completely open with you—I used Claude to help me with the frontend stuff because I never learned JS, CSS, or HTML.

The backend is also ready, and BobAI, powered by Gemini 2.5 Flash, works super fast and accurately. I also added tool calling, so BobAI can search for information on the internet just like ChatGPT.

Right now, the HCAI API is down, which means BobAI can’t speak at the moment. But he worked yesterday without any problems! As soon as HCAI is back up and running, BobAI will be working seamlessly again.

Open comments for this post

@Oleksandr on BobAI · 26 days ago

1h 10m 20s logged

New project architecture

I thought I would host the backend for my model on Hugging Face, but they don’t offer free Docker containers anymore. So, I tried Oracle Cloud. Unfortunately, they’ve been having registration issues for a year now, and I couldn’t even create an account (I tried 3 times :0).

Because of that, I decided to go with Hack Club Nest. I already applied, and now I’m just waiting for the admins to approve my request. They usually offer 2 GB of RAM, but my AI needs a minimum of 5 GB to run. I asked for 6 GB in my application, hope they accept it!

Honestly, I’m at absolute zero when it comes to web development, but… I already wrote an index.html for the frontend and an app.py for the backend. I used AI (Claude) to help me out, of course, since my main skill is ML, but this project is a great way for me to actually learn frontend and backend development too.

I’ll be taking a break for the next 15 days because tomorrow I fly to another country and won’t have my PC with me. It’s a bit sad because I’ll lose a lot of time I could have spent on the project. But to be honest, there isn’t much left to do. Mostly just hosting and debugging if I run into any issues. Anyway, I’ll finish it up as soon as I get back home

Open comments for this post

@Oleksandr on BobAI · 28 days ago

4h 0m 7s logged

Finished the first part of the project

I fine-tuned the model with DPO, then tested it and it went pretty well. He answers questions with no problem.

Then I pushed the model to Hugging Face Hub, to host it as the backend for my upcoming website.

I wrote the README file, and at the end of today’s work I extended the gitignore and fixed one small thing in the model test.

Open comments for this post

@Oleksandr on BobAI · 30 days ago

33m 19s logged

Project architecture

At first, I thought I could train a big LLM on my PC with a lot of parameters and create something similar to ChatGPT, but I was wrong. My RTX 4070 Super is too weak to train such a large model.

Because of that, I will use an open-source model and basically fine-tune it using DPO.

I was thinking about how I should build this AI Chatbot. I thought I could use CulturaX as a dataset for my LLM, but that was a horrible idea. For now, I’ll choose Argilla’s UltraFeedback dataset, basically to give my model an understanding of good and bad responses.

Today, I was thinking about the architecture of the future Chatbot. I actually used AI quite a lot to give me some advice.