fixed the keys . again
fixed
fixing
I completed the vision foundation for 0Keys. I added real time two hand tracking with MediaPipe, tracked all 21 landmarks on each hand, highlighted the five fingertips, kept hand IDs stable, estimated fingertip motion, and stored camera frames with timestamps for later audio matching
Ship
Made a RAG chatbot that mimics my friends personality using Llama and Hugging Face.
NOTE: I’m using zero GPU on HF so after 100 msgs it goes to sleeping mood , in that case the chatbot would reply a random thing like “system off” , “internet is down” or something like that , so don’t worry about that, come after 24Hr and she’ll talk normally
FACT: Senpai used to say a lot “Haii haii” which is a japanese phrase for “yes” so don’t worry about that either, she’s just being friendly with ya
Hope ya’ll love this version of chatbot
32m 47s logged
Guyss!! This is gonna be the last change I’ve made today, Senpai chatbot is live now and she’s talking like my friend (really couldn’t be more happy to see her talk to me again 🥹 )
25m 47s logged
Made some Minor UI changes on frontend nextjs
Built the audio foundation for our invisible keyboard. I replaced the ultrasonic approach with a real time table tap detector that uses causal filtering, adaptive noise calibration, transient features, precise timestamps, and debouncing. I also tested it against synthetic taps and background noise, then validated it with the laptop microphone and yayy it’s working,
Built spectrogram today and did some tests on it and this image im attaching is the spectrogram of an actual diagonal streak, the same shape as the theoretical chirp diagram from Day 1. Also learned spectrograms are plotted in decibels (log scale) specifically because raw sound energy varies by orders of magnitude as a linear scale would wash out anything quieter
Got recording working today and ran my first FFT on live mic input — watching a 1-second recording, made the graph using matplotlib of time domain vs frequency domain.
Just for some nerds: The FFT plot couldn’t tell when a sound happened during that second, only that it happened somewhere in it means a whistle at 0.1s and 0.9s would look identical. It will tell you what how much each frequency showed up in total time
Spent most of the day learning the Nyquist theorem ,the wild part is that sampling too slow doesn’t just lose data, it actively lies to u and reports a completely different, wrong frequency (called aliasing). Built a chirp generator that sweeps 18-20kHz, just under the inaudible to adults line, and burned an embarrassing amount of time debugging why I heard nothing before realizing it was a audio routing issue, not the DSP. Speakers confirmed working with a loud 440Hz test tone. On to recording and the FFT next.
9h 36m 12s logged
After testing the chatbot with around 50 beta users, I found several issues that weren’t obvious during development. Responses were too long, the personality became too generic when using RAG, Urdu and English were sometimes mixed incorrectly, and the mobile layout needed work. I shortened the responses, restricted web search to things like news and sports, improved the system prompt and input handling, redesigned the mobile UI, and added a message explaining the cold start delay
8h 28m 36s logged
I added logging, I started finding problems that weren’t obvious from the frontend. Web searches were sometimes failing silently, too many messages were causing token overflows, and using both max_length and max_new_tokens was causing unexpected behavior. I fixed these issues by adding proper error handling, limiting the conversation context, and simplifying the token settings. I also built a small debug dashboard to track searches, response times, token usage, and errors (Ughh I’m really exhausted now)
8h 47m 50s logged
I initially moved the deployment to Replicate to improve response speed. It worked and responses dropped from around 15–30 seconds to about 5–10 seconds but then it also introduced extra cost and complexity. In the end, I decided to stay with Hugging Face Spaces because the free setup is more suitable for this side project. Also improved the HF app.py file
8h 34m 24s logged
I noticed that the model was giving very generic and robotic responses even though it worked much better during training. After debugging, I realized I was using a different system prompt during inference than the one I used while fine tuning. Once I switched back to the exact same prompt, the responses became much more consistent. Also fixed Hackclub key errors , now it’s working fine
10h 9m 17s logged
wanted senpai to be more. Previous It was only talking with using
fine tuned Llama model but if i’ll ask it something like what’s
happening in the world, it won’t talk back like my friend cause it
didn’t have that latest information of the world so i added RAG. used
the hackclub api as a middleman llm that cleans up messy input and
decides if a web search is needed ( like “what’s the fifa score rn” this
will be flagged as a web search needed query)
two bugs along the way:
responses were so slow at first , turns out just zeroGPU cold starts, chills out after msg 1
model started sounding fake n robotic bc i overwrote it with a huge system prompt. fixed this by making the heavy prompt short
so now gng it’s live, plz talk to her
Ship
I built a AI RAG chatbot that mimics my friends personality , trained on whatsapp chats so it actually talks like my friend instead of generic robot ai. fine tuned llama 3b with LoRA on kaggle’s free gpu, deployed it on hf spaces zeroGPU gradio for backend and next.js on vercel for frontend, plus added RAG so it can clean up messy texts and pull live info from the web when needed. it genuinely feels like talking to my friend, all built for free. to test it. just start chatting, first msg might take a sec to load (model’s waking up)
10h 9m 17s logged
wanted senpai to be more. Previous It was only talking with using
fine tuned Llama model but if i’ll ask it something like what’s
happening in the world, it won’t talk back like my friend cause it
didn’t have that latest information of the world so i added RAG. used
the hackclub api as a middleman llm that cleans up messy input and
decides if a web search is needed ( like “what’s the fifa score rn” this
will be flagged as a web search needed query)
two bugs along the way:
responses were so slow at first , turns out just zeroGPU cold starts, chills out after msg 1
model started sounding fake n robotic bc i overwrote it with a huge system prompt. fixed this by making the heavy prompt short
so now gng it’s live, plz talk to her
7h 47m 18s logged
I uploaded my model on Hugging Face with zero GPU
(made a new hf space called amadeus, picked gradio as the sdk, zeroGPU as hardware)
If someone else is thinking about starting in HF, just wanna tell the annoying part: cold starts. first message takes 20-40s while the model loads into memory. after that it’s quick like 3-8s
then built an actual next.js frontend since raw gradio ui felt kinda mid, hooked it up through an api route, deployed on vercel with my env keys
And it’s live
11h 17m 11s logged
Trained the data ( This took soo much time fr)
I used kaggle’s free gpu (t4, sometimes p100 if i got lucky) to fine tune llama 3b with LoRA through PEFT.
quick explainer since people asked LoRA freezes the whole base model (32m params untouched) and only trains a tiny set of adapter params (420k). instead of retraining the entire model I put a extra layer of changed weights on top of the model which is way cheaper and faster than the traditional training
tested it after training and it WORKED. The model is giving the same energy, same little phrases, actual personality of my friend which is kinda scary ngl
notebook’s public if you wanna nerd out: https://www.kaggle.com/code/hamnamubarak/amadeus/edit
Loading more…