0Keys

fixed the keys . again

fixed

fixing

I completed the vision foundation for 0Keys. I added real time two hand tracking with MediaPipe, tracked all 21 landmarks on each hand, highlighted the five fingertips, kept hand IDs stable, estimated fingertip motion, and stored camera frames with timestamps for later audio matching

Built the audio foundation for our invisible keyboard. I replaced the ultrasonic approach with a real time table tap detector that uses causal filtering, adaptive noise calibration, transient features, precise timestamps, and debouncing. I also tested it against synthetic taps and background noise, then validated it with the laptop microphone and yayy it’s working,

Built spectrogram today and did some tests on it and this image im attaching is the spectrogram of an actual diagonal streak, the same shape as the theoretical chirp diagram from Day 1. Also learned spectrograms are plotted in decibels (log scale) specifically because raw sound energy varies by orders of magnitude as a linear scale would wash out anything quieter

Got recording working today and ran my first FFT on live mic input — watching a 1-second recording, made the graph using matplotlib of time domain vs frequency domain.
Just for some nerds: The FFT plot couldn’t tell when a sound happened during that second, only that it happened somewhere in it means a whistle at 0.1s and 0.9s would look identical. It will tell you what how much each frequency showed up in total time

Spent most of the day learning the Nyquist theorem ,the wild part is that sampling too slow doesn’t just lose data, it actively lies to u and reports a completely different, wrong frequency (called aliasing). Built a chirp generator that sweeps 18-20kHz, just under the inaudible to adults line, and burned an embarrassing amount of time debugging why I heard nothing before realizing it was a audio routing issue, not the DSP. Speakers confirmed working with a loud 440Hz test tone. On to recording and the FFT next.