Sign Language Translator

DEVLOG 3 — Training the First Gestures
Finally got everything working. I set up a data collection script that opens the webcam, lets a gesture label be selected, and then starts recording when space is pressed. From there, every frame grabs 63 normalized MediaPipe landmark values and saves them into a CSV row with the label attached. I collected about 300 samples per gesture, which sounds like a lot, but it’s actually pretty quick roughly 30–40 seconds per gesture since it just keeps recording while the sign is held, with natural movement built in. First quick test was super simple: yes/no, thumbs up vs thumbs down. Then I ran training and somehow hit 100% test accuracy by epoch 3, which honestly caught me off guard. The model itself is pretty small and just four dense layers (128 → 64 → 32 → output). But since the input is already structured landmark data instead of raw images, it picks up the patterns almost immediately.

Back with an update, so this took way longer than it should have. First thing, I run pip install tensorflow and immediately get slapped with ERROR: Could not find a version that satisfies the requirement tensorflow. Turns out TensorFlow just straight up doesn’t support Python 3.14 yet, so I had to go download Python 3.11 separately and run everything with py -3.11 from there on out. Thought I was done but then pandas starts giving ImportError saying Please upgrade numpy to >= 1.26.0, so I upgrade numpy like it asks… and now TensorFlow throws AttributeError: _ARRAY_API not found because apparently TensorFlow 2.13 needs numpy <= 1.24.3. So these two libraries are just directly fighting each other and I’m stuck in the middle. Ended up pinning numpy to 1.26.4 which somehow makes both of them happy, no idea why that exact version is the magic number but it works. And THEN on top of all that, VS Code kept ignoring the Python 3.11 interpreter I picked and running 3.14 anyway, so I gave up on the play button completely and just run everything from terminal now with py -3.11 script.py.

About a week ago I started my sign language translator and spent the first day mapping out the core architecture before writing any code. I planned out the 3 stage pipeline I’m building MediaPipe captures the webcam feed and extracts 21 points per hand, those get normalized relative to the wrist so distance from the camera doesn’t matter, then TensorFlow takes those 63 numbers and classifies them into a word. I also worked out why I’m going with this landmark based approach instead of the typical CNN on images method most tutorials use feeding a model 921,600 raw pixel values means it has to learn what a hand even looks like before it can learn gestures, whereas feeding it 63 clean numbers means it only has to learn the gesture itself. After locking in the architecture I spent the rest of that first day writing the code and split everything into three separate files so each part of the pipeline has its own job.

collect_data.py: opens up the webcam and runs MediaPipe hand detection, I hold a gesture and hit space to start recording samples for it. Every frame it grabs the 21 landmark points, normalizes them based on the wrist position and hand size, and saves those 63 numbers along with the gesture label as a row in a CSV. (have not tested it yet but it should work)

train_model.py: takes that CSV, encodes all the gesture labels, splits everything into training and test sets, and builds a small dense neural network 128 → 64 → 32 → output using TensorFlow. It saves the trained model and label encoder so the detector script can use them later.

run_detector.py: loads that trained model and runs live prediction, pulling the same normalized landmarks from the webcam and showing the predicted gesture in real time.
i didn’t get to test any of it BUT HOPING SOON