Gesture Link is a wearable assistive technology system that helps visually impaired, hearing impaired, and speech impaired individuals communicate more effectively. It combines AI-powered smart glasses, a gesture-to-speech glove, and a speech-to-Braille glove to create a real-time, offline communication platform using computer vision, speech processing, and haptic feedback.
After roughly 40 Hours spread across the last 7 days, Visora is finally working.
I genuinely thought this would be one of the easier parts of Gesture Link.
I was very very wrong.
Visora is the vision module of the project. The goal is simple: use an ESP32-CAM, run object detection using YOLOv8, figure out what’s around the user, and then provide audio feedback.
Sounds straightforward until you actually try building it.
The biggest problem was that nothing wanted to work consistently.
One minute OpenCV would connect to the camera stream perfectly:
cap = cv2.VideoCapture(CAM_URL)
The next minute it would just sit there refusing to open the stream.
while not cap.isOpened():
print("Cannot open stream...")
I spent hours thinking my code was broken when it turned out to be network issues, camera settings, and random ESP32-CAM problems fighting each other.
Then came YOLO.
Getting the model itself running wasn’t too bad.
Getting it to run smoothly with a live camera stream was another story.
results = model(frame, device=device)[0]
Sometimes detections worked.
Sometimes they didn’t.
Sometimes the frame processing would slow down enough to make the whole thing feel unusable. 
My favorite bug was when everything looked correct. lol
The camera worked.
The AI worked.
The ESP32 worked.
The code ran without errors.
And yet nothing happened.
After staring at the screen for far longer than I’d like to admit, the issue ended up being something completely unrelated to where I was looking.
What makes this extra painful is that the final code looks so simple.
You look at it and think: That’s it? 
But behind every line is probably an hour of me wondering why it wasn’t working.
With Glove A converting sign language into speech, Glove B converting speech into Braille through vibration motors, and Visora finally working, Gesture Link is now fully complete.
7 days
40 hours
Way too much debugging
** Gesture Link is Officially Done !!**
#Devlog: Glove A Done!
Finally wrapped up Glove A!
I don’t think I’ve ever spent this much time continuously debugging a single part of a project before. Since yesterday, I’ve probably put in around 20 hours trying to get everything working properly.
Every time I fixed one issue, another one appeared.
It started with the flex sensors. Two of them suddenly became unresponsive and started giving completely unreliable readings. Since the whole glove depends on accurately detecting finger movements, that basically meant the gesture recognition stopped working. After a lot of testing and checking connections, I ended up replacing both sensors.
Then the DFPlayer decided to stop working.
Naturally, I assumed it was a code issue and spent way too long looking through my program trying to figure out what I had broken. Turns out the code wasn’t the problem at all. The SD card had somehow become corrupted, which meant none of the audio files could be read.
So after replacing and setting up the SD card again, the audio finally came back.
At that point I thought I was done.
I wasn’t.
The OLED display suddenly stopped working.
After even more troubleshooting, I found a wiring issue along with an address conflict that was preventing the display from communicating properly. A few rewires, some code changes, and a lot of patience later, the screen finally came back to life.
After all of that debugging, soldering, rewiring, replacing components, and questioning my life choices, I’m finally here.
Glove A is designed to help people with speech impairments communicate more easily. The glove recognizes hand gestures and converts them into pre-compiled speech.
For example, bending a specific finger can trigger a phrase like “Hello”, and the glove will immediately speak it through the onboard speaker. Different gesture combinations can be mapped to different words or sentences, allowing the user to communicate quickly without needing to type or press buttons.
I’ve attached a video showing the glove working in real time.
In the demonstration, the glove detects the gesture, processes the finger positions from the flex sensors, matches it with the corresponding phrase, and instantly plays the audio output.
This project might look simple from the outside, but getting all the sensors, audio modules, display, and processing logic to work together reliably was a much bigger challenge than I expected.
Seeing the glove finally recognize gestures and speak them out loud made all those hours of debugging worth it.
With Glove A now complete and Glove B already working, Now it’s time to focus on Visora and bring the final part of the system together.
After weeks of development, testing, and debugging, I have finally completed Glove B of Gesture Link.
In the video, Glove B is working successfully and responding exactly as intended. You may notice a buzzing sound in the recording. That sound comes from the vibration motors activating to transmit Braille characters.
For this demonstration, the speech input “Hello, how are you?” was converted into Braille and sent directly to the glove. Each finger contains three vibration motors. Two fingers work together to form one Braille cell, while the other two fingers form a second Braille cell. By activating specific vibration patterns, the glove communicates Braille characters through touch, allowing information to be read without visual or audio output.
This marks a major milestone for the project, as the complete speech-to-Braille pipeline is now functioning on Glove B. The system can capture speech, process the text, convert it into Braille patterns, and deliver the output through haptic feedback in real time.
pulled an all nighter :D
worked on the VISORA code, it had some bugs took me a while to fix them. Also added a really nice UI to it.
almost done with the repo!
I am currently developing a colour recognition system using an ESP32-S3 Sense camera and Python. At this stage, the ESP32-S3 Sense is being used primarily to capture and stream camera data, while all image processing and colour recognition tasks are performed locally on my computer using Python. The system is designed to detect and classify colours in real time.
