HUGE UPDATE.

I think last work session we left off when I had teleoperation working with the hand. Aka I wore some flex resistors, and the robot mimicked my finger movements. The next task, I mentioned, would be to add buttons to allow for recording and replaying symbols. This would allow me to build an alphabet library for the hand.

Today, I did that. I added the button to record and replay movements, and stored the values of the motor positions in a google sheet. And honestly, this actually worked pretty well.

The hand’s method of communication relies on spelling out words, rather than actually signing the words, mostly because there are more words, than there are letters. Also most letters are easy to sign, because they are static. The issue is that when spellling out with letters, spaces needed their own symbol to indicate breaks in the sequence; so I came up with a wave pattern that is unlike any letter, thus can be interpreted as a break.

Also throughout this time, I rewired my servo setup to rely on a servo driver breakout board(PCA9685), rather than manually connecting the PWM lines, and in the process I must have switched a couple channels, and my sign language library was dysfunctional. So, I rerecorded the letters, and finally, I could comfortably type phrases and receive outputs from the hand that were somewhat discernable, with the slight difference that my hand is missing a thumb of course.

Anyway, after this I set up an ollama agent and wrote a script to have the ollama agent output serial messages, which the hand received and interpreted. This was crazy to see in person! I would be talking to ollama, and it would respond through sign language. This project is moving forward significantly faster than I’d expected, so I think I’ll make it better. I still have the sign language to text part to handle, but afterward, I’m thinking of adding a wrist and a thumb to the hand, to improve the clarity of signed letters.

That’s all for now, folks, see you in a bit…