Hand tracker - Stardance

Ship #2

I finished the wiring up the servos to the pca9685 board and connected the servos to my 3d printed hand model and its so much fun to use and mess around with, this project is by far my favourite.

Its officially done and there are two ways to use it:

you can use the website hand tracker just to mess around with it, see the values of the fingers and its reaction speed. Googles media pipe model has issues as you move your hand closer and further from the webcam but there’s nothing I can do to fix as it’s an issue with the model, not my code.
grab a pico, pca9685 board, breadboard and some wires, 3d printer/ access to a 3d printer and build the whole project, I will attach the 3d model and wiring diagram once I finish them so anyone can build it from scratch and have their own robotic hand at home!

2 devlogs
20h
8.87x multiplier
118 Stardust

Try project → See source code →

Open comments for this post

@Selim on Hand tracker · 14 days ago

3h 16m 41s logged

I finally finished all of the functional design for my code and refreshed my html! the big new stuff I changed was the switch over to tiny web and it felt soooo much cleaner and easier to use and just look at. Since the I changed the main structure it took me a while to re organize everything to work with tiny web. I added logging everywhere because I kept running into issues where my could would just stop working, and I refreshed the html because the old colours annoyed me. I think i’m almost done now and i’m so excited to put everything together and test it! (P.S look how much cleaner the terminal output looks im in awe)

Open comments for this post

@Selim on Hand tracker · 2 months ago

17h 11m 56s logged

This hand code has been traumatizing me for too long, ive restructured it and began adding PID tuning for the motors to turn more smooth and human-like, ive also been trying to bug fix packet loss and random noise being sent back and forth, im going to begin working on the pcb and maybe a third version of the projects CAD! im really excited to finally be finished with the code and take a step back and look at what ive made, hopefully you can all take this step back with me

Ship #1

@Selim on Hand tracker · 3 months ago

I’ve finally reached a point with this code as to where I feel comfortable moving onto reprinting the hand and cleaning it up, for a potential final product!
So far the code can:

✅ Host a website
✅ connect with the browser and host port
✅ allow for webcam access and capture
✅ track finger movements and turn them into percentages (with an overlay)
✅ turns the percentages into numbers that it can decipher
✅ turns the numbers into servo values
✅ move the servo’s within a 5ms time frame (requires external hardware)

Next steps:
📥 Motor smoothing to allow for more fluid movements

Excluding the code im going to start working on the pcb and adjusting the CAD for the actual hand model to allow for a better looking build.
Im really excited to see this project finally come to an end!

2 devlogs
3h
9.65x multiplier
17 Stardust

Try project → See source code →

Open comments for this post

@Selim on Hand tracker · 3 months ago

1h 20m 47s logged

Webpage refresh:
I added Css to make the website look a lot better and aesthetically pleasing, I also added a slider to allow multiple hands to be tracked.
For my next steps I want to add different texts boxes for each hand as currently it gets the percent openness from the least number, not the average.

Open comments for this post

@Selim on Hand tracker · 3 months ago

1h 38m 9s logged

This was my first ‘real’ project where I work on the design and printing, electrical and programming all of the same system, and its taken me all year but it has been one of the funnest things I’ve done in forever. Ive been working on this project on and off starting last summer, and it has been such an amazing learning experience, I’ve been working on the code recently but with the star dance challenge opening, I thought it would be good to post this here as I continue work on it. What I have here is going to be mainly focused on the programming until I feel good with it, then I’ll move on to and quickly finish the pcb. Please feel free to try out the hand tracking software and provide feedback!