Ship
Pending review
Devlog 3
Added Mouse and Zoom Functionality
I made this quite a while ago and then got sick, forgot to log this. I added Mouse scroll and zoom functionality. The zoom works by bringing thumb and index finger close, then using the middle finger to zoom as you can see in the video. You can also use the mouse in a similar way. your mouse moves with the midpoint of your index and thumb. when you tap them its a left click, when you touch your thumb and middle finger, the tracking stops for you to adjust your hand if it goes out of the camera. You can switch between the 3 modes by pressing “P” while keeping the camera window in focus
I decided to build this project which will have a really cool and responsive backlight system. Making it was quick, I just had to downscale the frames of the screen, assign each row/column to a LED in the strip and send the data. I struggled a bit the last step bit with a bit of searching I had 4 arrays with colours of each side which i rendered as 4 windows for demonstration.
Ship
Pending review
Devlog 2
Added Scrolling Functionality
Today I mapped the scroll wheel with pyautogui to scroll as I move my hand up or down while keeping my finger and thumb in contact.
To do so I calculated the mid point between my thumb and finger. This point acts as the main point whose position is considered for scrolling. You can see it in the video as well, it turns red when my fingers come in contact
Next to actually scroll I use the following code ->
diff = ti_mid[1]-prev_ti_midpt_y
if abs(diff) > small_movement:
pyautogui.scroll(int(diff*sensitivity))
prev_ti_midpt_y = ti_mid[1]
Here I Compare the position of the mid pt in each iteration, calculate the difference and multiply with a constant which acts a sensitivity. Finally we use pyautogui to scroll it by that amount. Also if the magnitude of diff is too small (ie. less than small_movement variable) then the movement is not considered.
I also defined these variables at the top for easier configurations
sensitivity = 4
small_movement = 5
tap_threshold = 0.3
hand_detection_confidence=0.2
tracking_confidence=0.1
scroll_mode = True
Tap threshhold here is how close the fingers have to be to consider them in contact
Further Plans
Next I will be adding the mouse movement. I tried to implement that today but its gonna take more time so I’ll continue it in the next session. For that reason only I also added the variable, scroll_mode
Devlog 1
Intro
I have some plans for this project. I want to build a software which will let me control my pc from my hands via gestures.
Functionality
- Left click - Tapping Thumb and Index finger
- Right click - Holding thumb and index finger still while in contact
- Moving cursor - Moving the hand with the thumb and finger close by
- Scrolling - Holding the thumb & middle finger and moving it up and down
- Middle click - Tapping thumb and middle finger
Stack
- Python
- Mediapipe
Today’s work
Today I had setup a git repo and started writing the code. Till now I have made a basic structure which captures input via opencv, makes a dictionary with coordinates of all points in a hand and finally calculates the distance between thumb and index finger. Then it also detects if the thumb and middle finger are in contact.
However I faced one issue when calculating the distance. when I move forward and backward, the distance between 2 fingers is constant in the real world but it keeps changing for the camera as I move hand closer and further from the camera.
To fix this we understand that the ratio between the distance of my wrist and the point right where the pinky finger starts and the distance between fingers is always constant (because the palm never bends so it gives us a reference point to calculate distance), even when I move hand closer or far away. so that’s what I did, I calculated the ratio and used it as the distance which you can also see in the video below. There is also score which is the confidence of hand tracking and contact which tells if the index and thumb are in contact.
Ship
Devlog 4
Finally done
I worked upon adding a shortcuts tab for 3 websites as you can see in the picture. I also added the title. Overall the project was fun and I enjoyed making this. I had almost lost my HTML CSS JS skills since It’s been a while I last made a webpage. This let me brush up my skills!!
Devlog 7
Completed the Firmware
The firmware is also ready!!! I made the firmware through which you could control the 4 relays mapped to 3 lights and a socket with either the webserver or the physical buttons. The led lights would respond accordingly. The rotary will be used to control my smart fan via an IR blaster API.
Right now it might not be he best but I will improve and fix bugs once I get the hardware.
Devlog 4
Finished the 3D model
I just finished with the 3D model. I have made the complete base with borders. I measured the position of screw holes in my current switchboard and placed the screw holes in this accordingly. However I don’t know how do I hide these screw holes. I just added these circular caps. It is not the best solution so if you have any ideas on how can I cover the screw holes then pls tell me in comments.
I also made and initialised the github repo.
Devlog 2
I reworked the schematic.
My light bulb has some kind proprietary way of controlling and doesn’t have any simple API. So I just added another relay. I also added LED’s which will act as indicators around my switches and rotary. I already have socket so I will reuse it. I did the wiring and since everything is not on a single pcb and things have to be placed far away I will just keep this as a schematic and not make a pcb. -_-
I have made a basic firmware that connects to the wifi and fetches current time and displays it. This is no where close to our finished firmware but I will be able to continue development once I actually get the hardware. AI helped me quite a lot with this one
Loading more…