Devlog 2

Added Scrolling Functionality

Today I mapped the scroll wheel with pyautogui to scroll as I move my hand up or down while keeping my finger and thumb in contact.
To do so I calculated the mid point between my thumb and finger. This point acts as the main point whose position is considered for scrolling. You can see it in the video as well, it turns red when my fingers come in contact
Next to actually scroll I use the following code ->

diff = ti_mid[1]-prev_ti_midpt_y                    
if abs(diff) > small_movement:                        
    pyautogui.scroll(int(diff*sensitivity))
prev_ti_midpt_y = ti_mid[1]

Here I Compare the position of the mid pt in each iteration, calculate the difference and multiply with a constant which acts a sensitivity. Finally we use pyautogui to scroll it by that amount. Also if the magnitude of diff is too small (ie. less than small_movement variable) then the movement is not considered.

I also defined these variables at the top for easier configurations

sensitivity = 4
small_movement = 5
tap_threshold = 0.3
hand_detection_confidence=0.2
tracking_confidence=0.1
scroll_mode = True

Tap threshhold here is how close the fingers have to be to consider them in contact

Further Plans

Next I will be adding the mouse movement. I tried to implement that today but its gonna take more time so I’ll continue it in the next session. For that reason only I also added the variable, scroll_mode

Devlog 1

Intro

I have some plans for this project. I want to build a software which will let me control my pc from my hands via gestures.

Functionality

• Left click - Tapping Thumb and Index finger
• Right click - Holding thumb and index finger still while in contact
• Moving cursor - Moving the hand with the thumb and finger close by
• Scrolling - Holding the thumb & middle finger and moving it up and down
• Middle click - Tapping thumb and middle finger

Stack

• Python
• Mediapipe

Today’s work

Today I had setup a git repo and started writing the code. Till now I have made a basic structure which captures input via opencv, makes a dictionary with coordinates of all points in a hand and finally calculates the distance between thumb and index finger. Then it also detects if the thumb and middle finger are in contact.

However I faced one issue when calculating the distance. when I move forward and backward, the distance between 2 fingers is constant in the real world but it keeps changing for the camera as I move hand closer and further from the camera.

To fix this we understand that the ratio between the distance of my wrist and the point right where the pinky finger starts and the distance between fingers is always constant (because the palm never bends so it gives us a reference point to calculate distance), even when I move hand closer or far away. so that’s what I did, I calculated the ratio and used it as the distance which you can also see in the video below. There is also score which is the confidence of hand tracking and contact which tells if the index and thumb are in contact.

Devlog 4

Finally done
I worked upon adding a shortcuts tab for 3 websites as you can see in the picture. I also added the title. Overall the project was fun and I enjoyed making this. I had almost lost my HTML CSS JS skills since It’s been a while I last made a webpage. This let me brush up my skills!!

Devlog 3

Made the Search bar
I have made a seach bar that allows you to seamlessly switch between search engines and AI’s just with a simple dropdown. I implemented this by placing the user’s query directly in the url itself while redirecting to the website. You can try it out right now!!

Devlog 2

I added a simple search bar and improved the background placement today. I am a bit occupied today so I couldn’t do much.

Devlog 1

Starting off
I am working on slick tab. It is will be a simple new tab page with shortcuts, multi search engine functionality and the most important… “APOD”!
Right now I made a basic page that extracts the APOD title and a description

Devlog 7

Completed the Firmware

The firmware is also ready!!! I made the firmware through which you could control the 4 relays mapped to 3 lights and a socket with either the webserver or the physical buttons. The led lights would respond accordingly. The rotary will be used to control my smart fan via an IR blaster API.
Right now it might not be he best but I will improve and fix bugs once I get the hardware.

Devlog 6

Made a simple webpage

Today I made a simple html and css page which will be used to control the devices. It just has a title and a few buttons in a grid. Later I will map javascript functions to actually control them.

Devlog 5

README!

Made a Readme specifying the entire project and also a Bill Of Materials (BOM). The firmware part is still left though.

Devlog 4

Finished the 3D model

I just finished with the 3D model. I have made the complete base with borders. I measured the position of screw holes in my current switchboard and placed the screw holes in this accordingly. However I don’t know how do I hide these screw holes. I just added these circular caps. It is not the best solution so if you have any ideas on how can I cover the screw holes then pls tell me in comments.
I also made and initialised the github repo.

Devlog 3

I made a simple 3D for the switchboard plate.

It has holes to place the LEDs, push buttons and rotary. The buttons and rotary will also be 3D printed. The plug socket will be stuck behind it.
Next I need to add screw holes to mount it and a border.

Devlog 2

I reworked the schematic.

My light bulb has some kind proprietary way of controlling and doesn’t have any simple API. So I just added another relay. I also added LED’s which will act as indicators around my switches and rotary. I already have socket so I will reuse it. I did the wiring and since everything is not on a single pcb and things have to be placed far away I will just keep this as a schematic and not make a pcb. -_-

I have made a basic firmware that connects to the wifi and fetches current time and displays it. This is no where close to our finished firmware but I will be able to continue development once I actually get the hardware. AI helped me quite a lot with this one

Made Readme and Fixed Case

I just made a nice Readme on github, you can check it out. I clicked some pictures for readme formatted it and also made some small improvements to the case. The blocks/pillars in the corners for heatsets were given a bit more support and made longer. The display’s pins were also intersecting with the case so I made a cut-out for them.

Made the case

Today I worked on the CAD model/case design. I preferred using Autodesk Fusion for making this since I had prior experience with it.
Lets begin with the case design, It has small hexagonal designs on the front and a text “4D!” which will be printed in a different colour and stuck in. It looks cool to me but if you have any suggestions how I can make it cooler, drop em down. on the sides it has triangular holes for sound to come out and a hole for usb in the back.
on the top there are 6 switches, Five normal switches and one big switch which will also be 3D printed. It is slightly smaller than the spacebar on our keyboard. The case has a lid and a bottom part, they will be screwed and the screw holes align with PCB’s screw holes so all 3 can be put together with just 4 screws. I have added blocks on the corners inside the case where I will be putting the heatset inserts.
PS - It kinda looks like microwave?

Devlog 1

Smart Switchboard
I am working on this smart switch board. It will have 4 buttons and rotary encoder switch controlling non smart devices with relays and a smartlight and smartfan with their API. 2 buttons will be controlling 2 lights via a relay and 1 button will be controlling a socket with relay. 1 button will control a smartbulb via esp 32. Then a smartfan will be controlled by a rotary encoder switch with smart fan api. I started working on the schematic and wasted so much time looking for the right symbol and footprints of a relay module. Right now its pretty messy I will fix this and also add LED indicators.

Just added cutie pigeon images API

Began working on “4th Dimension Tracker” (Cool, isnt it?). First i just sketched out a design. My clock will have Alarm, Timer and Stopwatch functionality. When alarm rings, there would be minigames which you would have to play and only after winning the clock would stop beeping. The games will be played by 6 switches, one of the switch will be very big, like spacebar, it will be 3d printed apart from the case. These 6 switches will also be used to navigate around the menu and set alarms, timers and so on. The homescreen will be customizable with gifs, motivational quotes and clock styles configured through a simple webserver that I could probably run on the seeed microcontroller(would have to figure that out). After that I proceeded by making the PCB which is also finished. It has layout for 6 switches, buzzer, seeduino and the display. It is less than 100mm x 100mm so it should be approved. Next I will be working on the CAD case and the big keycap.