I have added gesture identification. I used the cheap solution of calculating the angles between vectors formed by the existing landmarks. Given that each finger had four points(mcp, pip, dip, and tip), I measured the angle at the PIP joint between the mcp-pip vector and the pip-tip vector. Then I used a hodgepodge of if-statements to determine the gesture based on which fingers were curled.
Howdy! This is an ambitious and stubborn attempt to make a game that navigates based on tracked movements of the hand. I heavily referenced content from this paper:https://arxiv.org/abs/2006.10214. To start, I downloaded a pre-trained neural network (which was 2 convolutional neural networks in a trenchcoat) to track the joints of my hand. One of them is a palm detector that runs on my GPU (fast, but sometimes inaccurate… downsides of a single shot detector model). The other one is a landmark model that takes the palm to predict where the 21 joints could be using an array of coordinates that I yoinked from the MediaPipe Docs. The camera used is a Brio 100 and the model is ran on my RX6600. ( ps. this also works on feet :O )
