By
@Lawrence
Gesture Sandbox is a 3D environment where you can use hand gestures to spawn objects and manipulate it. It has the ability to simulate physics and 3D models.
39m 33s logged
What is Gesture Sandbox?
Gesture sandbox is a 3d modelling environment controlled entirely by your hand gestures (lowkey stole this from the Iron Man movie). You can use different hand gestures and motions to manipulate 3d objects on the screen.
Curious to learn more? Click Here
In this devlog, I mainly made optimizations to the site to boost fps (my laptop is lowkey buns please send me a framework hackclub 🙏)
38m 32s logged
What is Gesture Sandbox?
Gesture sandbox is a 3d modelling environment controlled entirely by your hand gestures (lowkey stole this from the Iron Man movie). You can use different hand gestures and motions to manipulate 3d objects on the screen.
Curious to learn more? Click Here
In this devlog, I made 2 main changes.
I noticed that Gesture Sandbox had become very laggy recently, and the average fps dropped from 40 to 20. I asked Claude to try to figure out why. The lag was caused by the glow affect when a user selected an object. To render this glow, the site would draw the scene with all of the objects, then it would redraw the whole scene again to add the glow effect. This was changed, and basically how it works now is the site draws the main scene, then feeds that scene into the bloomComposer() function that draws the glow effect where needed.
im too lazy to explain this one ive done enough typing today but basically the site wasnt recognizing pinch gestures and so i fixed this with math that i will explain sometime later
33m 45s logged
I have added a ton of new features to gesture sandbox recently. More importantly, I have found many critical bugs. Here’s what I’ve been working on:
For some gestures such as the delete gesture, the user is required to hold the gesture for multiple seconds. This used to require the user to keep the exact same position of their hand for every frame of those seconds. This would often get interrupted by slight hand movements or camera shake. I fixed this by having the AI ignore inconsistencies that only lasted a few frames.
When the site first opens, camera access is requested. The site used to completely crash if camera access was not granted for some reason. I changed the site so that it displays an error message and reattempts to get camera access instead of crashing.
Since gesture sandbox relies on MediaPipe for the gesture recognition AI, it would sometimes crash from a corrupted download (dont ask how i know this i definitely didnt try to upload my own model to replace mediapipe). I fixed this by doing a checksum verification before each download
