Devlog #3: BIG CHANGES HERE!
I was able to install LiDAR sensors to detect pallets moving under the ESP32 and quickly take a picture that gets sent to the HUB. The main problem I had was managing the communication between the ESP32 and the Python HUB, and the solution I came up with was ensuring only 1 request can be made at a time, so for example if the ESP32 is sending a frame to the HUB, I block all commands from the HUB. This made the overall experience much better since I didn’t have to reset the ESP because it was freezed on a non-existent request! Besides general improvements to the UI, the main change was the addition of another ESP32 with a ML model flashed on it, which does object detection to understand what is traveling on the pallet. I learned that finding a model that can fit on an ESP32-S3 is quite difficult, so the path I chose was Knowledge Distillation, where the ESP32 was the student. To experiment, I used a linear one layer model, which does recognize objects but has some problems, so I will probably try to redo the process with a more complex model.