Ship
Changes requested
.jpeg)
Devlog #4: I finished the project!
I was able to complete the object detection part, along with creating the model to flash onto the ESP32-S3. I also added another ESP32-S3 with the camera module to capture the side of the pallets so I can combine the pallet’s ID with the object on it. The OCR model was also made for the ESP32-S3 using the MNIST dataset + some images that I took.
.jpeg)
Devlog #3: BIG CHANGES HERE!
I was able to install LiDAR sensors to detect pallets moving under the ESP32 and quickly take a picture that gets sent to the HUB. The main problem I had was managing the communication between the ESP32 and the Python HUB, and the solution I came up with was ensuring only 1 request can be made at a time, so for example if the ESP32 is sending a frame to the HUB, I block all commands from the HUB. This made the overall experience much better since I didn’t have to reset the ESP because it was freezed on a non-existent request! Besides general improvements to the UI, the main change was the addition of another ESP32 with a ML model flashed on it, which does object detection to understand what is traveling on the pallet. I learned that finding a model that can fit on an ESP32-S3 is quite difficult, so the path I chose was Knowledge Distillation, where the ESP32 was the student. To experiment, I used a linear one layer model, which does recognize objects but has some problems, so I will probably try to redo the process with a more complex model.
Devlog #2: I completed the camera post-processing part, added the option to change the ESP32 IP directly from the web interface, and improved the data transfer between the ESP32 and the backend to make image capture faster and more reliable.
The next step is to add IR sensors to detect when pallets pass under the camera, so the system can trigger captures automatically.
Devlog #1: After rewriting the entire project base and switching the communication protocol from HTTP to TCP, I’m now working on camera exposure control to make object detection more reliable, even in poor lighting conditions.
