DEVLOG #10

What I did

So for this devlog, I mainly spent all of my time working on the camera. I looked up an example script from the Seeed Studio docs to learn about the main system. I then spent quite some time working on a way to convert the picture into a format that I could place into the JSON. I also shifted the old code a bit to better fit the way I wanted to use the camera. In the end, I was able to get the camera working and send a photo to the AI to reference in its response.

Challenges

Getting the conversion of the photo

So I went into this knowing literally nothing about how cameras like this one worked. So I started with the docs and was able to write the code that took the photos. Although this was a bit tedious, the main challenge came when I tried to convert the photos to a format that was compatible with the AI. You see, when you take the photo, it is saved as a frame buffer JPEG. But I needed to convert it to Base64, since that is what Ollama uses when integrating images in the POST request. So I did some digging and came across mbedtls_base64_encode() from the mbedtls/base64.h library. This was really annoying to work with because there are a lot of required parameters, and I didn’t know what any of them meant, let alone how to use my code with them. After an hour of searching, I asked Gemini what each of the parameters meant and what to include in them, and I was able to get the code completed. But when I tried to run it, my ESP kept rebooting. After some more research, I learned that the cause was insufficient memory to store the Base64 string. I was able to fix this by adding if statements as well as memset() to manage the memory space better. After this, I added some code so that it would work regardless of whether the camera was connected.

What is next

The thing I am going to work on next is to create a way for the AI to use the servos. I already started working on a tool to handle that, but I have not tested it yet. I am also thinking about remodeling the wheels because I noticed that rubber wrist bands might be a better alternative to the timing belts. Thanks.