DEVLOG #12
What I did
I know this is a lot of time for a single devlog, so I will try to post them more often moving forward. First, I worked on allowing the AI to control the movement of the treads. My first Idea was to create a JSON schema with all 4 directions that the AI could enter values into. The values would dictate the time (in milliseconds) for which the action should be executed. Initially, I decided the 4 directions should be forward, backward, left, and right. But I knew that this would not work, since the tank treads can really only move forward and backward, as well as turn. So I then decided to replace the directions with “Right tread forward”, “Left tread forward”, “Right tread backward”, and “Left tread backward”. I decided on this because I felt that it would allow the AI to reason through my commands, which would allow for much more abstract movement. But the model was not smart enough to map out its movements, and I didn’t really have any other ideas for movement except for forward, backward, and spinning. But that gave me the idea to just use those elements in the JSON scheme. So I set the directions to “forward”, “backward”, “spin left”, and “spin right”. This way, the AI would be able to move exactly how I wanted it to move, without overcomplicating the whole thing, giving me more accurate results. After this, I began working on the APP for OTIS’s controller. I have not decided on a layout, but I do know I want buttons for the 4 directions, a chat box, and a button to send voice commands and photos.
Challenges
422 Unprocessable Content
So while trying to find a way to get the AI to respond as JSON, I remembered Ollama allows you to send format requests as part of the API POST request to the AI. So after reading the docs, I was able to whip up a simple JSON format using Pydantic and BaseModel with the 4 directions. But when I tried to send the request, I kept getting the 422 error response code. So then I tried a bunch of different stuff, like removing the requirements and removing the nested JSON with the format data. But none of that worked. That’s when I did some research on the docs, and I found that it was the Pydantic JSON class that was causing all the problems. You see, Pydantic is used to check whether incoming data is formatted properly. So when I used it to create a JSON schema, it would constantly look for that data from the Arduino, which it wouldn’t find. This caused the 422 error. So I removed the class and created a JSON schema as a regular variable, and this solved the issue
What is next
The next thing on my agenda is going to be to create the OTIS app that will allow me to manually control OTIS like a mini rover. I am currently using Tkinter to create the GUI, and will likely use an ESP web server to allow the GUI to interact with the Arduino. Thank you.