DEVLOG 5

What I did

I began by updating the 3d model by updating the position of the screws. I did not like the visibility of the screws in the earlier design, which is why I decided to put them in from the back. I had placed some spacers between the front of the robot and the screw holes on the LCD, which meant I could screw it in from the back since the spacers created a place for the screws to screw in. This meant two things. The screws were now invisible from the exterior, and I didn’t need to use nuts.  Then I created the assembly. I created my own animation for the assembly by manually transforming each component. I also created a drawing of the exploded view with a parts list. Lastly, I updated the schematic because someone suggested adding net labels to make it look cleaner. I also did quite a bit of research trying to wrap my head around MCPs and other options. But I landed on using Ollama to install a local AI model, Open Web UI to use the model with MCPs, and Python that will handle the MSPs as well as the communication between the AI and the microcontroller.

Challenges

Creating the exploded view.

 Creating the animation was hard enough, especially since the animation system is really annoying to work with. But I decided that an animation of the disassembly guide would not suffice, so I wanted to make a drawing from the animation as an exploded view. But the problem came up when I created the drawing, and not all of the parts were visible. So I solved this by creating a separate animation where all the parts are moved far enough that they are all visible from the same angle, but still in their relative locations.

Figuring out the architecture workflow

This has been the worst part of this project, especially because I have practically no knowledge whatsoever regarding this kind of stuff. The main problems I was having were trying to find a way to allow the AI to interact with the microcontroller. I had no Idea where to start, and was losing faith in this project. I also considered scrapping the whole movement idea because I didn’t know how I would integrate it effectively. But I stumbled on MCPs, which I think will work. They basically work by creating tools that an AI can use to interact with other software, and they are very versatile. I want to make an MCP server (a custom set of tools) that controls the code that the microcontroller runs on. Only problems are that I don’t know how to use Python or MCPs, and therefore have no idea what my options are. After some extensive research, I have a rough idea that I think might work. The prompt and camera feed will be sent to a web server created by the esp32, which a Python script will be able to access via an API. This prompt is then fed into the AI, which will use the data to generate an answer. I will also create tools in the Python script that the AI can use that will turn the servos on and off through an API if it decides it needs to move. This data, along with the response from the AI, will be sent back to the web server, which the microcontroller will display on the LCD. 

What is next

Currently, I am waiting on my microcontroller, which is said to arrive later this week. While I wait, I will probably continue researching ways to get this to work and try to get a better idea of what I expect of OTIS. Any help or guidance with this matter would be greatly appreciated. Thank you.