Desa will be a desktop assistant robot powered by custom Python software. It uses computer vision, voice commands, and a robotic arm to interact with objects on a desk. My dream is to have my own personal Jarvis :)
GUYS I’M SO EXITED to share this with you!!!!! I finally managed to execute both programs (camera and jarvis) at the same time. I had to but the codes in different files and then create a third one and use subprocess and it finallyyy works. I’m so happy. And I’m sorry that my devlog is for 5 hours but I really only worked on Jarvis, adjusted it to my wishes ( listens all the time but only reacts when i say his name) and tried to let the two processes go at the same time.
I’m still trying to connect yolov8n with gemma3, so gemma knows whats on the camera.
.png)
I am still working on object detection and I’m planning on doing object tracking next and linking the recorded video to Gemma 3, so that it can talk about the objects. It may be that I end up doing it without the Ai (gemma) if i know how to do it without and if i have the time for it. I’m also going to build the robot with cardboard first so that i have a prototype to see how it looks.
I am currently working on the camera vision with OpenCV– I’m using YOlOv8 Nano for Object Detection. Side Note: My total coding time is actually now almost 13 h, idk why it is less here :(
I haven’t updated much about my progress, only because I got stuck at one particular thing. Here’s what i have done: I downloaded Ollama for Gamma3 and also installed pyttsx3, pyaudio and speech recognition because I want Desa (still trying to find another name) to recognize my voice when i say its name. So far I have achieved that when I ask, for example: “Hey Desa, where is the biggest mountain in Europe?” that Desa answers me but I still have a far way to go. I still need that it recognizes Objects and knows attributes so he can detect items easier later on. So in order to do so I have to install OpenCV and many other things for Vision and stuff, and because I’m still working on the software, I haven’t yet bought the hardware, so I can’t really upload anything, to show you:( But I plan to begin with the hardware-part in August!! I added a picture of the biggest part of my code. It still needs much improvement and so far I’m proud of it. The code only contains calling Desa and answers from it
I finished the command lines, and now I can ask DESA for anything and it will respond with what it’s giving me. Also I added a tts module (pyttsx3) and now I’m about to work with that, so that Desa can speak and react to my question and commands. (If you have any name ideas, please text me, because I’m not entirely happy with the name Desa) I’m adding a screenshot of my code, let me know, what you think (it’s not much so far)
Desa will be my personal desk assistant, in form of mechanical ROBOT ARM
