@miry

Working on choosing characters. The app’s side of the code is almost done (without testing…) and I have barely touched the ESP32 side. The wooden plates I need are not at my house yet. Nothing is assembled. One of the puppets doesn’t exist yet. I didn’t test the speaker yet. The competition is Sunday starting at 8am. The national OLYMPIAD. Yeah so I do put myself in these situations, wish me luck…

Adding the Eleven Labs API to my app. I won’t even lie, this was kinda hard and I used Gemini to help me with some parts. It took me a while to figure out how I wanted the code sequence to work, but eventually after a lot of back and forth I finally made my own sequence while getting some help from Gemini. Stick to your critical thinking, even if it takes more time to complete the code, you gotta understand what’s going on! Anyway, this is still untested code and I still have the esp32 side of things. I need to figure out a way to use the audio’s alignement meta data to move the servos accordingly.

Code for the app! So here I spent SO much time debugging with Gemini, as I am using the Firebase Ai Logic to connect my API to my web app for dialogue generation. I also fixed up the Ui at first because it was really ugly. Anyway, now I gotta set up the Eleven Labs Functionality (which will be completely new) and then send all of that over to the ESP32. ALSOOO I changed an important part of my system architecture: the puppet servos will no longer be connected to the arduino, and I will instead be using the ESP32 for that. Arduino will be in charge of ultrasonic sensors and stepper motors for the slider crank mechanism ONLY. i’m wondering what else I could do with it. One last thing: i love firebase documentation. This still took longer than expected, but I enjoyed it!

puppet body and face complete! I attached the servos and even came up with how I want to style it. I’m leaving the entire styling situation for the end bc I legit don’t even have the code yet and competition is in 4 days. I’m loving how everything is coming out so far. I sent my wooden box design to a wood shop and still waiting for it to get done so I can securely connect the puppet to the slider crank mechanism.

Figured out the puppet mouth mechanism!! it works pretty well, it’s straightforward, and when I make the puppet head I can easily hide it! Now gotta do the rest of the body and figure out how it will react to esp32 dialogue data. 5 days to go for the national robotics olympiad, wish me luckk

I connected my esp32 to my firebase app!! Now I can send over info from my app to the microcontroller, which will decode the string and send over information to arduino!!

creating a comprehensive pdf for the wood shop to understand what I want! Fingers crossed my design is good enough!! I included the first page for some insights. This will speed up my process a lot, because im ngl i wasted sooo much time trying to build the thing with FOAMBOARD (I thought it was sturdy enough), and it will make designing the puppet control mechanisms much easier

3D Model of the show box. I’m getting it made out of wood by a local shop! Now I need to break down my model into its pieces with dimensions, stay tuned! It’s a simple design that does the job, hides components well, and I can customize it and give it color later on.

Connecting my app and ESP32 to Firebase. Basically, users will be sending over their story to the app through text or audio. Next, Gemini 1.5 Flash will convert the input into a puppet show dialogue for two characters (not more than 3 minutes). It should send this output to the ESP32 through realtime database and save it to cloud firestore at the same time. My laptop is at 8% so I had to stop. I connected the app to firebase but I’m still having trouble with esp32 connections.

quickly reminding myself of the c++ syntax (i’ve been using dart for the past 3 years and theyre extremely similar) for writing code on the arduino ide because it’s been a while. All good so far though! Used a simple one hour video by programming with mosh. Just messed around a little honestly

Just prepared some basic UI elements for the app. Basically I want users to upload their stories to the app, and then the gemini flash 1.5 api will turn it into a two-character dialogue in a specific format and hand it over to the ESP32 via Firebase. It’s really ugly but for now I’m just focusing on getting things to WORK because I have less than a week left and still no build or software yet, but it’s fine we got this

Just imported the components into kicad to get started on the wiring diagram (there is no pcb for this). It’s almost 2am now so I’ll continue this later

So the National Robotics Olympiad in my country is next week, and I’m building my robot… now? In my defense, I only found out about it a while ago, somehow found a teammate (I don’t know many people personally who do robotics) and registered for the future innovators category despite all odds. I’m building an autonomous glove puppet show that uses Ai to turn someone’s orally transmitted story into a dialogue between two puppets through an app. Hopefully we can podium if we execute this properly and get funding from stardance because this went a lot over my budget lol