Updates on JADE!

So! A lot of stuff has happened since the last devlog. I went to Boston last week which is why I wasn’t able to do too much, but I’ve done a lot with the actual hardware component the last couple days…

I finished the first iteration of my tools. I have five mini-agents that control tools on the weather, web, calendar, writing (journals, notes, todos), and slack. These are controlled by a kind of global agent. Basically, it takes in the query from the user and selects one of the mini-agents to use, which will then select the tool to use. This was a kind of experiment to see how well agents would interface and control other agents, but it worked well from my testing!

I was able to solder pins onto a mini-amplifier, connect a speaker to it, and connect that to my raspberry pi 4 using jumper cables. I wrote a test of the microphone in python and it makes noise! However, I think I must’ve messed up the soldering or something because the audio quality is pretty bad…

I started on the Alexa/Siri-like component of the project by training an openwakeword wake-word model on the word “Jade” and am experimenting with having it work better in noisier environments. It does detect when I say “Jade”, which is awesome! Once I solder my mini-microphone and connect it to the pi, I should be able to run the full pipeline. Here are my next steps:

1. Finish “Jade” Wake Word Model + test
2. Train or implement speech to text model, test connection with global agent
3. Connect + test microphone to see if speech to text and wake word run on rpi from mic
4. Fix speaker or solder new speaker and test audio
5. Connect agent response to speaker and test full pipeline