Added a cooking recipes feature for no reason

• Created a cooking_helper.py in a cooking/ folder
• OCR’d half of a cooking book with help from Gemini and put everything in a beautiful recipes.csv
• Used the csv.DictReader object to access the info and organize it into a list, very useful.
• Updated main.py so that when the user’s prompt starts with “Cooking”, it takes a random recipe from the csv file (using the random.randint(a, b) function) and returns a formatted sentence with all the info for cooking that recipe.
• Made this cos I’m really bad at cooking and it could also benefit others, you never know.

Schematic and PCB tweaking

• Made an entry for today in JOURNAL.md
• I’ll keep this short, but basically: huge issues with my schematic, electrically incorrect for the most part, some useless components, which then translated into the actual PCB, having to rewire traces manually, making tons of mistakes along the way and spending waay too much time repairing them.
• Still waiting for the funding request to be approved before I can order these.
• Huge thanks to M-Mrk in the Slack electronics channel for all his help, he saved me

Error handling time! + bug and dependency fixing

• Now that I have a clean slate to test my project on, I ran python main.py and somwhow got three errors in a row.
• Spent an hour adding trys and except Exception as e everywhere to prevent this.
• Now debugging is way easier and more user-friendly.
• Fixed some bugs such as ssd1106 not being a real OLED screen part name
• As well as gpiozero missing the rpi-lgpio library, which needs system-wide packages swig and liblgpio-dev so yh dependency hell.
• Almost everything should be error-proof now. I tested it, and no errors popped up, just friendly reminders that I2C wasn’t enabled, or that my buttons weren’t detected, or no screen.
• Update install.sh with these changes, now works perfectly.

RPI5 OS Switch + Tiny_Jarvis easier to put in place

• Worst day in my life. Tried to flash a Raspberry Pi OS Lite (64-bit) on the raspberry pi 5 (which btw has an NVMe SSD), and it did NOT go well:
  • First flash using the Imager: didn’t work, nothing happened, the Pi started speaking in freaking morse code
  • Spent an hour trying to decode that morse code according to documentation, thought it was an EEPROM bootloader issue, made a diagnostic card, made a factory reset card, nope.
  • Realized I could plug it through HDMI to my projector; did that: it was asking me for a username.
  • Didn’t have a USB keyboard so couldn’t type it in. Almost threw the Pi out of the window cos I configured all that in the fricking Imager.
  • Realized I was using the Imager from Flathub and not rpi-imager
  • Retried the whole thing with that Imager: still didn’t work.
  • Spent two hours doing the same thing over and over: flashing the card with the Imager, going into the files, editing user-data, cloud-init-ouput.log, etc… realized the Imager wasn’t writing the information in those files but others, booting it up into HDMI, checking the logs, managing to bypass username but not keyboard layout, etc… I EVEN TRIED LEGACY VERSIONS OF THE OS AND CREATING A wpa_supplicant.conf.
  • Atp I was going to crash out but then I checked my rpi-imager version (even though I had updated it) and turns out my laptop just couldn’t get the latest version : it was 1.8.0, not 2.1.0 (the latest one).
  • Downloaded the newest version, ran it, and it worked. I have officially lost faith in humanity.
• Anyways I read my README.md and decided that it was getting way too complicated (the installation part) so I pasted that whole part into Kimi K3 and he made me a install.sh which is basically fire since you just run

chmod +x install.sh
./install.sh

and the whole system-wide-packages-and-dependencies are all installed in the right order, easily.

• This is tested and proved (i did it with the newly flashed OS).

Major software update + PCB testing

• Found out yesterday that Ollama is just the whole UI and simplificator of use for the C library llama.cpp
• And turns out there’s a python wrapper for that library, llama-cpp-python, which has two advantages over Ollama:
  • Faster
  • Way more flexible, we can get every last bit of performance from the rpi5
• So I updated ai.py with this new feature, there are more settings than Ollama for optimizing performance for CPU-only (4 cores) which is great, however I had to download a .gguf from Hugging Face and accidentally ended up being given access to a repository for the model?! so i can now propose changes to that repo, for some reason…
• Fixed a typo which would’ve raised a TypeError in main.py
• Made it so the second button shuts down the Pi safely
• Wrote another entry in JOURNAL.md, where I soldered and tested the prototype PCBs I ordered from JLCPCB

JOURNAL.md new entries!

• Added some entries for the 7th and 8th of July, where I delved into PCB making from scratch!
• As well as an entry for the 17th of July, when I received my first PCBs - spoiler alert: they do not work as intended
• Made a lot of mistakes, but managed to make a rather good first try, which will hopefully lead to a better final PCB (i’m working on that)
• This time I’ll check the DFM before ordering
• Hope you appreciate my storytelling!

JOURNAL.md started + Hardware time!

• Created a JOURNALmd file
• Documented everything I did on the 5th and 6th of July, with photos and some explanations as to what I was doing and trying to achieve
• To make it short: prototyping and creating some visuals for the OLED screen/UI, as well as assembling everything together.
• Connected README.md to JOURNAL.md
• You can read it here

Finishing up the software part

• Took on the task of making my code more readable, and as such easier to debug
• Created a AssistantDisplay class to better manage the OLED threads - way cleaner now
• Put sentence_buffer() in ai.py and get_lastest_recordings() in listener.py, now the only function in main.py is main(). Clean!
• Squeezed a bit more performance out of the faster-whisper model by setting cpu_threads to 4 and beam_size to 1, even if we’ll lose a tiny bit in quality. Still a good tradeoff
• Detected an avoided a critical issue: if glob detected the file while it was being written to and passed it on to faster-whisper, then it would be corrupted and be mashed garbage. So I read some documentation about the queue module and used that, it’s way better and easier in the end.

Faster response speed thanks to streaming audio

• Ok so ollama’s ChatResponse class allows, when creating an instance of it, to write stream = True.
• So I could stream the outputted response directly into Piper TTS and into the speakers.
• Except it would sound very weird, very robotic-like since the chatbot outputs tokens, not words. I then created a function called sentence_buffer() that seperates them into a list of words, that are then given to Piper TTS in speaker.py
• Then I used a library called pyaudio to stream the Piper TTS output into the speakers! it’s actually just a Python wrapper for a C library called portaudio, so I ran into a LOT of dependency issues, but that’s a battle for another day.
• Moved a few functions that were in main.py into other files for better management
• Other small modifications

Threading implementation for better speed

• Today I decided to learn about the python module threading, and realized it wasn’t that difficult to implement in my project.
• The issue was: when I ran oled("thinking_animation.gif") it ran sequentially, so the whole script had to wait for the animation to finish, which obviously can’t happen.
• So I created different threads for each animation/png, and made a “switch”, better known as a threading.Event() which enabled me to manage the “workers” way better (sorry, i’m getting used to the vocabulary)
• Also I added comments everywhere in main.py for better explainability.
• Finally I adjusted the code in OLED.py with the switch implemented and added some small user-friendly checks throughout my project.
• Hopefully everything works!

Note: sadly it will be difficult to use threads in the rest of the code, since the process is, at its core, sequential. Faster-whisper needs audio to proceed, the ollama AI chatbot needs a prompt to continue, etc…

README.md first commit

• Spent a LOT of time on the README.md, I want my project to be reusable so made a good How to install section, there are tons of different commands to run to ensure it works first try
• Didn’t write anything in the Hardware section, but will soon. (my first PCBs don’t work yet)
• In OLED.py: it now supports gifs, which is great because of Piskel which can turn simple animations into gifs.
• Made some visuals in Piskel for the screen (thinking_animation.gif and button_pressed.gif)
• Simplified drastically listener.py for future debugging.

One of the biggest issues with this project so far: the difficulty to test it! I don’t have all the required materials yet. I might use the pytest module? we’ll see

Linking everything together

• Updated the project structure with more python scripts so that it now looks like this:

Tiny_Jarvis/
├── .venv/
├── piper_voices/
├── recordings/     # New
├── assets/    # New
├── ai.py
├── listener.py    # New
├── main.py
├── OLED.py
├── speaker.py
├── transcriber.py    # New
├── README.md
└── requirements.txt

• What’s new:
  • Added an assets/ folder for the OLED screen visuals
  • Added a recordings/ folder for the audio captured by the mic when the user is speaking
  • Created listener.py: actively listens for a button press; when pressed, records audio through the usb mic until the button is released, then that audio is sent to transcriber.py
  • Created transcriber.py: takes that audio and passes it as input into faster-whisper to “translate” into text !
• And finally, first time linking everything together in main.py: there is still a lot to be done if we are talking speed/efficiency, but the core pipeline is done.
• The user presses a button to talk, releases it when finished, the audio is captured and translated into text by faster-whisper, then that text is given as prompt to the AI chatbot (like Qwen3-0.6B. The response is then turned into a voice using Piper TTS and outputted through the speaker!

Possible future implementation of the threading module to make it faster, even though I have no idea how it works.

Initializing the project

• Created a virtual environment .venv/ so that all necessary python modules and packages are cleanly installed together.
• Created a requirements.txt, where I put all the python modules I import in the project so that people can reuse it.
• Created a piper_voices/ directory and downloaded a medium voice via python -m piper.download_voices en_US-lessac-medium
• Created 4 different files:

Tiny_Jarvis/
├── ai.py   # the AI chatbot
├── main.py   # the main script
├── OLED.py    # The visuals!
└──  speaker.py    # the audio

• Wrote the basic code for each of those files. For the moment, I concentrate on making importable functions that function indepedently, and will link them together in main.py later on.
• Created a README.md. It’s empty for now, will fill it in later.