“Hey Tiny Jarvis. where’d I put my keys again?”

• Tiny_Jarvis now has memory!
• I’ve implemented a simple sqlite local database in Python to store every question the user asks. (first time doing this ! very satisifying)
• This means it can help you remember timings, dates, locations… anything!
• However, this also means that all these cumulatively added memories make the prompt for the AI longer and longer
• Which is why for the moment it only remembers the last 5 prompts. This may seem like a small amount, but for the most it is sufficient for remembering recent prompts.
• Should I make it remember more stuff? tell me

“Hey Tiny Jarvis. where’d I put my keys again?”

• Tiny_Jarvis now has memory!
• I’ve implemented a simple sqlite local database in Python to store every question the user asks. (first time doing this ! very satisifying)
• This means it can help you remember timings, dates, locations… anything!
• However, this also means that all these cumulatively added memories make the prompt for the AI longer and longer
• Which is why for the moment it only remembers the last 5 prompts. This may seem like a small amount, but for the most it is sufficient for remembering recent prompts.
• Should I make it remember more stuff? tell me

Devlog 3: Buddy Pre-release is still a pre-release.

I am so close to getting this available for all hack clubbers !?!?!

But the problem is it isnt polished yet…

😢

Recap for New comers - Skip this if you read the last 2 devlogs
Buddy is a Desktop AI assistant/agent that fetches info and runs commands on your computer (Mac specifically, others will come soon). But at first it was just a terminal assistant but one of you guys in slack said i should make a good gui for it like a nice cool user interface for it. and i did and then i go the idea of making this like something all hack clubbers can use and enjoy so now im stuck making a polished app for all hack clubbers

and so…

Warning Extremly long and descriptive devlog ahead

⚠️⚠️⚠️

What i did so far…

Basically I added..

• Tuff bear logo for the menubar icon
• Settings page - manage instructions, your info, links, api’s etc (not finished yet )
• Actually working chat history - You can access previous chats youve had with it
• Library Page - Lists every single chat youve ever had with it chronologically
• Attach files - you can add any files you would like to it (almost done)
• A base version of a long term memory - This is what bridges it from being a mindless ai robot to someone who can work alongside you and actually remember projects you have collaborated on with it. (very small thing using an offbrand RAG system)

In short i realized im gonna have to do a lot more to make this in good shape maybe at least 2 or 3 devlogs before i can send out a Buddy Beta for Mac and then windows and linux

Not much has changed in UI. just the different pages and soon i will add a way to change theme color

Next?

this is gonna be a long sequence but here we go

1. finish attach files update
2. Extensions page - allow user to add different extensions such as access to github or access to messages etc
3. finish settings
4. add different thinking modes
5. Long-term memory v2
6. polish UI further
7. Find UI bugs - debugging
8. make into an app and test on my mac
9. start making windows and linux versions

• this shouldnt be that hard because there are 3 parts of my code base,

The UI, The Model, and the backend.

The model is universal and my UI is too.

for the backend python is also mostly universal, what isnt universal is tool calls rn it uses applescript and other mac only commands. im gonna have to recreate that for the other systems.

I dont know why it says 8 hours i swear it should be a bit more and i dont know why i took so long to post.

Stay tuned in Devlog 5 or 6 i should release a beta for mac and next devlog should be linux and windows.

P.S Comment a cool thing i should add (I honestly have no ideas)
P.P.S Repost If You this is cool and you want more people to see this and have access to this tool

“Hey Tiny Jarvis, what’s going on in the world?”

• New feature: Tiny Jarvis automatically fetches the news from BBC World, chooses the most impactful headline, reads the articles and generates a summary!
• This happens in the background, every hour, to ensure that the news is recent.
• I discovered smth called RSS feeds, which is basically a tool that feeds website updates back to you in the form of XML files.
• Then I learnt about the feedparser python library, as well as trafilatura, for extracting text from HTML (for getting the text from the articles), which made the job ten times easier.
• However, this process does take some time:
  • Step 1: script parses the BBC World feed
  • Step 2: local AI chooses between the thirteen daily headlines
  • Step 3: script fetches the associated articles text
  • Step 4: local AI summarizes ALL THREE articles.
• On a Raspberry Pi 5, obviously, this can take at least 5 minutes.
• Now for the moment this is a bit biased, because I chose the “Technology” category for the RSS feed 😅 but i’ll add the option to change it later
• Is this a good feature? Please tell me

“Hey Tiny Jarvis, what’s going on in the world?”

• New feature: Tiny Jarvis automatically fetches the news from BBC World, chooses the most impactful headline, reads the articles and generates a summary!
• This happens in the background, every hour, to ensure that the news is recent.
• I discovered smth called RSS feeds, which is basically a tool that feeds website updates back to you in the form of XML files.
• Then I learnt about the feedparser python library, as well as trafilatura, for extracting text from HTML (for getting the text from the articles), which made the job ten times easier.
• However, this process does take some time:
  • Step 1: script parses the BBC World feed
  • Step 2: local AI chooses between the thirteen daily headlines
  • Step 3: script fetches the associated articles text
  • Step 4: local AI summarizes ALL THREE articles.
• On a Raspberry Pi 5, obviously, this can take at least 5 minutes.
• Now for the moment this is a bit biased, because I chose the “Technology” category for the RSS feed 😅 but i’ll add the option to change it later
• Is this a good feature? Please tell me

“Hey Tiny Jarvis, play Smooth Criminal”

• mopidy-youtube integrated! and it works! All you have to do is press the button, say out loud “Play (song) …” and it will search for it in Youtube music (FREE MUSIC!!!) and play it.
• You can then adjust the volume with the rotary encoder, which is satisfying asf.
• This was one hell of a debugging session, holy
• I had to search for forked repos on Github that fixed a obscure yt-dlp issue, and somehow it works.
• Anyways apart from that, i restructured the project code repo entirely, everything is now separated into different sections like firmware and pcb to distinguish hardware from software.
• New: the install.sh was finally tested and it works perfectly after a few tweaks! woohoo!
• Oh and finally: rewrote my README.md as recommended by the Stardance guides, and yh I agree it looks better
• Tell me if you have any other ideas of features to implement

“Hey Tiny Jarvis, play Smooth Criminal”

• mopidy-youtube integrated! and it works! All you have to do is press the button, say out loud “Play (song) …” and it will search for it in Youtube music (FREE MUSIC!!!) and play it.
• You can then adjust the volume with the rotary encoder, which is satisfying asf.
• This was one hell of a debugging session, holy
• I had to search for forked repos on Github that fixed a obscure yt-dlp issue, and somehow it works.
• Anyways apart from that, i restructured the project code repo entirely, everything is now separated into different sections like firmware and pcb to distinguish hardware from software.
• New: the install.sh was finally tested and it works perfectly after a few tweaks! woohoo!
• Oh and finally: rewrote my README.md as recommended by the Stardance guides, and yh I agree it looks better
• Tell me if you have any other ideas of features to implement

Music handler + system metrics idle screen

• Time to make Tiny_Jarvis a little more like the real Jarvis….
  SYSTEM METRICS SCREEN!

• Displays in real time: battery life, cpu temperature, current time on the OLED screen.

• Finished up the Mopidy part of the music player, can’t test it yet sadly (i don’t have access to my rpi5 right now) it was way more difficult than I thought

• AIM: “Play …” -> searches through free music online database like Youtube music for query and plays the music if it finds it.

• Added another quirky feature: flip a coin

• Tell me if you have ideas of other new useful features

Music handler + system metrics idle screen

• Time to make Tiny_Jarvis a little more like the real Jarvis….
  SYSTEM METRICS SCREEN!

• Displays in real time: battery life, cpu temperature, current time on the OLED screen.

• Finished up the Mopidy part of the music player, can’t test it yet sadly (i don’t have access to my rpi5 right now) it was way more difficult than I thought

• AIM: “Play …” -> searches through free music online database like Youtube music for query and plays the music if it finds it.

• Added another quirky feature: flip a coin

• Tell me if you have ideas of other new useful features

New timer feature implemented, plus more

• Created a handle_prompt() function in main.py to handle any task-specific prompt given by the user.
• For example: if I say “Play …” or “Set a timer of …” then it will recognize this specific pattern and handle those actions perfectly, and if it’s something else than the AI will just answer it.
• It’s scalable, so I can add more features in the future, such as “Definition of” or idk (if you have any ideas lemme know in the comments)
• Created a timer.py file which handles the timer feature, which is sort of complicated but now works (it displays on-screen in real time, which is pretty cool)
• All in all, main.py is now very clean and readable.

New timer feature implemented, plus more

• Created a handle_prompt() function in main.py to handle any task-specific prompt given by the user.
• For example: if I say “Play …” or “Set a timer of …” then it will recognize this specific pattern and handle those actions perfectly, and if it’s something else than the AI will just answer it.
• It’s scalable, so I can add more features in the future, such as “Definition of” or idk (if you have any ideas lemme know in the comments)
• Created a timer.py file which handles the timer feature, which is sort of complicated but now works (it displays on-screen in real time, which is pretty cool)
• All in all, main.py is now very clean and readable.

Cleaned up main.py and cooking_helper.py

• It’s been a while since I last worked on this project, I’m still waiting for my hardware funding request to be approved, which made me lose motivation.
• But I’m back, so here’s what’s new:
  • I separated the main function in cooking_helper.py Unix-style, so that everything is cleaner and easier to debug or scale (in case the CSV file gets bigger)
  • Now it doesn’t need to open the csv everytime, instead it just opens it once upon startup.
  • In main.py, created an initialize() function which separates the tasks into an easier format. The AssistantDisplay class and the list of recipes is then passed onto the while True loop.
  • The main() function is now much simpler: it calls the initialization function, then in the loop it checks whether there are new recordings thanks to an imported function, if there are not it waits 0.2 seconds before checking again, and if there is then it runs the process_recording() function.
• Very clean!

Cleaned up main.py and cooking_helper.py

• It’s been a while since I last worked on this project, I’m still waiting for my hardware funding request to be approved, which made me lose motivation.
• But I’m back, so here’s what’s new:
  • I separated the main function in cooking_helper.py Unix-style, so that everything is cleaner and easier to debug or scale (in case the CSV file gets bigger)
  • Now it doesn’t need to open the csv everytime, instead it just opens it once upon startup.
  • In main.py, created an initialize() function which separates the tasks into an easier format. The AssistantDisplay class and the list of recipes is then passed onto the while True loop.
  • The main() function is now much simpler: it calls the initialization function, then in the loop it checks whether there are new recordings thanks to an imported function, if there are not it waits 0.2 seconds before checking again, and if there is then it runs the process_recording() function.
• Very clean!

Upgraded the cooking helper + critical bug fixes

• Translated the recipes to english - will add more later
• Fixed a critical issue with initializing and importing different python files, now main.pyhas the real control of initialization thanks to a listener.init_listener() function.
• Added a speak(['Ingredients']) in main.py so Tiny_Jarvis will speak the ingredients of the chosen recipe out loud
• Switched from hardcoding random.randint(0,33) to random.choice(list) way simpler and easier to read, and created a format_cooking_sentence(recipe_object) function so that main.py doesn’t have to manage that, way better
• Future changes I might implement: apparently Raspberry Pi OS manages audio through the Advanced Linux Sound Architecture (ALSA) which is responsible for the hardware drivers, the audio routing and control, and the low-level utilities such as the commands I often use, such as alsamixer, aplay and arecord
• So when two python libraries that need to either record audio or output audio call ALSA to do so, it can lead to conflicts.
• In this case, sounddevice and pyaudio which are both wrappers of the C library PortAudio. So a future change would be only using sounddevice for example, or vice versa.
• This has another hidden advantage: removing a possibly finnicky dependency and making the whole process smoother, since compiling C on a Raspberry Pi can be difficult at times.

Upgraded the cooking helper + critical bug fixes

• Translated the recipes to english - will add more later
• Fixed a critical issue with initializing and importing different python files, now main.pyhas the real control of initialization thanks to a listener.init_listener() function.
• Added a speak(['Ingredients']) in main.py so Tiny_Jarvis will speak the ingredients of the chosen recipe out loud
• Switched from hardcoding random.randint(0,33) to random.choice(list) way simpler and easier to read, and created a format_cooking_sentence(recipe_object) function so that main.py doesn’t have to manage that, way better
• Future changes I might implement: apparently Raspberry Pi OS manages audio through the Advanced Linux Sound Architecture (ALSA) which is responsible for the hardware drivers, the audio routing and control, and the low-level utilities such as the commands I often use, such as alsamixer, aplay and arecord
• So when two python libraries that need to either record audio or output audio call ALSA to do so, it can lead to conflicts.
• In this case, sounddevice and pyaudio which are both wrappers of the C library PortAudio. So a future change would be only using sounddevice for example, or vice versa.
• This has another hidden advantage: removing a possibly finnicky dependency and making the whole process smoother, since compiling C on a Raspberry Pi can be difficult at times.

Added a cool silkscreen logo

• Made an entry for the 23rd of July in JOURNAL.md
• Found and added this logo from The Noun Project.
• Really cool website btw. This icon clicked with me bc it really represents what Jarvis (or AI chatbots are in general) is, that being SIMULATED intelligence, and makes us remember that’s its all just complex math equations and not real intellect.
• Fixed some DFM issues, and rerouted a lot of traces, some were going over the mounting holes
• Tried to make more space for the logo but broke the whole thing and had to Ctrl+z my way out of there (help)
• Looks cool no?

Added a cool silkscreen logo

• Made an entry for the 23rd of July in JOURNAL.md
• Found and added this logo from The Noun Project.
• Really cool website btw. This icon clicked with me bc it really represents what Jarvis (or AI chatbots are in general) is, that being SIMULATED intelligence, and makes us remember that’s its all just complex math equations and not real intellect.
• Fixed some DFM issues, and rerouted a lot of traces, some were going over the mounting holes
• Tried to make more space for the logo but broke the whole thing and had to Ctrl+z my way out of there (help)
• Looks cool no?

PCB + updates on the cooking feature

• Wrote an entry in JOURNAL.md for today’s work
• Worked a lot on the PCB in EasyEDA Pro, mostly clearance issues but also a lot of struggling with the GND pour.
• Updates on the cooking feature:
  • Thought it would be cool to be able to show the ingredients on the OLED screen
  • So updated the AssistantDisplay class and created a neat function to show text using the python library luma.oled
  • Problem is it would go off screen, so I implemented the built-intextwrap module and it can now display 5 lines.
  • Will get to translating everything to english later, promised
• Anyways if anyone has any recommendations for the PCB don’t hesitate to leave a comment!

Added a cooking recipes feature for no reason

• Created a cooking_helper.py in a cooking/ folder
• OCR’d half of a cooking book with help from Gemini and put everything in a beautiful recipes.csv
• Used the csv.DictReader object to access the info and organize it into a list, very useful.
• Updated main.py so that when the user’s prompt starts with “Cooking”, it takes a random recipe from the csv file (using the random.randint(a, b) function) and returns a formatted sentence with all the info for cooking that recipe.
• Made this cos I’m really bad at cooking and it could also benefit others, you never know.

PCB + updates on the cooking feature

• Wrote an entry in JOURNAL.md for today’s work
• Worked a lot on the PCB in EasyEDA Pro, mostly clearance issues but also a lot of struggling with the GND pour.
• Updates on the cooking feature:
  • Thought it would be cool to be able to show the ingredients on the OLED screen
  • So updated the AssistantDisplay class and created a neat function to show text using the python library luma.oled
  • Problem is it would go off screen, so I implemented the built-intextwrap module and it can now display 5 lines.
  • Will get to translating everything to english later, promised
• Anyways if anyone has any recommendations for the PCB don’t hesitate to leave a comment!