Found out yesterday that Ollama is just the whole UI and simplificator of use for the C library llama.cpp
And turns out there’s a python wrapper for that library, llama-cpp-python, which has two advantages over Ollama:
Faster
Way more flexible, we can get every last bit of performance from the rpi5
So I updated ai.py with this new feature, there are more settings than Ollama for optimizing performance for CPU-only (4 cores) which is great, however I had to download a .gguf from Hugging Face and accidentally ended up being given access to a repository for the model?! so i can now propose changes to that repo, for some reason…
Fixed a typo which would’ve raised a TypeError in main.py
Made it so the second button shuts down the Pi safely
Wrote another entry in JOURNAL.md, where I soldered and tested the prototype PCBs I ordered from JLCPCB
Comments 0
No comments yet. Be the first!
Sign in to join the conversation.