Model Library is fully implemented now!

Now model choosing is available

Multi-model downloading now enabled!

Did some algorithm efficienticising for a more balanced algorithm! More details to come…

Model suggestions are ready!

Tried to use WebLLM; reverted. Coding binge

Error handling and basic chat finalised

Major updates to Astilbe! More info to come when core functionality is implemented and I have time to start recordign detailed devlogs.

Updates to come when my devlogs are formatted properly in ~3 days

More updates to come soon

(I definitely did not rewrite the entire architecture)

Devlog 4

Devlog 3: Settings and Customisation

Hello everybody! @yf here with the third devlog. Thankfully, there was not much change to the project purpose (phew) so I can jump straight into what I did. A quick snapshot: I implemented a working settings panel that allows users to customise their experience and saves it to a temp file in the app via tauri-plugin-store.

What did I do?

I:

1. Created a settings panel that allows customisation of the following values:
   (For Dev mode:)

• temperature (float, 0.0-1.0)
• top_p (float, 0.0-1.0)
• top_k (int, 1-100)
  (For the everyday user, dev mode values are hidden, and the following values are the changeable ones:)
• streaming (bool, true/false)
• thinking (bool, true/false)
• theme (string, not implemented yet)

1. I then stored the values as a single json structure options structured in the following way:

{
  "temperature": 0.7,
  "top_p": 0.9,
  "top_k": 40,
  "streaming": true,
  "thinking": true,
  "theme": "default",
  "model": "Qwen3.5:0.8/2/4B"
}

(i.e.)

1. Then, I ditched yet again the rust backend message structure to make way for this new chat structure. I basically added a new key-value pair to the ChatMessage struct, which is options: ChatOptions, and fed it into ollama by flattening it using serde’s flatten attribute, structured the same way as the json above. Models cannot be changed as of the moment.

2. Made the settings panel collapsible via the simple classList toggling.

3. Implemented a simple tauri-plugin-store to save the settings to a temp file in the app, and then read it back on app start. This allows users to have their settings persist across sessions.

What’s next?

Of course, to truly streamline the user experience, I will have to install Ollama with the application. My next priorities are:

1. To bundle Ollama bin with the application and install it on the user’s machine WITH the model. (This will use quite a lot of space, but I guess the installer file will be small enough to accomodate this.)
2. To spawn the ollama process automatically rather than user required, and to kill it when the app is closed. I will do this all in the backend for security purposes.
3. I could realistically just use ollama.cpp, but this means users have to download models themselves, so I think, at least for now, I will stick with the ollama binary. I will consider this in the future.

Devlog 2: Streaming Responses

Hello everybody! I’m back very quickly with the second devlog. As promised, Oh! Chat is:

IN THE MODERN DIGITAL AGE, it has become increasingly difficult to control one’s exposure to the internet, regardless of how reserved one may be or what precautions one may take. With the advent of the large language model era, where people could discuss confidential information with chatbots, this problem - our inability to control the flow of our own data, relying on vague promises of ‘privacy’ that cloud-based services claim but never prove - has become more relevant than ever. Oh! Chat aims to combat this by giving users the premium UX that one experiences on existing AI chatbot applications and interfaces, retaining as much AI functionality as possible, while facilitating user operation, installation, and guaranteeing data privacy. (from a readme draft)

Anyhow, let’s get to the point. Since we want everyday users to feel comfortable, we want to replicate the experience of existing AI chatbots. An important part of this is the text streaming feature, which allows users to see the response as it is being generated: a typing-like effect, and it is also a good method to verify that the model is working. This is what I did in this devlog.

What did I do?

I:

1. Implemented a streaming response feature in the backend.
   • Firstly, we ditch the original old logic with match request.await. This is because the await means we have to WAIT for the entire response to output before continued function execution (which is unideal).
   • Instead, we use reqwest::Client to send a POST request to the Ollama instance, and then we use response.bytes_stream() to get a stream of bytes from the response. We then iterate over the stream using while let Some(chunk) = stream.next().await, and each chunk we convert it to a string and send it back to the frontend using emit_all("chunk", chunk_str). And the client gets it in real time via the listen("chunk", (event) => { ... }) function as an initialised await in main.js. Finally, at the end of the response, a rust event response_end is emitted, and the frontend can accordingly adjust. A very simple implementation, but it works effectively.
2. Drafted a basic readme.
3. … That’s it. (It was quite hard to implement, but it is necessary for user experience.)

What’s next?

I plan on:

1. Allowing chat customisation (to some degree) via settings. More details to come later.
2. UI overhaul no. 1 and replacing the boilerplate html structure with a more well-defined one.

Anyhow, thanks for reading! Watch for the next devlog.

Devlog 1: The Start

Hello everybody! This is the very first devlog for Oh! Chat.

Well, we have to begin somewhere, right? And for us, we begin in this devlog. For the first entry, I can’t really expect not to do all the dirty work of setting up, dependency checking, and all the other mess that comes with starting. A quick overview: I built the initial bridge between front/backend, and established communication with the local Ollama instance.

Also, a personal goal for me: maintain a level of professionalism when logging devlogs (so no more heavy slang, etc.).

What is this?

(Due to word limits please refer to the next devlog)

What did I do?

Let’s first talk about what we needed to finish before we could even start.

1. Ollama & a local LLM
   • As the top-tier consumer-grade inference engine for LLMs, Ollama is a good choice for the project. I also used the Qwen3.5:0.8/2/4B models for testing, as excellent edge models for testing.
2. Tauri v2
   • Tauri is a great framework for building cross-platform desktop applications with html, css and js which I am familiar with. It also uses rust for backend, and though I am not proficient in rust (I have written some basic things in rust before, but not a full backend yet), I aim to use this project to further develop my skills.

We now pivot to what I did in the first devlog:

1. Environment & Project Initialization

   • Created the core project structure utilizing the Tauri v2 CLI framework. (Following this tutorial)

2. Frontend Layout & Boilerplate Architecture

   • Initialised a boilerplate frontend chat template with minimal css and html, and I wired main.js to the backend via invoke and listen commands.

3. Local IPC & The Ollama Localhost Bridge

   • Edited the Rust backend inside src-tauri/src/lib.rs.
   • Creating async commands (#[tauri::command]) to handle inter-process communication (IPC):
     • A hello world tester command;
     • Mapping out the communication layer targeting localhost:11434 (the Ollama api server address for the local instance) to talk directly to the local Ollama instance running on the host machine with reqwest and tokio async libraries.:
       • check_status: checks status (obviously)
       • send_msg(message: ChatMessage): sends a message to the Ollama instance and sends the response back directly using match request.await to handle the response as a single coherent string. (This is temporary for the moment as I will be implementing streaming soon).
       • The message struct is also super simple: a simple json-like key-value pair: role: String, message: String. We use serde to serialize and deserialize the JSON data sent to and from the Ollama instance.

What’s next?

Well, there’s a lot to do. I have not yet done a lot of stuff, and I have many priorities with respect to this application, so I won’t be able to describe to you what I will do next; I guess you can just wait for the next devlog. But I can say that the following will definitely be likely to be done in the next update:

1. Allowing slight chat customisation
2. (Building on 1) giving users a settings panel to allow them to customise their experience.
3. Drafting a basic readme.
4. (non priority) Improve UI?

Anyhow, thanks for reading and I hope you are eager for the next devlog (Cause I am!)