You are browsing as a guest. Sign up (or log in) to start making projects!

10h 14m 8s logged

Sephyr (the mod)

  • A proper lifecycle of the detection system was added. The IPA model is loaded in background during game start, but the audio capture and processing start waits until you join a world, and stops once you leave it.
  • The Rust code now has a logging hook, so that it can do civilized logging using the Java logger, instead of blatantly writing to stdout.

Sepple (the voice casting detection system)

  • I made the pipeline system cancellable, so that the mod can cleanly stop the detection, which was not possible before.
  • Support for running the speech detection inference on GPU was added as a feature flag. For now it is not used by default, as it showed no measurable improvements over running on CPU on my PC, but it might be useful on other, weaker devices.

Collecting data for training

As I said too many times, the speech to IPA model used for magic speech detection is not quite the quality I would like it to be. I’m hoping to improve it by fine-tuning it on its weak areas, but for that I need some data.
My idea is to use wiktionary as a data source. It is a free online dictionary run by the same people who run Wikipedia. Most words on it have the pronunciation attached both as an IPA and an audio file, which makes it ideal for the task.
I downloaded a preprocessed jsonl dump file from kaikki and wrote a script to extract just words and ipa from it. Unfortunetely it was missing urls of the audio files. So I had to download a raw xml dump of the entire english wiktionary, and spend few hours writing a script to extract them. Then I used deepseek a generate a third script, to download all the audio files from a list. There is going to be almost 20k of them, and the script is running right now as I’m writing this, and it is going to take multiple hours as I can only download around 1 file per second to respect rate limits and not overburden wikimedia. And just now while writing I realized I’m an idiot, because kaikki provides a 20GB tar file with all the audio, downloading which would be more efficient.

0
8

Comments 0

No comments yet. Be the first!