Project Demo, small Fixes & Deployment

In order to be ready to ship, I created a demo video using OpenScreen showing how to use the app, I also did some fixes here and there and finally deployed the app to GitHub Pages.

Tuanble Strings & Expandable Fretboard with Dynamic Staff Sizing

Users can now set how individual strings are tuned on the fretboard, as changing the tuning is required for some pieces on the guitar.

Additionally, players can now choose how many frets they are comfortable using - the staff resizes (more/less staff lines) so that the entire fretboard can be utilised without cluttering the screen with notes you cannot play at the selected fret count.

More Notes and Open Strings

Besides updating the layout I “added new notes”. Of course you can have as many as you want, but I wanted to make sure lines outside of the staff become muted, so everything is easier to see.

More importantly, open strings are now visualized on the fretboard - previously only notes where guitar players had to press down showed up. The architecture I talked about in previous devlogs worked wonderfully :)

Colored Notes and Architecture

I fixed the architectual issues i wrote about in my previous devlog using a dedicated class for visualization. It uses the builder pattern and should be extendable so that new visualizations on the fretboard can easily be added.

Moreover I gave each note its own color, numbered the frets and added little circle markers in the same pattern they appear on a guitar.

Fretboard shows Notes

The notes you pick in the staff show up on a guitar fretboard!

One thing I learned from this project so far, it is to always make dumb React components: Don’t manage the component’s state internally when it can make sense wanting to access it from the parent component in the future. In my case: The fretboard is a “smart” component and when passed a note it knows and shows all the possible locations to play it - this works for now, but it’s hardly extendable: For a feature I’m planning, the parent component needs to control precisely where to show the dots - which as of now is impossible, because the fretboard only works with notes + it will show all possible locations, not precisely the ones I want to show.

To fix this, I will expose a prop that controls not what notes to show, but what frets and strings dots should appear on.

Improved Note Picking and Support for Sharps/Flats

I improved the note picking: To prevent notes from overlapping the note now follows the mouse position, users can place notes freely wherever they want.

Additionally, I added support for sharp and flat notes, which can be toggled on the left side of the note’s name. It’s a bit quirky as e.g. a sharp F becomes “F#” but a sharp H becomes “C”, not “H#” (thanks music theory!)

Note Picker

First version of the note picker, it will need a re-do to prevent overlapping notes and to support sharps and flats.

I also wrote a system that converts into an internal representation from which it’s easy to get the names of the notes and hopefully to locate them on the guitar fretboard.

Finish Demo Website (Pages 4-8)

I completed the last part of the demo website including the most exciting part about the word arithmetic.

Embedding Demo Website (Pages 1-3)

I really like the way the embeddings turned out. That’s why I want to make a website to make word embeddings as accessible and understandable as possible.

I’m assuming no prior knowledge from the visitors, and try to interactively guide them through the world of embeddings, with each slide explaining something about embeddings or being interactive to try it yourself. This time i finished slides 1-3, with the last one being the interactive one.

Since it should feel engaging and alive, I focus a lot on subtle animations and micro-interactions. I’ve never built a website with heavy animations before so this is the perfect opportunity to try GSAP!

Maths with English words

With the new dataset, training took a lot longer. My computer worked for around 50 hours to run through 26GB of training data.

The long wait really payed off though, the model is so much better than the previous model and it is finally capable of doing the embedding arithmetic I’ve been chasing the whole time.

Embedding arithmetic is so fun, here are some examples:

• throw - throwing + running -> run (-ing form)
• better - good + friendly -> friendlier (comparasion)
• woman - man + king -> queen (man compared to woman is like king compared to queen)
• germany - berlin + paris (capitals)

The model learned a lot more syntactic and semantic meaning than I had expected.

Next I want to build a web app, so that you can play with the embeddings yourself.

New Dataset + Improved Pre-Processing

In an effort to increase the model’s performance I switched from the previous model’s Wikipedia Simple English dataset to a subset of the Fine Web Edu dataset. As an additional measure to improve the results I decided to use a larger context of 4 words left and right to the root word.

I also completely re-wrote all the preprocessing steps. Even though the dataset is high-quality, language is very nuanced and needs filtering. But since words next to each other (4 words before and after the root word) get grouped, filtering an entire word means the group mustn’t be formed (as information is now missing), on the other hand removing punctuation from the end of a word should still allow that word to appear in a group.
Overall I filter/handle:

• quotes
• brackets
• non-English Characters
• apostrophes
• other special characters (dashes, underscores, etc.)

Another challenge is ensuring that grouped words actually belong together. In the previous dataset I simply grouped the words next to each other in a giant blob of text. This time I only group words from the same text source (the same article, document, …) and only within the sentences of that source. This ensures groups don’t span multiple sentences.

This taught me a lot about working with larger amounts of data - I couldn’t load everything in memory as it would be too large, but had to read from disk in batches to process the data. After ~50 min of leaving my computer untouched I had the final dataset with ~26GB (~18GB compressed using word IDs) of filtered word groups.

Next, I will have to find a way to determine unrelated words, so that I can start training the third iteration of the model.

New (worse) model

In an effort to improve on the previous model, I created a new one. Now it uses separate weights for the context instead of re-using the embedding weights in the context. The new model has 30.000.000 parameters which is double the previous one.

Yet unfortunately, the results I got were disappointingly miserable: In contrast to the previous model, it doesn’t even group related words. According to the embeddings, the word “cat” is very similar in meaning to “motor” or “caution” 😂😭

I don’t know why it’s performing so poorly, but I’m curious to find out.

First Embedding Model

Using the data obtained previously I could get my first embeddings 🎉

The model (image) has 15.000.000 parameters and took roughly 1/2 hour to train. Unfortunately, the performance is below my expectations: It’s not yet good enough so that “uncle - aunt + mum” would output “dad”.

However, the embeddings definitely captured some meaning: For example, the vector of “android” is close to “ios” and to “smartphones”. The vector of “cat” is close to the words “goat”, “monkeys” and “snakes”.

I want to improve this model significantly until it’s good enough to meaningfully calculate with the embedding vectors.

Data preprocessing

Before the model can train to retrieve the embeddings, it needs data. I found a dataset on Kaggle with the text of 249.396 wikipedia articles.

Today I did some preprocessing, first I removed any non-english words and delete special characters,
then I determined the 50.000 most frequent words, and limited my vocabulary to these words.
To compute the embeddings, I took the left and right words of each word in the data and put them into a pandas data frame along with two other unrelated words: (left word, center word, right word, unrelated word 1, unrelated word 2)

The final dataset has 24.780.670 rows, so I likely cant train on the full dataset on my computer.

The goal is for the model, when presented with the center word to predict which of the other words are similar to it which it can only do by learning the meaning of each word - which is what we want.

GitHub Pages Deployment

I deployed the app to GitHub pages.

I encountered one issue: The images were not loading, because GitHub page URLs end with the name of the repository, so the root of the app is not at / but on /<Repository-Name>. The path of my images were preceded with a / which is incorrect when the root is not at /. Removing the / resolved the issue.

Plant Details & UI Tweaks

Plants can now be clicked on, showing a details menu on the right with details and a tip for growing the plant.

I had troubles with the CSS of the legend at the bottom right corner which is positioned absolutely: The library used to make the panels resizable sets position: relative during dragging, causing the legend to be in an incorrect position only during dragging. I fixed the issue with some clever div wrapping.

Bookmarking + Refactoring

Users can now bookmark the plants they want to see. Bookmarked plants save in localStorage.

Moreover, there is now an empty state when there are no plants to show due to filters.

I also did some refactoring and simplified the code here and there.

For my Angular learning goal: I learnt how to use effect(), which seems to do basically the same as the useEffect() hook in React.