First Embedding Model

Using the data obtained previously I could get my first embeddings 🎉

The model (image) has 15.000.000 parameters and took roughly 1/2 hour to train. Unfortunately, the performance is below my expectations: It’s not yet good enough so that “uncle - aunt + mum” would output “dad”.

However, the embeddings definitely captured some meaning: For example, the vector of “android” is close to “ios” and to “smartphones”. The vector of “cat” is close to the words “goat”, “monkeys” and “snakes”.

I want to improve this model significantly until it’s good enough to meaningfully calculate with the embedding vectors.