Maths with English words

With the new dataset, training took a lot longer. My computer worked for around 50 hours to run through 26GB of training data.

The long wait really payed off though, the model is so much better than the previous model and it is finally capable of doing the embedding arithmetic I’ve been chasing the whole time.

Embedding arithmetic is so fun, here are some examples:

• throw - throwing + running -> run (-ing form)
• better - good + friendly -> friendlier (comparasion)
• woman - man + king -> queen (man compared to woman is like king compared to queen)
• germany - berlin + paris (capitals)

The model learned a lot more syntactic and semantic meaning than I had expected.

Next I want to build a web app, so that you can play with the embeddings yourself.