New (worse) model

In an effort to improve on the previous model, I created a new one. Now it uses separate weights for the context instead of re-using the embedding weights in the context. The new model has 30.000.000 parameters which is double the previous one.

Yet unfortunately, the results I got were disappointingly miserable: In contrast to the previous model, it doesn’t even group related words. According to the embeddings, the word “cat” is very similar in meaning to “motor” or “caution” 😂😭

I don’t know why it’s performing so poorly, but I’m curious to find out.