Implemented Model Evaluation with Perplexity

Perplexity (a metric borrowed from information theory) acts a summarized evaluation metric for your model, that intuitively evaluates how much your model branches off in generating texts.

Mathematically, it’s a geometric mean of the inverse probabilities as calculated by the model, normalized by sequence length.

For seen texts, my model achieved a perplexity around 20-30, and for unseen texts it varied alot due to random picking of tokens in that case.