Implemented Laplace Smoothing in the model
Smoothing refers to generalization of n gram model to texts they haven’t seen before. Laplace smoothing (also known as add-one smoothing) is the simplest type of smoothing that avoids the probability estimates to reach zero when handling unseen token or sequences.
It is quite easy to implement this in code, just add one in the numerator of probability formula and add vocabulary size in the denominator.
Comments 0
No comments yet. Be the first!
Sign in to join the conversation.