Overview: N Gram Model is a fundamental model in Natural Language Processing used to predict next word in a given sequence. It relies on markov’s assumption that the probability estimate of a word relies on it’s immediate preceding words rather than the whole text/sentence.

What I have made uptill now: I have made the first module required for this project which is a Tokenizer, a tokenizer takes a text corpus and separately stores each word or token in a lookup table.

Future Direction: The core mechanism of N gram models, which is probability estimates for each n word sequence using frequency found in the corpus. Further, I would want to add smoothing to these probability estimates and add model evaluating metrics like perplexity.