Furthermore, Non-occurring n-grams produce a sparsity problem — the granularity with the probability distribution could be pretty minimal. Word probabilities have couple of unique values, so almost all of the words have the similar probability. With a little retraining, BERT can be a POS-tagger because of its summary ability https://financefeeds.com/best-copyright-presales-early-bird-opportunities-you-cant-miss/