Skip to content
Quantoros

What are N-grams? How can we use them?

NLPLevel: IntermediateText classification

What are N-grams? How can we use them?

Hints
  • Separate text representation, modeling and evaluation.
  • Check ambiguity, rare terms and training versus target text distributions.

Answer

The function to tokenize into consecutive sequences of words is called n-grams. It can be used to find out N most co-occurring words (how often word X is followed by word Y) in a given sentence.

Source and licence: Alexey Grigorev & contributors — Data Science Interviews · CC-BY-4.0

Related questions