Skip to content
Quantoros

What are the advantages and disadvantages of bag of words?

NLPLevel: IntermediateText classification

What are the advantages and disadvantages of bag of words?

Hints
  • Separate text representation, modeling and evaluation.
  • Check ambiguity, rare terms and training versus target text distributions.

Answer

Advantages:

  1. Simple to understand and implement.

Disadvantages:

  1. The vocabulary requires careful design, most specifically in order to manage the size, which impacts the sparsity of the document representations.
  2. Sparse representations are harder to model both for computational reasons (space and time complexity) and also for information reasons
  3. Discarding word order ignores the context, and in turn meaning of words in the document. Context and meaning can offer a lot to the model, that if modeled could tell the difference between the same words differently arranged (“this is interesting” vs “is this interesting”), synonyms (“old bike” vs “used bike”).

Source and licence: Alexey Grigorev & contributors — Data Science Interviews · CC-BY-4.0

Related questions