Skip to content
Quantoros

Why is HashingVectorizer attractive when new vocabulary arrives continuously and a full vocabulary pass is impossible?

Classic MLLevel: IntermediateStateless streaming vocabulary

Why is HashingVectorizer attractive when new vocabulary arrives continuously and a full vocabulary pass is impossible?

Hints
  • Feature indices need not be learned from a complete corpus.
  • Fixed width does not mean zero collisions.

Answer

It maps tokens into a fixed-dimensional space without fitting a growing vocabulary. New terms can be encoded immediately, with collisions traded against memory and model width.

Source and licence: scikit-learn 1.7.2 — feature extraction · BSD-3-Clause · answer: Quantoros editorial

Related questions