Skip to content
Quantoros

What is word embedding, and how does Word2Vec learn embeddings?

NLPLevel: FoundationNatural language processing

What is word embedding, and how does Word2Vec learn embeddings?

Hints
  • What does closeness mean in this representation and metric?
  • Check normalization, exact versus approximate search and alignment with the training objective.

Answer

Word embeddings represent words as dense, low-dimensional vectors that capture semantic relationships (words with similar meaning/usage end up close together in vector space), unlike sparse one-hot encodings that carry no notion of similarity. Word2Vec learns these embeddings via a shallow neural network trained on one of two tasks: CBOW (predict a target word from its surrounding context words) or Skip-gram (predict surrounding context words from a target word). The learned weight matrix (mapping words to hidden-layer activations) becomes the embedding — words used in similar contexts end up with similar vectors, famously enabling analogies like king - man + woman ≈ queen.

Source and licence: Ishan Dutta — Awesome Data Science Interview Q&A · MIT

Related questions