If you have a sentence with multiple words, you may need to combine multiple word embeddings into one. How would you…
NLPLevel: IntermediateText classification
If you have a sentence with multiple words, you may need to combine multiple word embeddings into one. How would you do it?
Hints
- What does closeness mean in this representation and metric?
- Check normalization, exact versus approximate search and alignment with the training objective.
Answer
Approaches ranked from simple to more complex:
- Take an average over all words
- Take a weighted average over all words. Weighting can be done by inverse document frequency (idf part of tf-idf).
- Use ML model like LSTM or Transformer.
Source and licence: Alexey Grigorev & contributors — Data Science Interviews · CC-BY-4.0