What is the curse of dimensionality? Why do we care about it?
Classic MLLevel: IntermediateDimensionality reduction
What is the curse of dimensionality? Why do we care about it?
Hints
- Which information should a lower-dimensional representation preserve?
- Check centering, scaling and the difference between projection and feature selection.
Answer
Data in only one dimension is relatively tightly packed. Adding a dimension stretches the points across that dimension, pushing them further apart. Additional dimensions spread the data even further making high dimensional data extremely sparse. We care about it, because it is difficult to use machine learning in sparse spaces.
Source and licence: Alexey Grigorev & contributors — Data Science Interviews · CC-BY-4.0