What is the difference between generative and discriminative models?
Classic MLLevel: FoundationClassical machine learning
What is the difference between generative and discriminative models?
Hints
- State the task, inputs and prediction target.
- Compare with a simple baseline: assumptions, objective and independent validation.
Answer
A discriminative classifier learns a prediction rule directly from inputs to labels. It may estimate P(Y|X), as logistic regression does, or learn a decision score or boundary, as a standard SVM does. A discriminative model need not produce calibrated probabilities.
A generative classifier models P(X|Y) and P(Y), or equivalently P(X,Y), then applies Bayes' rule; naive Bayes is an example. More broadly, an unconditional generative model learns or represents a distribution over X without labels. A GAN therefore need not model P(X,Y), and may only provide an implicit sampler rather than a tractable density.
The useful distinction is what distribution or decision rule is learned, not a blanket claim that one family always classifies better.
Source and licence: Ishan Dutta — Awesome Data Science Interview Q&A · MIT · answer: Quantoros editorial