Proba ML
9. Linear Discriminant Analysis
9.1 Intro

9. Linear Discriminant Analysis

We consider classification models of the form:

p(y=cθ,x)=p(xy=c,θ)p(y=c,θ)cp(xy=c,θ)p(y=c,θ)p(y=c|\theta, x)=\frac{p(x|y=c,\theta)p(y=c,\theta)}{\sum_c' p(x|y=c',\theta)p(y=c',\theta)}

where p(y=c,x)p(y=c,x) is the prior over the class labels, and p(xy=c,θ)p(x|y=c,\theta) is the class conditional density for cc.

  • The overall model is a generative model since it specifies the distribution over the feature xx, p(xy=c,θ)p(x|y=c,\theta)
  • By contrast, a discriminative model directly estimates the class posterior p(y=cθ,x)p(y=c|\theta,x)