Generative Modeling
I want to go through the Wikipedia series on Machine Learning and Data mining. Data mining is the process of extracting and discovering patterns in large data sets involving methods at the intersection of machine learning, statistics, and database systems.
References
Notes
In statistical classification, two main approaches are called the generative approach and the discriminative approach. These compute classifiers by different approaches, differing in the degree of statistical modelling. Terminology is inconsistent, but three major types can be distinguished:
- A generative model is a statistical model of the joint probability distribution on a given observable variable and target variable ; A generative model can be used to
generate
random instances (outcomes) of an observation . - A discriminative model is a model of the conditional probability of the target , given an observation . It can be used to
discriminate
the value of the target variable , given an observation . - Classifiers computed without using a probability model are also referred to loosely as
discriminative
The distinction between these two classes is not consistently made.
Standard examples of each, all of which are linear classifiers, are:
- generative classifiers
- naive Bayes classifier
- linear discriminant analysis
- discriminative model
- logistic regression
In application to classification, one wishes to go from an observation to label (or probability distribution on labels). One can compute this directly, without using a probability distribution; one can estimate the probability of a label given an observation (discriminative model) and base classification on that, or one can estimate the joint distribution (generative model).
An alternative division defines these symmetrically as:
- a generative model is a model of the conditional probability of the observable , given a target , symbolically,
- a discriminative model is a model of the conditional probability of the target , given an observation , symbolically,
Regardless of precise definition, the terminology is constitutional because a generative model can be used to generate
random instances (outcomes), either of an observation and target , or of an observation given a target value , while a discriminative model or discriminative classifier (without a model) can be used to discriminate
the value of the target variable m given an observation .
The term generative model
is also used to describe models that generate instances of output variables in a way that has no clear relationship to probability distributions over potential samples of input variables. Generative adversarial networks are examples of this class of generative models, and are judged primarily of particular outputs to potential inputs.
Comments
You have to be logged in to add a comment
User Comments
There are currently no comments for this article.