Homepage > MCMC

## Mixture Model

A simple mixture model has a probability density function

with weight parameter . Here the components are probability density functions, and assumed to be entirely known.

Posterior density. Given the data of independent observations from the mixture density and the flat prior, the posterior density of weight parameter is proportional to

However, the numerical analysis of posterior density on the simplex

is rather very hard. Alternatively, we can devise an MCMC scheme.

Latent Variables. Then we introduce the following latent variable setup: Let be a latent variable indicating to which component the -th observation  belongs, and let be the vector of latent variables on the space

By we denote the indicator function or 0 accordingly as or not. So that we can define

In short, denotes the number of 's set to . In this manner the latent variable  can be together lumped into .