MCMC

Mixture Model

A simple mixture model has a probability density function

$\displaystyle \theta_1 f_1(\xi) + \cdots + \theta_k f_k(\xi)$

with weight parameter $\theta = (\theta_1,\ldots,\theta_k)$ . Here the components $f_1, \ldots, f_k$ are probability density functions, and assumed to be entirely known.

Posterior density. Given the data $x = (x_1,\ldots,x_n)$ of independent observations from the mixture density and the flat prior, the posterior density $\pi^\star(\theta)$ of weight parameter is proportional to

$\displaystyle f(x \:\vert\: \theta) = \prod_{i=1}^n \: [\theta_1 f_1(x_i) + \cdots + \theta_k f_k(x_i)].$

However, the numerical analysis of posterior density $\pi^\star(\theta)$ on the simplex

$\displaystyle \mathcal{Q} = \{\theta\in (0,1)^k: \sum_{j=1}^k \theta_j = 1\}$

is rather very hard. Alternatively, we can devise an MCMC scheme.

Latent Variables. Then we introduce the following latent variable setup: Let be a latent variable indicating to which component the -th observation belongs, and let $z = (z_1,\ldots,z_n)$ be the vector of latent variables on the space