Homepage > MCMC

Gibbs Distribution

Let $ G = (V,E)$ be a finite undirected graph. The site $ v$ is said to be a neighbor of $ w$ if $ \{v,w\}\in E$. $ C \subset V$ is called a clique if $ \{v,w\}\in E$ whenever $ v,w\in C$. We denote the set of all cliques by $ \mathcal{C}$. Let $ \Lambda := \{1,\ldots,L\}$ be a common state space. Given a parameter $ \theta > 0$, we can define a Gibbs distribution with respect to $ G$ by

$\displaystyle \pi(x) := \frac{1}{z_\theta} \exp(-\theta H(x)),
\quad x\in S


$\displaystyle z_\theta = \sum_{x\in S} \exp(-\theta H(x))

is the normalizing constant.

Hamiltonian. In the language of statistical mechanics, a vertex $ v\in V$ is called a site. The parameter $ \theta$ and $ z_\theta$ are respectively called a ``inverse temperature'' and ``partition function.'' And $ H(x)$, called a ``Hamiltonian,'' has the form

$\displaystyle H(x) = \sum_{C \in \mathcal{C}} W_C(x),

where each $ W_C$ depends only on those coordinates $ x_v$, $ v\in C$. The Hamiltonian $ H(x)$ is a energy function, and the model abhors to retain a high energy when the temperature $ T = 1/\theta$ is low.

Markov random field. Let $ X = (X_v)$ be a $ S$-valued random variable. $ X$ is an MRF (Markov random field) with respect to $ G$ if for every $ v\in V$ and $ \mathbf{x}\in\mathcal{S}$

  1. $ P(X = x) > 0$;
  2. $ P(X_v = x_v \vert X_w = x_w, w \neq v )$

    $ = {P(X_v = x_v \vert X_w = x_w, \{w,v\}\in E )}$.

Hammersley-Clifford theorem. $ \mathbf{X}$ is an MRF with respect to $ G$ if and only if $ \pi(\mathbf{x}) = P(\mathbf{X} = \mathbf{x})$ is a Gibbs distribution with respect to $ G$. In this sense a Gibbs distribution is also known as GRF (Gibbs random field).

Site update. In the Gibbs distribution, the conditional probability for site update is given by

$ \pi(x_v \vert (x_w)_{ w\neq v})$ $ = \dfrac{\exp(-\theta H_v(y))}
{\sum_{\lambda\in\Lambda} \exp(-\theta H_v(x; x_v\gets\lambda))}$

Here the normalizing constant $ z_\theta$ is canceled and

$\displaystyle H_v(x) = \displaystyle\sum_{v\in C, C\in\mathcal{C}} W_C(x)

is only the partial sum over neighboring cliques of $ v$.

© TTU Mathematics