Homepage > MCMC

## Frequentist Approach

A random sample

is regarded as independent and identically distributed (iid) random variables governed by an underlying probability density function  . A value represents the characteristics of this underlying distribution, and is called a parameter. A point estimate is a best guess'' for the true value . Suppose that the underlying distribution is the normal distribution with . Then the sample mean

is in some sense a best guess of the parameter .

Maximum likelihood estimate. Having observed a random sample from an underlying pdf , we can construct the likelihood function

and consider it as a function of . Then the maximum likelihood estimate (MLE)  is the value of  which maximizes'' the likelihood function  . It is usually easier to maximize the log likelihood

Bernoulli trials. Let be a random variable taking value only on 0 and 1. An experiment observing such a variable is called Bernoulli trial. It is determined by the parameter (which represents the probability that ). Suppose that are observed from independent Bernoulli trials. Then the joint density function is given by

By solving the equation

we obtain . In fact, maximizes , and therefore, it is the MLE of .