A

random sample

is regarded as
independent and identically distributed (iid) random variables
governed by an underlying probability density function

.
A value

represents the characteristics of this underlying
distribution, and is called a

parameter.
A

point estimate is a ``best guess'' for the true value

.
Suppose that the underlying distribution
is the normal distribution with

.
Then the sample mean

is in some sense a best guess of the parameter

.

Maximum likelihood estimate.
Having observed a random sample
from an underlying pdf
,
we can construct the likelihood function

and consider it as a function of

.
Then the

maximum likelihood estimate (MLE)
is the value of

which ``maximizes'' the likelihood
function

.
It is usually easier to maximize the

log likelihood

Bernoulli trials.
Let be a random variable taking value only on 0 and 1.
An experiment observing such a variable is called
Bernoulli trial.
It is determined by the parameter
(which represents the probability that ).
Suppose that
are observed from independent Bernoulli trials.
Then the joint density function is given by

By solving the equation

we obtain

.
In fact,

maximizes

,
and therefore, it is the MLE of

.

© TTU Mathematics