A random sample
is regarded as
independent and identically distributed (iid) random variables
governed by an underlying probability density function
represents the characteristics of this underlying
distribution, and is called a parameter
A point estimate
is a ``best guess'' for the true value
Suppose that the underlying distribution
is the normal distribution with
Then the sample mean
is in some sense a best guess of the parameter
Maximum likelihood estimate.
Having observed a random sample
from an underlying pdf
we can construct the likelihood function
and consider it as a function of
Then the maximum likelihood estimate (MLE)
is the value of
which ``maximizes'' the likelihood
It is usually easier to maximize the log likelihood
Let be a random variable taking value only on 0 and 1.
An experiment observing such a variable is called
It is determined by the parameter
(which represents the probability that ).
are observed from independent Bernoulli trials.
Then the joint density function is given by
By solving the equation
and therefore, it is the MLE of
© TTU Mathematics