Hypothesis Test

The process of determining ``yes'' or ``no'' from the outcome of experiment is called a hypothesis test. A widely used formalization of this procedure is due to Neyman and Pearson. Suppose that a researcher is interested in whether a new drug works. Then null hypothesis may be that the drug has no effect --it is often the reverse of what he or she actually believe, why? Because the researcher hopes to reject the hypothesis and announce that the new drug leads to significant improvements. If the null hypothesis is not rejected, the researcher announces nothing and goes on to a new experiment.

Here we are interested in the plausibility of the hypothesis

$ H_A:\hspace{0.05in}\mu$ $ \mu_0 =$

regarding the ``true'' population mean $ \mu$. $ H_A$ is called an alternative hypothesis, and together with the null value $ \mu_0$ it forms the basis of hypothesis testing. The null hypothesis  $ H_0: \mu = \mu_0$ is used in the context of determining whether we can reject ``$ H_0$ in favor of $ H_A$.''

The test procedure, known as t-test, is based upon the sample mean $ \bar{X}$ and the sample standard deviation $ S$ from a data set of size $ n$. Then the discrepancy between the sample mean $ \bar{X}$ and the ``assumed'' null value $ \mu_0$ of population mean is measured by the test statistic $ T = \displaystyle\frac{\bar{X} - \mu_0}{S/\sqrt{n}} $

The significance level $ \alpha =$ has to be chosen from $ \alpha =$ 0.01 or 0.05 ($ \alpha =$ 0.1 is not common in this particular test). Under the null hypothesis $ H_0$, it is ``unlikely'' that the t-statistic $ T$ lies in the critical region specified in the table below. If so, it suggests significant evidence against the null hypothesis $ H_0$ in favor of $ H_A$.

Alternative hypothesis Critical region to reject $ H_0$
$ H_A: \mu \neq \mu_0$ $ \vert T\vert > t_{\alpha/2,n-1} =$
$ H_A: \mu < \mu_0$ $ T < -t_{\alpha,n-1} =$
$ H_A: \mu > \mu_0$ $ T > t_{\alpha,n-1} =$

Alternatively, the p-value can be calculated so that ``p-value $ < \alpha$'' is equivalent to the t-statistic $ T$ being observed in the critical region. When the null hypothesis $ H_0$ is rejected (i.e., p-value $ < \alpha$), it is reasonable to calculate the confidence interval estimating the population mean $ \mu$.