e-Statistics

Inference on Parameters

The concepts of hypothesis tests and confidence intervals can be applied to linear regression models. Inferences on the slope parameter $ \beta_1$ is of particular interest since it determines the nature of relationship between the explanatory and the dependent variable.

The data set consists of

  1. explanatory variable for $ x_i$'s;
  2. dependent variable for $ Y_i$'s.
The analysis for simple linear regression are summarized in the following table. Under the standard assumption of regression model we can make hypotheses for the coefficients $ \beta_0$ and $ \beta_1$, and test them via p-value.

  1. $ \displaystyle S_1 = \frac{\hat{\sigma}}{\sqrt{S_{xx}}}$ is the standard error for the estimate $ \beta_1$ of slope. The null hypothesis

    $\displaystyle H_0:\: \beta_1 = 0
$

    can be constructed to find whether the response variable is dependent of the explanatory variable. Under the null hypothesis the test statistic $ T_1 = \displaystyle\frac{\hat{\beta}_1}{S_1}$ is distributed as the $ t$-distribution with $ (n-2)$ degrees of freedom. Thus, we reject $ H_0$ at significance level $ \alpha$ if $ \vert T_1\vert > t_{\alpha/2,n-2}$. By computing the p-value $ p_1^*$ we can equivalently reject $ H_0$ if $ p_1^* < \alpha$. When $ H_0$ is rejected, we can proceed to construct the confidence interval of level $ (1-\alpha) =$ for the coefficient $ \beta_1$ as

    $ \displaystyle\left(\hat{\beta}_1 - t_{\alpha/2,n-2} S_1,\:
\hat{\beta}_1 + t_{\alpha/2,n-2} S_1 \right) =$ ( , )

  2. $ \displaystyle
S_0 = \hat{\sigma} \sqrt{\frac{1}{n} + \frac{\bar{x}^2}{S_{xx}}}$ is the standard error for the estimate $ \beta_0$ of intercept. The null hypothesis

    $\displaystyle H_0:\: \beta_0 = 0
$

    may not be of particular interest, but the procedure for hypothesis testing can be similarly proceeded. Under the null hypothesis the test statistic $ T_0 = \displaystyle\frac{\hat{\beta}_0}{S_0}$ has the $ t$-distribution with $ (n-2)$ degrees of freedom, and $ H_0$ is reject at significance level $ \alpha$ if $ \vert T_0\vert > t_{\alpha/2,n-2}$, or, equivalently when the $ p$-value $ p_0^*$ satisfies $ p_0^* < \alpha$. Then the confidence interval of level $ (1-\alpha)$ for the coefficient $ \beta_0$ as
    $ \displaystyle\left(\hat{\beta}_0 - t_{\alpha/2,n-2} S_0,\:
\hat{\beta}_0 + t_{\alpha/2,n-2} S_0 \right) =$ ( , )