Confidence Interval for the Difference Between Means

A confidence interval for the difference between means provides a range of values where the true difference between two population means likely falls, with a certain level of confidence. It gives both an estimate of the difference and a measure of the uncertainty associated with that estimate.

To compute a confidence interval for the difference between means, we use the following formula:

Where:

• $\bar x_1$ and $\bar x_2$ are the sample means
• The critical value depends on the confidence level and whether you're using a z-test or t-test
• $\text{SE}_{\bar x_1 - \bar x_2}$ is the standard error of the difference between means, and it varies depending on the test type:
  • $z$-test (known population variance): $\sqrt{\frac{\sigma_1^2}{n_1} + \frac{\sigma_2^2}{n_2}}$
  • Independent $t$-test (equal variances): $s_p \sqrt{\frac{1}{n_1} + \frac{1}{n_2}}$
  • Independent $t$-test (unequal variances): $\sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}$
  • Paired $t$-test: $\frac{s_d}{\sqrt{n}}$

In the case of paired data (e.g., before-and-after measurements), the formula for the standard error of the difference between means is:

Where:

• $s_1$ and $s_2$ are the standard deviations of the two groups
• $n$ is the sample size (number of pairs)
• $r$ is the correlation coefficient between the paired observations

The confidence interval for paired data is calculated as:

Where:

• $\bar d$ is the mean difference between pairs
• $t_{\alpha/2, n-1}$ is the critical t-value for the chosen confidence level and degrees of freedom

A 95% confidence interval means that if you repeated the sampling process many times, about 95% of the intervals calculated would contain the true difference between population means. However, it does not mean there's a 95% probability that the specific interval from your sample contains the true difference.

To accurately interpret and apply the confidence interval, the following assumptions should hold:

• Random, representative samples from the populations
• The data in each group follows a normal distribution or the sample sizes are large enough ($n \gt 30$)
• For z-tests, population standard deviations are known
• For independent t-tests, the two groups are independent
• For paired t-tests, the paired observations are dependent