This simulation illustrates the behavior of p-values and hypothesis testing outcomes. By adjusting parameters like sample size and significance level, users can observe how these affect the likelihood of Type I errors and the stability of sample means.
The probability of observing results as extreme as, or more extreme than, the observed results, assuming the null hypothesis is true. A p-value smaller than the significance level (α) suggests that the observed data is unlikely under the null hypothesis, leading to its rejection.
The hypothesis that there is no effect or no difference. In hypothesis testing, the null hypothesis is assumed to be true unless there is strong evidence against it.
What we assume if the null hypothesis is rejected. The alternative hypothesis is considered if the data provides sufficient evidence to reject the null hypothesis.
The threshold used to decide whether to reject the null hypothesis. It represents the Type I error rate (i.e., the probability of rejecting a true null hypothesis). It's often set to 5% (α = 0.05).
The proportion of times the null hypothesis is rejected in the simulations. A high rejection rate, especially when the null hypothesis is true, suggests a higher risk of Type I errors (false positives).
The probability of correctly rejecting the null hypothesis when the alternative hypothesis is true. Power is influenced by factors like sample size, effect size, and significance level. Higher power means the test is more likely to detect true effects and avoid Type II errors (false negatives).