A Complete Guide to ANOVA Post Hoc Tests
Introduction
Post hoc (Latin for "after this") tests are statistical analyses carried out after an ANOVA to explore differences between group means. These tests help identify which specific group pairs are significantly different while adjusting for the increased risk of Type I errors that arise when multiple comparisons are made.
The ANOVA and Post Hoc Testing Process
- Conduct ANOVA to determine if there are significant differences between groups
- If ANOVA is significant, proceed with post hoc tests
- Choose appropriate post hoc test based on data characteristics
- Perform pairwise comparisons with adjusted p-values
- Interpret results to identify specific group differences
While ANOVA evaluates the overall variance among groups, it doesn't detail which pairs of groups differ significantly. Performing multiple pairwise comparisons without correction increases the likelihood of making errors, which post hoc tests aim to mitigate.
Common Post Hoc Tests
Tukey's Honestly Significant Difference (HSD) Test
As one of the most commonly used post hoc tests, Tukey's HSD is designed for pairwise comparisons between all possible pairs of groups. It effectively controls the familywise error rate by adjusting p-values and assumes equal variances among groups.
Bonferroni Correction
The Bonferroni correction is a simple and conservative method that adjusts the significance level based on the number of comparisons being made. While it's effective at reducing Type I errors, it can increase the risk of Type II errors due to its conservative nature.
Scheffé's Test
Scheffé's test allows for both pairwise and more complex comparisons. It's more conservative than Tukey's HSD but maintains control over the familywise error rate. This test is particularly suitable when sample sizes are unequal or variances cannot be assumed equal.
Dunnett's Test
Dunnett's test is specifically designed for comparing multiple treatment groups to a single control group. It's less conservative compared to other methods, as it focuses on fewer comparisons. This makes it ideal when you're not interested in all pairwise comparisons but only in comparisons with a control group.
Games-Howell Test
The Games-Howell test is particularly useful when you can't assume equal variances or sample sizes. It's more appropriate for situations where group variances differ significantly, though it's less commonly used than Tukey's HSD.
Post Hoc Tests Comparison
| Test | Best Used When | Assumptions | Conservativeness |
|---|---|---|---|
| Tukey's HSD | All pairwise comparisons needed | Equal sample sizes, equal variances | Moderate |
| Bonferroni | Small number of planned comparisons | Any | Very Conservative |
| Dunnett's | Comparing to control group | Equal variances | Liberal for control comparisons |
| Games-Howell | Unequal variances | No equality of variances needed | Moderate |
| Scheffé's | Complex comparisons needed | Equal variances | Very Conservative |
Find Your Post Hoc Test
Adjust these parameters to get a recommendation:
Recommended Test:
Tukey's HSD
Test Your Knowledge
Which post hoc test is most appropriate when comparing multiple groups to a single control group?
Additional Resources
Help us improve
Found an error or have a suggestion? Let us know!