EZ Statistics

Sample Size Calculator

Calculator

Parameters

Range: 0.1 to 2 (standard deviations)

Results

n = 0

Notes:

  • Effect size interpretations vary by field and context
  • For mean difference tests, effect size is in standard deviation units
  • Power of 80% (0.8) is typically considered adequate
  • Significance level of 5% (0.05) is conventional in many fields
  • The power curve shows how the statistical power changes with different effect sizes
  • Larger sample sizes can detect smaller effect sizes with the same power

Learn More

Why Sample Size Matters in Research?

The Importance of Sample Size

Sample size calculation is a crucial step in research design and hypothesis testing. It helps you:

  • Ensure your study has adequate statistical power to detect meaningful effects
  • Avoid wasting resources on studies that are too large
  • Maintain ethical standards by not using too few or too many participants
  • Make informed decisions about resource allocation

Warning: Conducting a study with inadequate sample size can lead to:

  • False negatives (Type II errors) - failing to detect real effects
  • Unreliable results and wasted resources
  • Inability to draw meaningful conclusions

A/B Testing Example

Scenario: Website Conversion Rate

You're testing a new button design and want to detect a 2% increase in conversion rate (from 10% to 12%).

Without proper sample size calculation:

Too Small (100 visitors/group)
  • Control: 10 conversions (10%)
  • Test: 12 conversions (12%)
  • Result: Not statistically significant despite real effect
Proper Size (2000 visitors/group)
  • Control: 200 conversions (10%)
  • Test: 240 conversions (12%)
  • Result: Can detect the real difference

Required Calculations

For this example, we need:

  • Significance level: α = 0.05
  • Power: 1-β = 0.80
  • Baseline rate: p₁ = 0.10
  • Expected rate: p₂ = 0.12
  • Effect size: |p₂ - p₁| = 0.02

Common Mistakes to Avoid

Underpowered Studies

  • Unable to detect meaningful effects
  • Waste of time and resources
  • Inconclusive results
  • Potential ethical issues

Overpowered Studies

  • Excessive resource usage
  • Detection of trivial effects
  • Unnecessary participant burden
  • Inflated costs

Best Practices

  • Always calculate sample size before starting data collection
  • Consider practical significance, not just statistical significance
  • Account for potential dropout or missing data
  • Document your sample size calculations and assumptions
  • Consider conducting a pilot study if parameters are unknown

Sequential Testing and Early Stopping

While traditional sample size calculation is crucial, modern A/B testing platforms often use sequential testing approaches:

Sequential Analysis

  • Continuously monitor results
  • Stop early if effect is clear
  • Adjust for multiple looks
  • More efficient use of resources

Required Adjustments

  • Use adjusted significance levels
  • Account for peeking
  • Consider false discovery rate
  • Monitor effect size stability

Key Takeaway

Whether using traditional fixed-sample approaches or modern sequential methods, proper planning of sample size and monitoring procedures is essential for valid and reliable results.

How to Calculate Sample Size for Different Tests?

Two-Sample Mean Difference

For comparing two independent means, the sample size per group is:

n=2(zα/2+zβ)2d2n = \frac{2(z_{\alpha/2} + z_{\beta})^2}{d^2}

where:

  • zα/2z_{\alpha/2}: Critical value for Type I error rate (1.96 for α = 0.05)
  • zβz_{\beta}: Critical value for Type II error rate (0.84 for power = 0.80)
  • dd: Cohen's d (standardized effect size) = (μ₁ - μ₂)/σ
Note: For unequal group sizes with allocation ratio r = n₂/n₁:
n1=2(zα/2+zβ)2(1+1/r)d2n_1 = \frac{2(z_{\alpha/2} + z_{\beta})^2(1 + 1/r)}{d^2}n2=r×n1n_2 = r \times n_1

Paired Difference Test

For paired samples, the required number of pairs is:

n=2(zα/2+zβ)2(1−ρ)d2n = \frac{2(z_{\alpha/2} + z_{\beta})^2(1-\rho)}{d^2}

where:

  • ρ\rho: Correlation between paired measurements
  • dd: Effect size = (μ₁ - μ₂)/σ

Note: Higher correlation between pairs reduces the required sample size, making paired designs more efficient when correlation is strong.

Proportion Test

For comparing two proportions, the required sample size per group is:

n=2(zα/2+zβ)2h2n = \frac{2(z_{\alpha/2} + z_{\beta})^2}{h^2}

where:

  • hh: Cohen's h = 2arcsin⁡(p1)−2arcsin⁡(p2)2\arcsin(\sqrt{p_1}) - 2\arcsin(\sqrt{p_2})
  • p1,p2p_1, p_2: Expected proportions in each group

Cohen's h Effect Size Guidelines:

  • Small: h = 0.2
  • Medium: h = 0.5
  • Large: h = 0.8

One-Way ANOVA

For one-way ANOVA with k groups, the sample size per group is:

n=(zα/k+zβ)2f2n = \frac{(z_{\alpha/k} + z_{\beta})^2}{f^2}

where:

  • ff: Cohen's f effect size = η2/(1−η2)\sqrt{\eta^2/(1-\eta^2)}
  • kk: Number of groups
  • η2\eta^2: Proportion of variance explained

Cohen's f Effect Size Guidelines:

  • Small: f = 0.10
  • Medium: f = 0.25
  • Large: f = 0.40

Advanced Learn More

Statistical Assumptions and Derivations

Two-Sample Mean Difference: Assumptions & Derivation

Key Assumptions

  • Data is normally distributed in each group
  • Equal variances between groups (homoscedasticity)
  • Independent observations within and between groups
  • Random sampling from the target population

Derivation Steps

  1. Start with the formula for the t-test statistic:
    t=Xˉ1−Xˉ2s12n1+s22n2t = \frac{\bar{X}_1 - \bar{X}_2}{\sqrt{\frac{s^2_1}{n_1} + \frac{s^2_2}{n_2}}}
  2. Under H₁, this follows a non-central t-distribution with non-centrality parameter:
    λ=μ1−μ2σ12n1+σ22n2\lambda = \frac{\mu_1 - \mu_2}{\sqrt{\frac{\sigma^2_1}{n_1} + \frac{\sigma^2_2}{n_2}}}
  3. For equal sample sizes and variances:
    λ=δσn2\lambda = \frac{\delta}{\sigma}\sqrt{\frac{n}{2}}
  4. Solve for n using the relationship between λ and power:
    n=2(zα/2+zβδ/σ)2n = 2\left(\frac{z_{\alpha/2} + z_{\beta}}{\delta/\sigma}\right)^2

Paired Difference Test: Assumptions & Derivation

Key Assumptions

  • Differences between pairs are normally distributed
  • Pairs are independent of each other
  • Measurements are collected under similar conditions
  • The relationship between pairs is linear

Derivation Steps

  1. For paired data, define the difference scores:
    Di=Xi1−Xi2D_i = X_{i1} - X_{i2}
  2. The variance of differences is related to original variances:
    σd2=σ12+σ22−2ρσ1σ2\sigma^2_d = \sigma^2_1 + \sigma^2_2 - 2\rho\sigma_1\sigma_2
  3. For equal variances:
    σd2=2σ2(1−ρ)\sigma^2_d = 2\sigma^2(1-\rho)
  4. Apply to the standard sample size formula:
    n=(zα/2+zβ)2σd2δ2n = \frac{(z_{\alpha/2} + z_{\beta})^2\sigma^2_d}{\delta^2}

Proportion Test: Assumptions & Derivation

Key Assumptions

  • Binary outcome (success/failure)
  • Independent observations
  • Random sampling
  • Large enough sample size for normal approximation

Derivation Steps

  1. Start with the normal approximation to binomial:
    Z=p^1−p^2p1q1n1+p2q2n2Z = \frac{\hat{p}_1 - \hat{p}_2}{\sqrt{\frac{p_1q_1}{n_1} + \frac{p_2q_2}{n_2}}}
  2. For equal sample sizes:
    n=(zα/22pˉqˉ+zβp1q1+p2q2)2(p1−p2)2n = \frac{(z_{\alpha/2}\sqrt{2\bar{p}\bar{q}} + z_{\beta}\sqrt{p_1q_1 + p_2q_2})^2}{(p_1-p_2)^2}
  3. Include continuity correction:
    ncorrected=n(1+1+4n∣p1−p2∣)2/4n_{corrected} = n\left(1 + \sqrt{1 + \frac{4}{n|p_1-p_2|}}\right)^2/4

One-Way ANOVA: Assumptions & Derivation

Key Assumptions

  • Normal distribution within each group
  • Equal variances across groups
  • Independent observations
  • Random sampling from populations

Derivation Steps

  1. F-statistic under alternative hypothesis:
    F=MSbetweenMSwithin∼Fk−1,N−k,λF = \frac{\text{MS}_\text{between}}{\text{MS}_\text{within}} \sim F_{k-1,N-k,\lambda}
  2. Non-centrality parameter:
    λ=n∑i=1k(μi−μ)2kσ2\lambda = \frac{n\sum_{i=1}^k(\mu_i-\mu)^2}{k\sigma^2}
  3. For equal differences between means:
    λ=nkδ22σ2(k−1)\lambda = \frac{nk\delta^2}{2\sigma^2(k-1)}
  4. Solve for n:
    n=2(k−1)(Fα,k−1,∞+Fβ,k−1,∞)σ2kδ2n = \frac{2(k-1)(F_{\alpha,k-1,\infty}+F_{\beta,k-1,\infty})\sigma^2}{k\delta^2}

Related Calculators

Power Analysis Calculator

Go to Calculator

Two Sample Paired T-Test Calculator

Go to Calculator

Two Proportion Z-Test Calculator

Go to Calculator

One-Way ANOVA Calculator

Go to Calculator

Help us improve

Found an error or have a suggestion? Let us know!