What is ANOVA?#

ANOVA (Analysis of Variance) is a statistical method for comparing the means of three or more groups. It tests whether at least two groups differ significantly from each other.

Why Not Just Run Multiple t-Tests?#

With three groups (A, B, C), you could run three t-tests: A vs. B, A vs. C, B vs. C. The problem:

P(\text{at least one error}) = 1 - (1 - 0.05)^3 = 0.143

The probability of at least one false positive rises to 14.3% instead of the intended 5%. With 10 groups, there would be 45 pairwise comparisons and an error probability of nearly 90%.

ANOVA solves this problem by testing all groups simultaneously.

The Core Logic#

ANOVA partitions the total variance of the data into two components:

Between-group variance (systematic variation): Differences explained by the group factor
Within-group variance (error variance): Random variation within each group

SS_{\text{total}} = SS_{\text{between}} + SS_{\text{within}}

If the variance between groups is substantially larger than the variance within groups, this suggests an effect.

The F-Statistic#

F = \frac{MS_{\text{between}}}{MS_{\text{within}}} = \frac{SS_{\text{between}} / df_{\text{between}}}{SS_{\text{within}} / df_{\text{within}}}

F ≈ 1: Group differences are no larger than random variation
F >> 1: Group differences are larger than expected → significant

Types of ANOVA#

One-Way ANOVA#

One factor (independent variable) with k groups.

Example: One-way ANOVA

Comparing three diet programs on weight loss:

Factor: Diet program (A, B, C)
DV: Weight loss in kg
Question: Do the three programs differ in mean weight loss?

Two-Way (Factorial) ANOVA#

Two or more factors are examined simultaneously. Allows analysis of main effects and interaction effects.

Example: Two-way ANOVA

Factor 1: Therapy type (Drug A, B)
Factor 2: Gender (male, female)
DV: Symptom reduction

Possible results:

Main effect of therapy: Drug A works better than B (regardless of gender)
Main effect of gender: Women show more reduction (regardless of therapy)
Interaction: Drug A works better for women, but worse for men compared to B

Repeated-Measures ANOVA#

The same participants are measured multiple times (e.g., before, during, and after treatment).

Advantage: More power because individual differences are controlled
Additional assumption: Sphericity (Mauchly's test)

Mixed ANOVA#

Combination of between-subjects factors and within-subjects factors.

Example: Mixed ANOVA

Between-subjects factor: Therapy group (treatment vs. control)
Within-subjects factor: Time point (pre, post, follow-up)
DV: Anxiety level

The key question: Does the trajectory over time differ between groups? (Group × Time interaction)

Assumptions#

Metric dependent variable
Independence of observations (between groups)
Normal distribution of residuals in each group
Homogeneity of variance (Levene's test)
Sphericity (repeated measures only, Mauchly's test)

Post-Hoc Tests#

ANOVA only tells you that at least two groups differ, but not which ones. Post-hoc tests are needed for this:

Test	Property
Tukey HSD	Most commonly used, all pairwise comparisons
Bonferroni	Conservative, for few planned comparisons
Scheffé	Very conservative, also for complex contrasts
Games-Howell	For variance heterogeneity (homogeneity not required)

Interpreting post-hoc results

One-way ANOVA with three groups (A, B, C) yields F(2, 57) = 5.34, p = .008.

Tukey HSD post-hoc test shows:

A vs. B: p = .006 (significant)
A vs. C: p = .042 (significant)
B vs. C: p = .784 (not significant)

→ Group A differs from B and C, but B and C do not differ from each other.

Effect Sizes#

Measure	Small	Medium	Large
η² (Eta-squared)	0.01	0.06	0.14
ω² (Omega-squared)	0.01	0.06	0.14
Cohen's f	0.10	0.25	0.40

Conversion:

f = \sqrt{\frac{\eta^2}{1 - \eta^2}}

ANOVA Table#

A typical ANOVA result is reported as a table:

Source	SS	df	MS	F	p
Between	120.5	2	60.25	5.34	.008
Within	643.2	57	11.28
Total	763.7	59

Non-Parametric Alternatives#

ANOVA type	Non-parametric alternative
One-way ANOVA	Kruskal-Wallis test
Repeated-measures ANOVA	Friedman test

Common Misconceptions#

"ANOVA compares variances." The name is misleading. ANOVA uses variances as a tool, but the goal is comparing means.

"A significant ANOVA result tells me which groups differ." No. The ANOVA result (omnibus test) only says that a difference exists somewhere. Post-hoc tests identify the specific differences.

"ANOVA only works with equal group sizes." No. ANOVA works with unequal groups too, but becomes more sensitive to violations of variance homogeneity.

"You cannot use ANOVA with only two groups." Yes, you can. With two groups, ANOVA yields the same result as the t-test. It holds that: $F = t^2$ .

What is ANOVA?