Randomized experiments tell us *if* a program works. But to build better theories and policies, we need to know **how and why**. This is the challenge of causal mediation.
We want to know if a Treatment (T) causes an Outcome (Y) by changing a Mediator (M). The problem lies in the second half of this chain.
While we randomize T → M, the M → Y link is purely correlational. Hover over the red arrow to see why this is a huge problem for causal inference.
The best way to test a mechanism isn't with post-hoc stats, but with better *a priori* experimental designs. The stronger the design, the more credible the causal claim.
Randomize T, then measure M and Y. This is the most common but weakest design.
LIMITATION: Relies entirely on the untestable sequential ignorability assumption. High risk of bias.
Design a treatment that directly blocks or mimics the proposed mechanism (M) to see its effect on Y.
STRENGTH: Great for ruling out theories without measuring the mediator. Strong causal leverage.
Conduct two separate experiments: one to prove T→M, and a second to prove M→Y.
LIMITATION: Vulnerable to the "product fallacy" if populations differ across experiments.
Randomly encourage uptake of the mediator, using the encouragement as an instrumental variable.
STRENGTH: State-of-the-art. Identifies mediation with weaker, more plausible assumptions.
A famous "mechanism experiment" tested if social pressure is the key mechanism in Get-Out-The-Vote mailings. Instead of measuring "pressure," they designed treatments with different levels of it. Click the buttons to see the results.
The "Neighbors" mailing, which revealed recipients' voting records to their neighbors, had a massive effect. This provides strong, implicit evidence that social pressure is the key causal mechanism.
Don't just test for mediation... DESIGN for it.
A well-designed experiment provides more compelling evidence for a causal pathway than a complex statistical analysis of a poorly designed one. The future of mediation analysis is in the design.