Beyond "What Works" to "How it Works"
A randomized experiment can tell us *if* a program works. But to build better theories and policies, we need to know *why* it works. Mediation analysis aims to uncover these causal pathways, but it comes with a fundamental challenge.
The Central Challenge
In a typical experiment, we want to know if a Treatment (T) causes an Outcome (Y) by changing a Mediator (M).
While we randomize the Treatment (T → M), we only *observe* the relationship between the Mediator and the Outcome (M → Y). This link is correlational. Claiming it's causal requires the strong, untestable assumption of **Sequential Ignorability**. Hover over the arrow above to learn more.
A Hierarchy of Experimental Designs
The most robust way to test a mechanism isn't with post-hoc statistics, but with better *a priori* experimental designs. Explore the toolkit below to see how different designs provide stronger causal evidence.
Measurement-of-Mediation
This is the standard, most common design. You randomize the treatment, then measure both the mediator and the outcome.
Strength: Feasible & Simple. Weakness: Relies entirely on the untestable Sequential Ignorability assumption.
Implicit Mediation (Mechanism Experiment)
Instead of measuring the mediator, you design a treatment that directly blocks or mimics the proposed mechanism.
(e.g., Social Pressure Mail)
Strength: Powerful for falsifying theories without measuring M. Weakness: Doesn't quantify the indirect effect.
Experimental-Causal-Chain
Conduct two separate experiments: one to prove T causes M, and a second to prove M causes Y.
Strength: Provides experimental evidence for both links. Weakness: Vulnerable to the "product fallacy" if populations differ.
Parallel Encouragement Design (PED)
A state-of-the-art design where you randomly encourage uptake of the mediator, using the encouragement as an instrumental variable.
Strength: Identifies mediation with weaker assumptions. Weakness: Complex; requires a valid "encouragement" instrument.
Case Study: Social Pressure & Voter Turnout
A classic "mechanism experiment" tested if social pressure is the key mechanism in Get-Out-The-Vote mailings. Instead of measuring "pressure," they designed treatments with different levels of it. Click the buttons to see the results.
The "Neighbors" mailing, which revealed recipients' voting records to their neighbors, had a massive effect. This provides strong, implicit evidence that social pressure is the key mechanism.
Practical Challenges in the Field
Real-world experiments are messy. Two common issues that can complicate mediation analysis are noncompliance and attrition.
Treatment Noncompliance
What happens when people don't do what they're assigned? For example, some people offered a training program (treatment) don't attend. This means we can only estimate the causal effects for the sub-population of "compliers," often using an instrumental variables approach.
Participant Attrition
What happens when people drop out of your study? If dropouts are systematically different between treatment and control groups (**differential attrition**), it can bias the estimated causal pathways. This requires careful checks and potential statistical corrections like bounding or weighting.