**Attrition**, the loss of participants over time, is the most persistent threat to the validity of field experiments. It can break the balance created by randomization, reintroducing the very selection bias an experiment is designed to eliminate.
Bias arises from **differential attrition**, where the rate or type of dropout differs between groups. This means the people who remain in the treatment group are no longer comparable to those in the control group.
A higher dropout rate in one group is the first warning sign. Here, the treatment group's rate is more than double the control's.
Because of differential attrition, the remaining groups are no longer balanced on key pre-treatment characteristics like income.
When attrition occurs, we can't be certain of the true treatment effect. Bounding calculates a range of plausible effects under different assumptions about the missing participants.
Manski bounds are assumption-free but wide. Lee bounds are tighter but require the untestable "monotonicity" assumption.
When willing to make stronger assumptions, researchers can use statistical models to generate a single point estimate of the treatment effect.
⚖️
Gives more weight to observed individuals who are similar to those who dropped out, creating a re-weighted "pseudo-population".
🔧
Explicitly models the attrition process itself, attempting to correct for bias caused by unobserved factors.
✨
"Fills in" the missing data multiple times based on observed patterns, creating several complete datasets for analysis.
The most effective strategy is to minimize attrition from the start with thoughtful fieldwork and design.
Be transparent, use empathetic survey staff, and express appreciation to make participants feel like valued partners.
Use cash-equivalent incentives and a phased or completion-bonus structure to motivate long-term participation.
At baseline, get multiple phone numbers, emails, and contact details for at least two secondary informants.
Keep surveys concise, pilot them extensively, and respect participants' time by offering flexible scheduling.