Beyond the Average

**Option 1 (Focus on Individuals):** Shifting from functionality ("Does it work?") to impact ("Who benefits?") through Heterogeneous Treatment Effects. **Option 2 (Emphasizing Differentiation):** Beyond "Does it work?" – exploring "For whom?" via Heterogeneous Treatment Effects analysis. **Option 3 (More Concise):** From "Does it work?" to "Who benefits?" – a focus on Heterogeneous Treatment Effects.

The Limit of the Average

While the Average Treatment Effect (ATE) offers a program's overall impact, it masks nuance. ATE averages belie individual differences; Heterogeneous Treatment Effects (HTE) analysis reveals these varied responses.

Average Treatment Effect (ATE)

One effect for everyone.

+5% Effect

Conditional ATE (CATE)

Different effects for different subgroups.

Group A +15%
Group B +2%
Group C -3%

Case Study: The Moving to Opportunity Experiment

The MTO program gave housing vouchers to families in poor areas. Early studies showed no impact on adults' earnings, on average. However, a key HTE re-analysis unveiled a starkly different outcome, tied to the children's ages at relocation.

Long-Term Income Gains Varied Dramatically by Age at Move

Here are a few options, all similar in length: * This finding reshaped policy, revealing the program's success, specifically for families with young children. * The study changed policy understanding, demonstrating high program efficacy, especially for families with young kids. * Policy shifted following this, as the program excelled, yet only when focused on families with young children. * This changed policy; the program proved effective, but its impact maximized in households with young children.

The HTE Methodological Toolkit

Here are a few options, all similar in length and capturing the meaning: * **HTE identification has shifted from hypothesis-driven testing to data-driven discovery.** * **Approaches to HTE now move beyond testing assumptions and into exploratory data analysis.** * **The search for HTE now emphasizes data exploration, rather than pre-defined hypothesis testing.** * **Finding HTE now relies more on data exploration than on testing predetermined ideas.**

1. Confirmatory Analysis

Begin with a theory. Then, rigorously test pre-defined hypotheses via regression with interaction terms; this represents the ideal for theory validation.

Y = β₀ + β₁T + β₂S + β₃(T x S) + ε

2. Exploratory Discovery

Employ machine learning to uncover key data subgroups. Techniques such as Causal Forests identify areas with varying treatment effects.

ΨΤ(x)

Perils and Best Practices

Rigorous analysis is crucial; power demands responsible handling to avoid flawed conclusions.

⚠️The Peril of "P-Hacking"

Analyzing numerous subgroups boosts the odds of a spurious, "significant" finding, thereby damaging research integrity.

A 5% test across 10 groups yields a ~40% false positive risk.

The Solution: Pre-Analysis Plans (PAPs)

Pre-registering your analysis plan is key: A PAP clarifies planned tests, creating an audit trail distinct from exploratory findings.

  • Specify hypotheses before analysis.
  • Define primary outcomes and models.
  • Plan for multiple comparisons.
  • Conduct power calculations for interactions.