20 Advanced Causal Inference Questions for Experts and Researchers

Of course. Here are some original questions and answers about causal inference, progressing from foundational to advanced concepts.

Foundational Q&A

Question: Why is the phrase "correlation does not imply causation" so fundamental to the entire field of causal inference? Please provide a real-world example.

Answer: This phrase is the bedrock of causal inference because it addresses the most common error in interpreting data: mistaking a simple association for a direct cause-and-effect link. Two variables can be correlated (i.e., they move together) for reasons other than one causing the other. The most common reason is a confounding variable—a third, unobserved factor that influences both variables, creating a spurious relationship.

• Real-World Example: There is a well-known positive correlation between a city's ice cream sales and its crime rate. A naive interpretation would be that eating ice cream causes criminal behavior, which is absurd. The confounding variable here is weather/temperature. When it's hot, more people are outside, leading to more opportunities for social interaction and crime. Hot weather also directly causes more people to buy ice cream. The weather is the common cause that makes the two variables appear related. Causal inference provides the frameworks to identify and control for such confounders to determine the true causal effect, if one even exists.

Question: What is the "fundamental problem of causal inference," and how does the concept of a "counterfactual" define this problem?

Answer: The "fundamental problem of causal inference" is that for any single unit of study (like a person, a school, or a company), it is impossible to observe its outcome under both the treatment and control conditions at the same point in time. We can only ever observe one reality.

The counterfactual is the outcome that we don't see. For example: * Factual: We observe the exam score of a student who attended a tutoring session. * Counterfactual: We can never know the exam score of that exact same student, at that exact same time, had they not attended the tutoring session.

Because we can never observe both the factual and the counterfactual outcome for the same unit, we can never calculate an individual's true causal effect. The entire field of causal inference is therefore dedicated to solving this missing data problem, primarily by using groups of similar individuals to credibly estimate the average causal effect across a population.

Intermediate Q&A

Question: Explain the difference between "adjustment" for a confounder and "adjustment" for a collider. Why is one necessary and the other a critical mistake?

Answer: This question gets at the heart of using Directed Acyclic Graphs (DAGs) for causal reasoning.

• Adjusting for a Confounder: A confounder is a common cause of the treatment and the outcome. In a DAG, this looks like Confounder → Treatment and Confounder → Outcome. This structure creates a "backdoor path" between the treatment and outcome that is non-causal. To estimate the true causal effect, you must control for, or "adjust for," the confounder to block this backdoor path and isolate the direct relationship. Failing to adjust for a confounder leads to omitted-variable bias.

• Adjusting for a Collider: A collider is a variable that is a common effect of two other variables. In a DAG, this looks like Treatment → Collider ← Outcome. Before adjustment, the path between the treatment and outcome through the collider is naturally blocked. However, if you control for or "adjust for" the collider (e.g., by including it in a regression or stratifying your data by it), you artificially open this path, creating a spurious association between the treatment and the outcome. This introduces a nasty form of bias known as "collider bias" or "endogenous selection bias."

In summary: Adjusting for a confounder is necessary to close a backdoor path and remove bias. Adjusting for a collider is a critical error because it opens a path that should be closed, thereby introducing bias.

Question: What is a Directed Acyclic Graph (DAG), and how does it help a researcher move from a simple regression model to a more robust causal claim?

Answer: A Directed Acyclic Graph (DAG) is a visual representation of the assumed causal relationships between a set of variables. * Directed: Arrows (edges) between variables (nodes) indicate a presumed causal direction (e.g., A → B means A causes B). * Acyclic: There are no feedback loops; you cannot start at a node and follow the arrows back to itself.

A DAG transforms causal analysis in several ways: 1. Makes Assumptions Explicit: Instead of implicitly choosing variables for a regression model, a DAG forces the researcher to explicitly map out all their assumptions about how the world works before looking at the data. This transparency is crucial for critique and replication. 2. Identifies Confounding: DAGs provide a formal, graphical way to identify all "backdoor paths" (spurious associations) between a treatment and an outcome. 3. Provides a Valid Adjustment Set: Based on the backdoor criterion, a DAG tells you the minimal sufficient set of variables you need to control for (adjust for) to get an unbiased estimate of the causal effect. It also tells you which variables you absolutely should not control for (like colliders or mediators on the causal pathway). 4. Moves Beyond "Kitchen Sink" Regression: It prevents the common practice of throwing every available variable into a regression model. DAGs show that controlling for the wrong variables can be just as bad as, or worse than, failing to control for the right ones.

By using a DAG, a researcher can build a regression model that is not just predictive but is principled and defensible as an estimator of a causal effect.

Advanced Q&A

Question: You are trying to estimate the causal effect of a new fertilizer on crop yield. You cannot run a randomized trial. Explain what an instrumental variable is and propose a plausible instrument for this scenario. What are the three core assumptions your proposed instrument must satisfy to be valid?

Answer: An instrumental variable (IV) is a third variable that can be used to estimate a causal effect in the presence of unmeasured confounding. It works by finding a source of variation in the treatment (fertilizer use) that is "as-if random" and is not itself confounded with the outcome (crop yield).

• Plausible Instrument: A plausible instrument could be the geological variation in the soil's natural phosphorus content across different plots of land.

For this to be a valid instrument, it must satisfy three core, untestable assumptions: 1. The Relevance Assumption: The instrument must have a causal effect on the treatment. In this case, the natural phosphorus level of the soil must be a strong predictor of how much of the new (likely phosphorus-based) fertilizer a farmer chooses to apply. Farmers on low-phosphorus soil would be much more likely to use the fertilizer. This is a testable assumption. 2. The Exclusion Restriction: The instrument can only affect the outcome through its effect on the treatment. Here, the natural phosphorus content of the soil can only affect crop yield by influencing the amount of fertilizer applied. It cannot have any other independent effect on yield (e.g., by also being correlated with better water retention). This is a strong, untestable assumption that relies heavily on domain expertise. 3. The Independence Assumption (or Ignorability): The instrument must be independent of any unmeasured confounders that link the treatment and the outcome. The natural phosphorus level must not be correlated with other factors like farmer skill, wealth, or access to other advanced technologies that also affect crop yield. The geological randomness of the soil provides a strong argument for this assumption.

If these assumptions hold, IV analysis can isolate the causal effect of the fertilizer on crop yield for the subpopulation of farmers whose fertilizer use was influenced by the natural soil content.

Question: What is treatment effect heterogeneity, and why is it often more important for policy decisions than the Average Treatment Effect (ATE)? Describe a modern method, such as Causal Forests, that is specifically designed to uncover this heterogeneity.

Answer: Treatment effect heterogeneity is the phenomenon where the causal effect of an intervention varies across different individuals or subgroups within a population. While the Average Treatment Effect (ATE) provides a single summary number for the entire population, it can be highly misleading. For policy, knowing for whom a program works, doesn't work, or is harmful is often far more critical than knowing the average effect. A policy with a positive ATE might be celebrated, while masking the fact that it is actively harming a vulnerable subgroup.

Causal Forests are a machine learning method, adapted from the random forest algorithm, that is specifically designed to estimate Conditional Average Treatment Effects (CATEs)—that is, the causal effect for an individual, conditional on their specific set of characteristics (covariates).

Here’s how it works at a high level: 1. Grows "Honest" Trees: Like a random forest, it builds many individual decision trees. However, it uses a principle of "honesty" by splitting the data: one subsample is used to determine the splits in the tree, and an entirely separate subsample is used to estimate the treatment effect within the leaves of that tree. This prevents the model from overfitting and finding spurious heterogeneity. 2. Optimizes for Heterogeneity: The splitting criterion for each tree is not to improve outcome prediction, but to find splits that maximize the difference in the treatment effect between the resulting child nodes. It actively searches for the covariates that define the subgroups with the most different treatment effects. 3. Averages for Robustness: The final CATE estimate for any given individual is the average of the estimates from all the trees in the forest.

The output of a Causal Forest is not a single number but a function that can predict the treatment effect for any individual, allowing policymakers to move beyond one-size-fits-all solutions and design more targeted, effective, and equitable interventions.

Questions

Here are 20 advanced and difficult questions to ask in the field of causal inference, suitable for researchers, data scientists, or graduate-level discussions. These cover theoretical depth, practical modeling challenges, and limitations of causal methods.

🔬 Theoretical & Conceptual Questions

1. How do you distinguish between confounding and colliders in complex DAGs, and what are the implications of conditioning on each?

2. Explain the difference between exchangeability, ignorability, and unconfoundedness. Are these interchangeable in practice?

3. In what scenarios can causal effects be identified without the positivity assumption? How do truncations of support affect estimation?

4. How would you assess the identifiability of a causal effect from observational data using do-calculus?

5. How do you formally define and interpret a counterfactual in the potential outcomes framework vs. the structural causal model framework?

📐 Modeling & Assumptions

1. Describe how you would validate the assumptions of a Marginal Structural Model (MSM) when using inverse probability weighting.

2. What are the limitations of using propensity score matching in high-dimensional settings? What alternatives would you use?

3. How do you handle violations of the Stable Unit Treatment Value Assumption (SUTVA) in real-world applications like social networks or marketplaces?

4. How do instrumental variables work when the instrument is weak or violates the exclusion restriction? How can one test these assumptions?

5. What are the challenges of applying causal inference to time-varying treatments and time-varying confounders? How does g-computation help?

📊 Estimation & Computation

1. Compare Targeted Maximum Likelihood Estimation (TMLE) and Double Machine Learning (DML). When would you prefer one over the other?

2. How do you interpret and mitigate bias-variance tradeoff when using machine learning models for causal effect estimation?

3. How do you estimate treatment effects when the treatment variable is continuous (dose-response)? What modeling approaches are suitable?

4. Discuss how to handle missing data in causal inference. How is this different from missing data in prediction problems?

5. Explain the concept of partial identification and bounds (e.g., Manski bounds). When is this approach useful?

🧠 Causal Discovery & Generalization

1. How can causal discovery algorithms (e.g., PC, GES, NOTEARS) distinguish causation from correlation without experimental data?

2. How would you evaluate the external validity of a causal estimate obtained from a specific sample or setting?

3. What are the implications of causal transportability and how can selection diagrams be used to assess it?

4. In multi-treatment scenarios, how do you define and estimate individual treatment effects (ITE) or conditional average treatment effects (CATE)?

5. How do you incorporate prior knowledge or expert input into a causal inference model when data alone is insufficient to identify causal effects?