Citation

Discussion Paper Details

Please find the details for DP10455 in an easy to copy and paste format below:

Full Details   |   Bibliographic Reference

Full Details

Title: Natural Experiment Policy Evaluation: A Critique

Author(s): Christopher Hennessy and Ilya Strebulaev

Publication Date: March 2015

Keyword(s): causal effects, dynamic environments, natural experiments, policy evaluations and random treatment

Programme Area(s): Financial Economics

Abstract: We argue exogenous random treatment is insufficient for valid inference regarding the sign and magnitude of causal effects in dynamic environments. In such settings, treatment responses must be understood as contingent upon the typically unmodeled policy generating process. With binary assignment, this results in quantitatively significant attenuation bias. With more than two policy states, treatment responses can be biased downward, upward, or have the wrong sign. Further, it is not only generally invalid to extrapolate elasticities across policy processes, as argued by Lucas (1976), but also to extrapolate within the same policy process. We derive auxiliary assumptions beyond exogeneity for valid inference in dynamic settings. If all possible policy transitions are rare events, treatment responses approximate causal effects. However, reliance on rare events is overly-restrictive as the necessary and sufficient conditions for equality of treatment responses and causal effects is that policy variable changes have mean zero. If these conditions are not met, we show how treatment responses can nevertheless be corrected and mapped back to causal effects or extrapolated to forecast responses to future policy changes.

For full details and related downloads, please visit: https://cepr.org/active/publications/discussion_papers/dp.php?dpno=10455

Bibliographic Reference

Hennessy, C and Strebulaev, I. 2015. 'Natural Experiment Policy Evaluation: A Critique'. London, Centre for Economic Policy Research. https://cepr.org/active/publications/discussion_papers/dp.php?dpno=10455