The COVID-19 pandemic has sparked an explosion of research evaluating various policies using observational data (e.g. Courtemanche et al. 2020, Deb et al. 2020, Hsiang et al. 2020, Pei et al. 2020, and Abouk and Heydari 2020), but there is no consensus on what would have happened to cases and deaths if these policy measures had not been implemented.
The evaluation of counterfactual policy effects on the spread of COVID-19 is complicated. People rationally and voluntarily respond to information on transmission risks even without any policies. This makes it difficult to distinguish the effect of polices from that of voluntary behavioural responses.
To properly evaluate the role of policies relative to people’s voluntary responses, we develop an empirical framework based on the following causal path diagram (Chernozhukov et al. 2020). There are five key components: (1) confounders, such as socio-demographic characteristics, (2) information about current infection levels and growth rates, (3) policies reacting to the information, and (4) local behaviour of people reacting to the information and effective polices, all of which together determine (5) final outcomes, such as case and death growth rates. The system is dynamic: the health outcomes today become information for the next period.
Figure 1 Causal path diagram
Notes: W denotes confounders, I information, P policy, B behaviour, all realised in that sequence by the time t in state i; Y denotes outcomes realised at future date t+l.
This causal framework explicitly recognises that policies not only directly affect the spread of COVID-19 (via masks) but also indirectly – by changing people’s behaviour (through restaurant closures, school closures, stay-at-home orders). It also recognises that people react to new information, such as new cases and deaths in their state, and voluntarily adjust their behaviour (social distancing, hand washing, masks) even without any policy in place.
The framework, coupled with parametric structural equations, can quantitatively decompose the growth of COVID-19 cases and deaths into three components: direct policy effect, policy effect through behaviour, and direct behaviour effect in response to new information.1
Many of the existing papers analyse different parts of the causal diagram. In our paper we take a more holistic approach and analyse the entire causal dynamic model, analysing both the direct and indirect effects of policies on infections and on behaviour as captured by Google Mobility Reports. Using US state-level data, we examine how a mask mandate and other lockdown policies have affected the growth rates of cases and deaths by conducting counterfactual experiments within the model.
Mask mandates for employees directly reduced transmissions
Figure 2 illustrates the raw-data fact: the US states with mask mandates for employees of publicly facing businesses tend to have lower case and death growth rates than states without mask mandates since the end of April 2020.
Our causal analysis, which controls for many confounding factors and dynamics, confirms this raw-data finding. The effect is mostly direct (without affecting people’s behaviour), suggesting that wearing masks lowers transmission risk per contact.
Figure 2 Evidence from raw US data: The average growth rates in cases and deaths for states without mask mandates (red) and with mask mandates (blue)
Figure 3 shows the result of our counterfactual analysis based on the estimated causal dynamic model. Nationally implementing mandatory face masks for employees on 1 April could have reduced the weekly growth rate in cases by as much as 10%, which translates into a nearly 40% relative reduction in cumulative deaths (90% confidence interval of [17,55]%). This implies that as many as 17 to 55 thousand lives could have been saved through the beginning of June; as of 27 May 2020, the US Centers for Disease Control and Prevention reports 99,031 deaths in the US.
Figure 3 Relative decrease in deaths induced by nationally mandating masks for employees of publicly facing businesses on 1 April in the US
Our finding is corroborated by a different causal methodology based on synthetic control using German data in Mitze et al. (2020),2 which reported an absolute 20% reduction in the growth rates. Our finding is further consistent with the laboratory results in Hou et al. (2020), who show that the nasal cavity may be the principal initial site of infection, thus supporting “the widespread use of masks to prevent aerosol, large droplet, and/or mechanical exposure to the nasal passages”.3
Could we have kept non-essential business open?
In Figure 4, another counterfactual experiment indicates that keeping non-essential businesses open (other than movie theatres and gyms, and keeping restaurants ‘takeout-only’ mode) could have increased cumulative cases and deaths by 15% (with a 90% confidence interval of [-20, 60]%).
Figure 4 Effect of leaving non-essential businesses open on COVID-19 cases in the US
These findings are in line with other analyses that found that closing non-essential businesses hardly affected social-distancing behaviour (Maloney and Taskin 2020). This suggests that universal masks policies could have compensated for keeping substantial parts of the economy open.
Stay-at-home (or shelter-in-place) orders
Figure 5 shows that, without stay-at-home orders, there could have been an 80% increase in total cases by the start of June (90% confidence interval of [25,170]%). This implies that 0.5–3.4 million more Americans could have been infected without stay-at-home orders, providing suggestive evidence that reopening via removal of stay-at-home orders could lead to a substantial increase in cases and deaths.4
Figure 5 Effect of not implementing stay-at-home order on COVID-19 cases in the US
Policy or private behavioural response?
Hsiang et al. (2020) and Chernozhukov et al. (2020) find that the containment policies have substantially reduced the COVID-19 growth rates in the US, with policies roughly explaining one-third to two-thirds of the observed decline in growth rates of confirmed cases and deaths.
We further find that both policies and information on past cases and deaths determine people’s social-distancing behaviour, where policy effects explain about 50% of the observed decline in Google mobility variables and behaviour effects account for the other half.5 The estimates suggest that there are both large policy effects and large behavioural effects on the growth of cases and deaths. Except for mandatory masks, the containment policies affect cases and deaths indirectly through their impact on behaviour.
There are numerous strong claims in social media that containment policies did not matter and that the observed decline in COVID-19 growth rates and distancing behaviour is almost entirely due to private responses to information. We believe these claims are generated by misinterpretations of statistical analyses.
One reason for this, for example, is the difficulty of identifying school-closure effects. As shown in Figure 6 and 7 , the timing of school closures (Figure 7) predates or nearly coincides with a substantial drop in mobility as reported by Google reports (Figure 6).
Figure 6 Intensity of visits to ‘workplaces’
Notes: The figure shows the dynamics of the intensity of visits to ‘workplaces’ from Google Mobility Reports, which shows the sudden drop in the last two weeks of March.
Figure 7 School closures in the US
Notes: The figure shows the portion of states with closed schools, which slightly leads the mobility drop in Figure 6.
The increase in school closures also coincides with the large increase in total COVID-19 cases, an information variable. Lack of cross-sectional variation in school closures makes it difficult to distinguish the information effect on behaviour from the policy effect on behaviour. It is clear that school closures potentially affect behaviour by imposing very hard constraints on parents’ mobility, but the exact quantification of this policy effect requires richer data.6
The economic evaluation of various policies is underway in many new papers (e.g. Chetty et al. 2020, Coibion et al. 2020). Understanding the effect of containment policies on fatalities and cases is crucial to understand the trade-off between health and economic wellbeing since fatalities entail direct economic losses but some COVID-19 survivors may suffer from long-term health complications (see e.g. Alvarez et al. 2020, Baqaee et al. 2020, Fernández-Villaverde and Jones 2020, Acemoglu et al. 2020, Keppo et al. 2020, McAdams 2020).
While the economic effect of some containment policies is hard to evaluate immediately, the effect of mask policies is clear and could be gauged from a simple calculation: 40,000 saved lives, in April and May, times the statistical value of life, e.g. $5 million, translates to a $200 billion economic impact – an order of magnitude bigger than the cost of implementing the universal mask policy.
Abaluck, J, J A Chevalier, N A Christakis, H P Forman, E H Kaplan, A Ko and S H Vermund (2020), “The case for universal cloth mask adoption and policies to increase supply of medical masks for health workers”, Covid Economics 5.
Abouk, R, and B Heydari (2020), “The immediate effect of COVID-19 policies on social distancing behavior in the US”, medRxiv, preprint.
Acemoglu, D, V Chernozhukov, I Werning and M D Whinston (2020), “Optimally targeted lockdowns in a multi-group SIR model”, NBER 27102.
Alvarez, F E, D Argente and F Lippi (2020), “A simple planning problem for COVID-19 lockdown”, Covid Economics 14.
Baqaee, D, E Farhi, M J Mina, and J H Stock (2020), “Reopening scenarios”, NBER 27244.
Baron, R M, and D A Kenny (1986), “The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations”, Journal of Personality and Social Psychology 51: 1173–82.
Chernozhukov, V, H Kasahara and P Schrimpf (2020), “Causal impact of masks, policies, behavior on early COVID-19 pandemic in the US”, Covid Economics 35.
Chetty, R, J N Friedman, N Hendren and M Stepner (2020), “Real-time economics: A new platform to track the impacts of COVID-19 on people, businesses, and communities using private sector data”, mimeo.
Coibion, O, Y Gorodnichenko and M Weber (2020), “Labor markets during the COVID-19 crisis: A preliminary view”, Covid Economics 21.
Courtemanche, C, J Garuccio, A Le, J Pinkston and A Yelowitz (2020), “Strong social distancing measures in the US reduced the COVID-19 growth rate”, Health Affairs 39(7).
Deb, P, D Furceri, J D Ostry and n Tawk (2020), “The effects of containment measures on the COVID-19 pandemic”, VoxEU.org, 5 June.
Fernández-Villaverde, J, and C I Jones (2020), “Estimating and simulating a SIRD model of COVID-19 for many countries, states, and cities”, NBER 27128.
Gitmez, A, K Sonin and A L Wright (2020), “Political economy of crisis response”, University of Chicago, Becker Friedman Institute for Economics Working Paper No. 2020-68.
Greenland, S, J Pearl and J M Robins (1999), “Causal diagrams for epidemiologic research”, Epidemiology January: 37–48.
Haavelmo, T (1944), “The probability approach in econometrics”, Econometrica: Journal of the Econometric Society 12: iii–115.
Hernán, M A, and J M Robins (2020), Causal inference: What if, Chapman & Hall/CRC.
Hines, O, S Vansteelandt and K Diaz-Ordaz (2020), “Robust inference for mediated effects in partially linear models”, arXiv:2007.00725.
Hou, Y J, K Okuda, C E Edwards et al. (2020), “SARS-CoV-2 reverse genetics reveals a variable infection gradient in the respiratory tract”, Cell 182: 1–18.
Howard, J, A Huang, Z Li et al. (2020), “Face masks against COVID-19: An evidence review”, Preprints 2020.
Hsiang, S, D Allen, S Annan-Phan et al. (2020), “The effect of large-scale anti-contagion policies on the coronavirus (COVID-19) pandemic”, Nature 2020.
Jones, T C, B Mühlemann, T Veith et al. (2020), “An analysis of SARS-CoV-2 viral load by patient age”, medRxiv 2020.06.08.20125484, preprint.
Keppo, J, E Quercioli, M Kudlyak, L Smith, and A Wilson (2020), “The behavioral SIR model, with applications to the Swine Flu and COVID-19 pandemics”, in Virtual Macro Seminar.
L’Huillier, A G, G Torriani, F Pigny, L Kaiser and I Eckerle (2002), “Shedding of infectious SARS-CoV-2 in symptomatic neonates, children and adolescents”, medRxiv 2020.04.27.20076778, preprint.
Maloney, W F, and T Taskin (2020), “Determinants of social distancing and economic activity during COVID-19: A global view”, Covid Economics 13.
McAdams, D (2020), “Nash SIR: An economic-epidemiological model of strategic behavior during a viral epidemic”, Covid Economics 16.
Mitze, T, R Kosfeld, J Rode and K Walde (2020), “Face masks considerably reduce COVID-19 cases in Germany”, Covid Economics 27.
Peters, J, D Janzing, and S Bernhard (2017), Elements of causal inference: Foundations and learning algorithms, Cambridge MA: MIT Press.
Wright, A L, K Sonin, J Driscoll and J Wilson (2020), “Poverty and economic dislocation reduce compliance with COVID-19 shelter-in-place protocols”, SSRN.
Wright, P G (1928), Tariff on animal and vegetable oils, New York: Macmillan Company.
Wright, S (1923), “The theory of path coefficients a reply to Niles’s criticism”, Genetics 8(3): 239.
1 The causal model is framed using the language of structural equations models and causal diagrams of econometrics (P Wright 1928, Haavelmo 1944) and genetics (S Wright 1922). See Greenland et al. (1999), Peters et al. (2017), and Hernán and Robins (2020) for modern developments, especially in computer science and epidemiology. The particular causal diagram has several ‘mediation’ components, where variables affect outcomes directly and indirectly through other variables called mediators; these structures go back at least to S Wright (1922); see e.g. Baron and Kenny (1986) and Hines et al. (2020) for modern treatments. The father and son, P Wright and S Wright, closely collaborated to develop structural equation models and causal path diagrams; P Wright’s key work represented supply-demand system as a directed acyclical graph and established its identification using exclusion restrictions on instrumental variables.
2 Our study was first released in ArXiv on 28 May 2020; Mitze (2020) was released at SSRN on 8 June 2020.
3 Our findings also align well with the medical observational evidence reviewed in Howard et al. (2020), who state that “mask wearing reduces the transmissibility per contact by reducing transmission of infected droplets in both laboratory and clinical contexts.” Abaluck et al. (2020) find the association between growth rates of cases and of deaths with mask-wearing norms: countries with pre-existing norms where sick people wear masks are lower by 8-10% than those rates in countries with no pre-existing mask norms.
4 This finding is in line with those implied by Courtemanche et al. (2020), who attribute a 6% absolute reduction in growth rates to these orders, using US county-level data.
5 This is in line with theoretical study by Gitmez et al. (2020) that investigates the role of private behaviour and negative external effects for individual decisions over policy compliance as well as information acquisition during pandemics.
6 There is another reason why school closures could have played a role in transmission reduction: emerging evidence suggests that children infected with COVID-19 transmit the virus (L’Huillier et al. 2020, Jones et al. 2020), though to a lesser extent than adults.