The Impact Of The Covid 19 Pandemic And Policy Responses On Excess Mortality

Summary

NBER Working Paper No. 28930, The Impact of the COVID-19 Pandemic and Policy Responses on Excess Mortality, by Virat Agrawal, Jonathan H. Cantor, Neeraj Sood and Christopher M. Whaley, dated June 2021 and revised September 2022. Using an event study approach and data from 43 countries and all U.S. states, the paper measures changes in excess deaths following shelter-in-place (SIP) policies. The abstract states that the authors do not find that places implementing SIP policies earlier had lower excess deaths. The data section describes SIP measures from the Oxford COVID-19 Government Response Tracker and weekly excess deaths computed against the 2015 to 2019 average. The paper closes with appendix figures of event study estimates, including changes in the Oxford Stringency Index, and individual country and U.S. state comparisons.

Summary drafted by a model from the document's text below and checked by script against that text before publication. It is a navigation aid, not a reading of what the document proves. Where AI is used

Cite this document

“The Impact Of The Covid 19 Pandemic And Policy Responses On Excess Mortality.” Original at www.nber.org. Archived at Pandemic Darlings, https://pandemicdarlings.com/source-documents/the-impact-of-the-covid-19-pandemic-and-policy-responses-on-excess-mor-ff23f148c2/, SHA-256 ac24a9b0fac1eccd4dd6e59a57ebbcde41dfc0fd516842295582da9298145153.

Full text

NBER WORKING PAPER SERIES

THE IMPACT OF THE COVID-19 PANDEMIC AND POLICY RESPONSES
ON EXCESS MORTALITY

Virat Agrawal
Jonathan H. Cantor
Neeraj Sood
Christopher M. Whaley

Working Paper 28930
http://www.nber.org/papers/w28930

NATIONAL BUREAU OF ECONOMIC RESEARCH
1050 Massachusetts Avenue
Cambridge, MA 02138
June 2021, Revised September 2022

Funding provided by NIA K01AG061274 and the Conrad N. Hilton Foundation. We thank
Matthew Baird, Jessie Coe, Dhaval Dave, Christine Eibner, Andrew Gelman, Jonathan Ketcham,
Paul Koegel, David Powell, Elizabeth Stuart, and seminar participants at ASHE 2022 for helpful
comments. The views expressed herein are those of the authors and do not necessarily reflect the
views of the National Bureau of Economic Research.

NBER working papers are circulated for discussion and comment purposes. They have not been
peer-reviewed or been subject to the review by the NBER Board of Directors that accompanies
official NBER publications.

© 2021 by Virat Agrawal, Jonathan H. Cantor, Neeraj Sood, and Christopher M. Whaley. All
rights reserved. Short sections of text, not to exceed two paragraphs, may be quoted without
explicit permission provided that full credit, including © notice, is given to the source.
The Impact of the COVID-19 Pandemic and Policy Responses on Excess Mortality
Virat Agrawal, Jonathan H. Cantor, Neeraj Sood, and Christopher M. Whaley
NBER Working Paper No. 28930
June 2021, Revised September 2022
JEL No. I1,I12,I18,I28

ABSTRACT

As a way of slowing COVID-19 transmission, many countries and U.S. states implemented
shelter-in-place (SIP) policies. However, the effects of SIP policies on public health are a-priori
ambiguous. Using an event study approach and data from 43 countries and all U.S. states, we
measure changes in excess deaths following the implementation of COVID-19 shelter-in-place
(SIP) policies. We do not find that countries or U.S. states that implemented SIP policies earlier
had lower excess deaths. We do not observe differences in excess deaths before and after the
implementation of SIP policies, even when accounting for pre-SIP COVID-19 death rates.

Virat Agrawal Neeraj Sood
USC Sol Price School of Public Policy Schaeffer Center for Health Policy and Economics
Los Angeles, CA Sol Price School of Public Policy
United States University of Southern California
viratagr@usc.edu University Park Campus
Los Angeles CA 90089
Jonathan H. Cantor and NBER
RAND Corporation nsood@healthpolicy.usc.edu
1776 Main Street
Santa Monica, CA 90401 Christopher M. Whaley
jcantor@rand.org RAND Corporation
1776 Main Street
Santa Monica, CA 90401
cwhaley@rand.org
1. INTRODUCTION

The COVID-19 pandemic has caused an unprecedented policy response from public health

officials and governments. As of September 16, 2021, approximately 4.7 million deaths

worldwide and 669,000 deaths in the United States were attributed to COVID-19 (Johns Hopkins

Coronavirus Resource Center 2022). The number of worldwide excess deaths in 2020 and 2021

totalled 18.2 million (COVID-19 Excess Mortality Collaborators 2022). To slow COVID-19

transmission, many countries and United States localities implemented non-pharmaceutical

interventions (NPIs), one of the most common of which is “shelter-in-place” (SIP) policies.

These policies are designed to reduce COVID-19 transmission by restricting non-essential

mobility.

Conceptually, the impacts of SIP policies on public health are ambiguous and complex. The

effects of SIP policies depend on both behavioral responses and the efficacy of behavioral

responses in averting COVID-19 transmission. Early evidence suggests that SIP policies slowed

COVID-19 transmission early in the pandemic in the United States (Courtemanche et al. 2020;

B. G. Kaufman et al. 2021) and globally (Ge et al. 2021). More recent studies suggest that

individuals changed behavior in response to the severity of the pandemic and the enactment of

SIP policy implementation lagged this individual behavior change. While social distancing is an

important mechanism to avoid COVID-19 spread, studies that use mobility tracking data find

modest additional social distancing responses following SIP policies (Cantor et al. 2020; Berry et

al. 2021; Askitas, Tatsiramos, and Verheyden 2021; Xu 2021; Nguyen et al. 2020). Individuals

concerned about COVID-19 risk may change behavior even in the absence of regulations or

shelter-in-place advisories. Thus, it is unclear how much change in COVID-19 risk mitigation is

2
due to formal SIP policies, compared to risk mitigation that would have occurred in the absence

of these policies.

Behavioral responses to SIP policies could have unintended consequences on COVID-19

transmission. SIP policies can transfer gatherings from regulated environments, such as

workplaces, to informal household and family environments. The risks of COVID-19

transmission may also be lower in workplaces than in households, as many employers adopted

risk mitigation protocols (Mulligan 2021). Individuals may not have the same levels of risk

mitigation when gathering informally with family and friends as in formal settings (Whaley et al.

2021). SIP policies also led to school and childcare closures, which led to greater

intergenerational childcaring and mixing, with grandparents and older adults supervising

children (Gilligan et al. 2020; Stokes and Patterson 2020). Among people hospitalized for

COVID-19, the mortality risk of COVID-19 for persons above the age of 80 is more than 10

times higher than persons between the ages of 18 to 39 (Shruti Gupta et al. 2020). In a study of

21 countries, COVID-19 mortality rates were 62 times higher for persons above the age of 65

compared to those below 54 (Yanez et al. 2020). Intergenerational mixing in response to SIP

policies could inadvertently increase exposure among high-risk elderly adults (Aparicio Fenoll

and Grossbard 2020).

At the same time, even if SIP policies reduce COVID-19 transmission, their impact on

overall health is unclear, as SIP policies likely have several important unintended consequences

(Correia, Luck, and Verner 2020; Mulligan 2021; Baral et al. 2020). SIP policies might lead to

unemployment or reduced economic activity (S. Chen et al. 2020), or stress and anxiety due to

social isolation (Xie et al. 2020; Patrick et al. 2020; Babin, Foray, and Hussey 2021), potentially

leading to increased substance use and suicides (Mason et al. 2021; Holland et al. 2021).

3
Reduced physical mobility may have also caused increases in child abuse and domestic violence

(Leslie and Wilson 2020; Cappa and Jijon 2021; Hsu and Henke 2021), which can adversely

affect health in both the short and long run (Sonu, Post, and Feinglass 2019). SIP policies also

reduced use of high value non-COVID care (Cantor et al. 2020; Whaley et al. 2020; McBain et

al. 2021; H. W. Kaufman et al. 2020; Velazquez et al. 2021; Ziedan, Simon, and Wing 2020).

Cancer screenings that occurred after lockdowns were associated with a higher tumor burden and

subsequentially worse outcomes (Thierry et al. 2021). These complex effects suggest that SIP

policies might increase or decrease COVID-19 related mortality, and at the same time might also

increase mortality from other causes. Thus, the impact of SIP policies on all cause mortality is a

priori ambiguous.

In this paper, we examine the impacts of SIP policies on excess deaths, which compare

deaths from all causes in a given geography at a given point in time to expected deaths from all

causes based on historical data (Krelle, Barclay, and Tallack 2020). Using excess deaths as a key

outcome captures both potential changes in COVID-19 deaths and potential changes in non-

COVID-19 deaths. Using excess deaths, rather than confirmed COVID-19 deaths, also alleviates

concerns about the accurate classification of COVID-19 deaths (Sanmarchi et al. 2021). While

emerging evidence has documented increases in excess deaths during the pandemic, the existing

literature has not systematically examined the role of SIP policies in averting or contributing to

excess deaths. Further, no studies have compared the impact of SIP policies between countries

and compared these impacts within U.S. states.

To do so, we use data from 43 countries that implemented SIP policies and all 50 U.S. states.

We estimate the effect of SIP policies using an event study approach. In both settings, we fail to

find that SIP policies reduced excess deaths. For the international comparison, we find that

4
excess deaths increased following the implementation of SIP policies. At the U.S. state-level,

excess mortality increases in the immediate weeks following SIP implementation and then trends

below zero following 20 weeks of SIP implementation, although the confidence intervals overlap

zero. It is important to note that we do not identify the cause for the increase in excess mortality.

Instead, we simply report that that an increase occurred.

A key challenge with evaluating the impacts of these policies is that the timing of SIP

policies is endogenous to the COVID-19 pandemic. If SIP policies were implemented when

excess deaths were rising, then the results from the event study would be biased towards finding

that SIP policies lead to excess deaths. However, we find the opposite: countries that

implemented SIP policies experienced a decline in excess mortality prior to implementation

compared to countries that did not implement SIP policies. We find no pre-existing trends in

excess mortality prior to implementation of SIP policies in U.S. states. We also do not find

differences based on exposure to the COVID-19 pandemic prior to implementation of SIP

policies or when controlling for variations in the trajectory of the pandemic.

It is also possible that the average effects in our event studies might mask heterogeneity in

the impact of policies across countries and U.S. states. For example, SIP policies might be more

effective when implemented early in the pandemic or might work better when community

transmission is high. We separately estimate event studies for each country or U.S. state and for

groups of countries and states with high COVID-19 transmission in the two weeks prior to the

implementation of the SIP policy. We find little evidence of heterogenous effects, except that

SIP policies seem to be more effective in island nations or Hawaii. Islands have the geographic

benefit of limiting entry and spillovers from other areas with high infection rates, which has

contributed to COVID-19 transmission, even in locations with SIP policies (Dave, McNichols,

5
and Sabia 2021). These potential heterogeneous impacts and the staggered timing of SIP policies

raises a more formal concern about the validity of two-way fixed effects models (Goodman-

Bacon 2021; Weill et al. 2021). As a solution, we also apply the dynamic treatment effect

approach outlined by Sun and Abraham (2020) and find similar results. We also find similar

results when using alternative functional forms, controlling for the trajectory of the pandemic,

when separating COVID-19 and non-COVID-19 deaths, and when controlling for anticipiatory

implementation of SIP policies.

These findings do not mean that COVID-19 risk mitigation approaches are ineffective.

Substantial clinical and epidemiological evidence convincingly finds benefits from risk

mitigation strategies, such as mask wearing, testing, and social distancing (Abaluck et al. 2021;

Banerjee et al. 2020; Chu et al. 2020; Cheng et al. 2021; Honein 2020). Rather, our findings

imply that the incremental level of risk mitigation induced by SIP policies, combined with

potential adverse effects of SIP policies, was not effective in averting excess deaths. Political

dysfunction and cues likely contributes to limited responses to SIP policies (Bisbee and Lee

2020). It is also possible, that when countries implemented SIP policies they relaxed other risk

mitigation strategies, such as testing high-risk populations. Prior research suggests that countries

with higher testing capacity were correlated with a reduced number of total deaths (Sanmarchi et

al. 2021).

This study adds to a growing literature on the effect of NPIs on the number of COVID-19

cases, deaths, and social distancing behaviors. Existing studies have used cellular phone mobility

tracking and have found that the implementation of SIP policies leads to small changes in

mobility (Berry et al. 2021; Singh et al. 2021; Askitas, Tatsiramos, and Verheyden 2021; Xu

2021). Important differences exist based on income, with larger increases in physical distancing

6
for individuals in high-income neighborhoods than individuals in low-income neighborhoods

(Jay et al. 2020; Singh et al. 2021). Other work predicts that the COVID-19 pandemic may lead

to a significant increase in the number of “deaths of despair” due to economic stagnation and

social isolation associated with the pandemic (Petterson, Westfall, and Miller 2020).

The rest of the paper proceeds as follows. We first present the data on SIP polices and excess

deaths and methods for estimating the impact of SIP policies on excess deaths. Finally, we

discuss results, limitations and implications for public policy, and future research.

2. DATA

2.1. Data on Social Distancing Policies

To measure the impact of state-level SIP or Stay-at-Home requirement policies, we use data

from the Oxford COVID-19 Government Response Tracker (OxCGRT) for both country-level

and (Hale et al. 2021), and U.S. state-level data (Hale et al. 2020). The data contains information

on 16 indicators of government responses. We used the SIP indicator recorded for each day. This

indicator is an ordinal index scaled from zero (no SIP measures) to three (required not leaving

house with minimal exceptions).2 We average the daily SIP order value for the seven days of a

week to create a weekly SIP value for each week. We classify a SIP policy in effect if there is a

SIP measure in effect for the majority of days in a week. In sensitivity tests, we measure SIP

policies using the most strict SIP policy measure. Related work has used eleven categorical

variables from the OxCGRT dataset to estimate the impact of containment and closure

2
A value of one indicates recommended not leaving house. A value of two indicates required not leaving house with
exceptions for daily exercise, grocery shopping, and essential trips.

7
interventions on COVID-19 transmission across countries, and found none of the interventions

had a statistically significant effect (Rannan-Eliya et al. 2021).

2.2. Deaths Data

2.2.1. Country: National-level Weekly Excess Deaths

Data on weekly deaths for each country comes from the Our World In Data (OWID)

webpage (Giattino et al. 2021). OWID uses all-cause mortality data from the Human Mortality

Database and the World Mortality Dataset. We calculate excess deaths for each week of 2020 as

the difference in total deaths in a given week of 2020 and the average of total deaths in that week

from 2015 to 2019:

Excess Deaths (Week n, 2020) = Total Deaths (Week n, 2020) – Average Deaths (Week n, 2015 – 2019) (1)

While the full dataset contains 82 countries, only 43 countries have data available on total deaths

for each week of 2020. The remaining 39 countries have monthly data of total deaths that

occurred in 2020. Therefore, in our study we focused on the 43 countries that have total deaths

data for each week of 2020.

2.2.2. Country: National-level Weekly Covid-19 Deaths

We used weekly COVID-19 death data to identify the week of first COVID-19 death for our

analysis. The data on COVID-19 deaths, which are a subset of the total deaths described above,

are obtained from the OWID (Ritchie et al. 2020). The country-level dataset contains the number

of deaths caused by COVID-19 on each day for 192 countries. We used data for 43 countries that

we analyze for excess deaths and filtered out the rest. With these data, we calculated the weekly

8
deaths caused by COVID-19 in each country by summing the daily COVID-19 deaths in seven

days of each week.

2.2.3. U.S. States: State-level Weekly Excess Deaths and Weekly COVID-19 Deaths

To capture the severity of the COVID-19 pandemic on total deaths in the U.S., we use the

number of excess deaths for each state and the District of Columbia for each week of 2020 from

The Economist COVID-19 excess deaths tracker GitHub repository (The Economist 2020). We

used the same dataset to identify the week of first COVID-19 death in each state. The data for

the U.S. are generated by The Economist using information from the Centers for Disease Control

and Prevention, USAFacts, and NYC Health. The data track the total number of excess deaths in

each state and the District of Columbia for each week.

Lastly, as our unit of analysis is “excess deaths per 100,000 population,” we calculated

excess deaths per 100,000 people for both country-level and U.S. state-level analysis. The data

on the population of each of the 43 countries in our study comes from the same COVID-19

deaths dataset obtained from OWID, and the data on the population of 50 U.S. states and the

District of Columbia are included in The Economist excess deaths dataset.

3. ESTIMATION APPROACH

3.1. Effects of Social Distancing Policies on Excess Deaths

We examine the impacts of SIP policies on excess deaths per 100,000 population at both the

country-level and the U.S. state-level. To descriptively asses this relationship, we identified the

week in which the first COVID-19 death occurred in each of the 43 countries in our study and in

the 50 U.S. states and the District of Columbia. We then calculated the total excess deaths per

9
100,000 population in the 24 weeks (or 6 months) after the week of first COVID-19 death in

each country and U.S. state.

We first measured the speed of the implementation of SIP policy and the length of the

implementation of SIP policy during our period of analysis, i.e., the 24 weeks after the first

COVID-19 death. To measure the speed of the implementation of SIP policy, we calculate the

number of weeks a country or a U.S. state took to implement the first SIP policy from the week

of the first COVID-19 death. The duration, or length of the implementation of the SIP policy,

was measured as the number of weeks the SIP policy was in place from the week of first

COVID-19 death in a given country or a U.S. state to 24 weeks after the first COVID-19 death.

We start by descriptively estimating the association between excess deaths and speed of

implementation of SIP policies. To do so, we plot the total number of excess deaths per 100,000

population during the 24 weeks after the week of first COVID-19 death against the speed of the

implementation of SIP policy. We separately plot this relationship for countries and U.S. states.

We similarly compare the total number of excess deaths per 100,000 population during the 24

weeks after the first COVID-19 death with the duration of implementation of SIP policy in 25

weeks, i.e., the week of first COVID-19 death and 24 weeks from that first week.

To formally estimate the effect of SIP policy on excess deaths per 100,000 we used an event

study approach, which allows us to analyze the pre- and post-SIP implementation trends in

excess deaths per 100,000 population. We include week fixed effects and the respective country

and state fixed effects. For each event study, we estimated a regression model of the form:
𝑚 𝑛

𝑌𝑖𝑡 = D + ∑ E𝑗 (𝐿𝑎𝑔 𝑗)𝑖𝑡 + ∑ O𝑘 (𝐿𝑒𝑎𝑑 𝑘)𝑖𝑡 + J𝑖 + G𝑡 + H (2)
𝑗=2 𝑘=1

10
In this model, the outcome of interest, 𝑌𝑖𝑡 , represents excess deaths per 100,000 population.

Lags and leads indicate that the given state and/or country is a number of weeks away from the

time when the policy is implemented. Lags represent the pre-treatment period, while the lead

terms represent the post-treatment period. The first lag period is chosen as the baseline period;

hence it is omitted from the model specification. J𝑖 and G𝑡 are unit (state or country) and week

fixed effects, respectively. Here, i represents a given unit, i.e., a country for the country-level

analysis and a state for the U.S. state-level analysis, t represents the week number, and H is the

error termIn all specifications, we cluster standard errors at the respective geographic unit

(country or U.S. state).

These event studies provide estimates of the mean effect of SIP policies on excess deaths.

However, there are three empirical challenges associated with assessing the impacts of SIP

policies. First, the timing of implementation of SIP policies might be endogenous. Second, SIP

policies might have heterogeneous effects. And third, the distribution of excess deaths might be

skewed. We conduct several robustness checks to address these empirical challenges. These

additional checks and their results are described in section 4.2.

4. RESULTS

4.1. Association Between SIP Policies and Excess Mortality

Figure 1 compares excess deaths per 100,000 population against the number of weeks each

country/U.S. state took to implement the SIP policy following the first COVID-19 death in that

country/U.S. state. If SIP policies impact the trajectory of the COVID-19 pandemic, then regions

that implement SIP policies sooner after initial exposure to the pandemic are likely to experience

lower deaths. When comparing across countries (Panel A), we observe the expected relationship.

11
A one-week increase in the week in which a country implements a SIP policy (i.e., a one-week

delay in the implementation of a SIP policy), is associated with a 9.0 (95% CI: -0.3 to 18.3)

increase in excess deaths per 100,000 population. When comparing between U.S. states (Panel

B), we find the opposite relationship—a 0.4 (95% CI: -7.8 to 7.0) decrease, but the association is

small in magnitude and the confidence interval is large.

Figure 2 presents similar descriptive evidence on the duration of the implementation of SIP

policies following the first COVID-19 death in each country/U.S. state, as measured by the

number of weeks with an active SIP policy in the 24 weeks post first COVID-19 death, and the

number of excess deaths per 100,000 population in that country/U.S. state. For each geography,

the figure plots the number of weeks between the implementation of SIP policies and excess

deaths per 100,000 population in the 24 weeks following the first COVID-19 death. If SIP

policies reduce excess deaths, then there should be a negative association between the length of

time a SIP policy has been implemented and cumulative deaths. However, when comparing

across countries (Panel A), we observe a general upward trend, indicating that countries with a

longer duration of SIP policies are the ones with higher excess deaths per 100,000 residents in

the 24 weeks following a COVID-19 death. Across countries, a one-week increase in the

duration of SIP policies is associated with a 2.7 (95% CI: 1.0 to 4.3) increase in excess deaths

per 100,000 population. In the U.S. comparison (Panel B), the association is smaller in

magnitude and the confidence interval overlaps with zero. A one-week increase in the duration

of SIP policies is associated with a 1.3 (95% CI: -0.5 to 3.1) increase in excess deaths per

100,000 population.

These descriptive figures suggest that the implementation of SIP policies does not lead to

reductions in excess mortality. However, while suggestive, these figures are challenging to

12
interpret causally. Countries and U.S. states endogenously implement SIP policies in response to

their exposure to the COVID-19 pandemic. In addition, a variety of unobserved factors might

influence the decision to implement a SIP policy and also might be correlated with excess deaths.

To more formally estimate the effects of SIP policies on excess deaths, event studies that

estimate trends in excess mortality before and after the implementation of SIP policies are

presented in Figure 3. For both the country (Panel A) and U.S. state (Panel B) comparison, we

find that following the implementation of SIP policies, excess mortality increases. The increase

in excess mortality is statistically significant in the immediate weeks following SIP

implementation for the international comparison. In the ten weeks following SIP

implementation, we observe an approximately 10-per 100,000 population increase in excess

deaths, relative to pre-implementation trends and relative to countries without a SIP policy. For

the within-U.S. comparison, excess mortality trends below zero following 20 weeks of SIP

implementation. While the estimation magnitudes reach approximately 5-per 100,000

population, the confidence intervals are wide and overlap zero in all but one post-implementation

week.

The results from the event study regression models suggest that difference in excess

mortality between countries that implemented SIP versus countries that did not implement SIP

was trending downwards in the weeks prior to SIP implementation. Had this pre-existing

difference in mortality trends continued, we would expect lower excess mortality in the weeks

following SIP implementation in countries that implemented SIP policies relative to countries

that did not implement policies. However, we find that the pre-existing trend reversed following

implementation of SIP policies. This suggest that our estimates of the effects of SIP on excess

mortality are conservative, as pre-existing trends are biased towards finding a protective effect of

13
SIP. We find no evidence of pre-existing differences in trends in excess mortality across U.S.

states prior to implementation of SIP. However, similar to international comparisons, we find

that SIP implementation was associated with an increase in excess mortality.

Figure 4 presents results that separately estimate event studies for countries/states with pre-

SIP COVID-19 death rates below and above the median rates. For both the countries (Panel A)

and U.S. states (Panel B) comparisons, we do not observe differences in excess death trends

before and after the implementation of SIP policies based on pre-SIP COVID-19 death rates. In

all four cases, we do not observe that the implementation of SIP policies leads to reductions in

excess deaths. This finding suggests that the effect of SIP policies is not differentially impacted

by the severity of the COVID-19 pandemic prior to implementation of SIP policies.

4.2. Robustness Tests

We conduct several additional tests to assess the validity of our results. The primary

empirical challenge with assessing the impacts of SIP policies is the potential endogeneity of the

timing of implementation of these policies. For example, U.S. states and countries might

implement SIP policies when the trajectory of the pandemic creates an expectation of rising

COVID-19 cases (Cantor et al. 2020). Related to the results in Figure 4, we first test the

robustness of our results by non-parametrically controlling for the trajectory of the COVID-19

pandemic. For each country and state, we add fixed effects for the number of weeks between the

first COVID-19 death and the current week. As shown in Appendix Figures A1, we find little or

no protective effect of SIP policies.

Second, we consider alternative functional forms. COVID-19 deaths are a skewed outcome,

and our linear model may be influenced by outliers. As a solution, we log-transform excess

deaths. For weeks with negative excess deaths, we recode the log-transformed excess deaths as

14
zero. As shown in Appendix Figure A2, we find increases in log-transformed deaths following

the implementation of SIP policies in both the international and US comparisons.

Third, we examine the potential of heterogeneity in the impact of SIP policies. In particular,

in Appendix Figures B1 and B2, we present additional event studies that separately estimate the

effects of SIP policies in each country/U.S. state. When comparing across countries, the only

countries in which we observe the introduction of SIP policies negatively changes the trajectory

of excess deaths are Australia, Malta, and New Zealand. All three countries are islands. In every

other country, we observe either no visual change in excess deaths or increases in excess deaths.

The results are similar for U.S. states. We observe small reductions in excess deaths for Hawaii,

which is also an island, following the implementation of a SIP policy. The other U.S. states

either show no change in excess deaths, or in many U.S. states, spikes in excess deaths following

the implementation of SIP policies. Further, we do not see that countries or U.S. states that

implemented SIP policies earlier, and thus in which SIP policies had more time in effect, had

lower excess deaths than countries/U.S. states that were slower to implement SIP policies.

This descriptive evidence of the heterogeneity in the effect of SIP policies raises important

methodological concerns that correlation between differences in the timing of SIP policies and

heterogeneity in the effect of SIP policies could bias our results (Goodman-Bacon 2021). Our

fifth robustness test applies a dynamic treatment effects approach to re-estimate the event studies

(Sun and Abraham 2020). As shown in Appendix Figure A3, this alternative event study

estimation approach produces similar results. In both the international and U.S. state-level

comparisons, the implementation of SIP policies is followed by increases in excess deaths. In the

international comparison, the magnitude of the increase is lower and less precisely estimated

15
than in our main results. In the U.S. state-level comparison, the standard event study and

dynamic event study both show similar increases in excess deaths following SIP implementation.

Fourth, we examine the mortality components of excess deaths—COVID-19 deaths and non-

COVID deaths, although we note that there is likely measurement error in determining the cause

of death. In Appendix Figure A4, we find the increase in excess deaths is partially driven by

increases in COVID-19 deaths. However, it is unclear whether this is a true increase in COVID-

19 deaths or reclassification of non-COVID deaths to COVID-19 deaths due to increased

salience of the pandemic after implementation of SIP. We also find that COVID-19 deaths were

declining in states that implemented policies prior to implementation of policies. This decline

again suggests that we are potentially biased in favor of finding a protective effect of SIP

policies. But despite this potential bias, we find that the pre-existing trend is reversed and

COVID-19 deaths increase following SIP implementation in both the international and US

comparisons. The increase in COVID-19 deaths also exists when using the dynamic treatment

effect event study approach (Appendix Figure A5). We also find increases in non-COVID-19

deaths (Appendix Figure A6 and A7), although the magnitude is smaller, and the estimates are

less precise.

Fifth, we examine the effect of SIP policies on confirmed COVID-19 cases, although we note

that there is likely measurement error in the counting of COVID-19 cases as many cases go

undiagnosed as they are asymptomatic or due to limited testing of symptomatic cases early in the

pandemic (Bendavid et al. 2021; Sood et al. 2020). It is also possible that measurement error for

COVID-19 cases is correlated with implementation of SIP policies. In the international

comparison, we find SIP policy implementation is followed by increases in confirmed cases

(Appendix Figure A8). This comparison also shows a large and decreasing pre-implementation

16
trend in confirmed cases. In the U.S. state-level comparison, we find that implementation of SIP

policies does not change confirmed COVID-19 cases. Around week 20 post-implementation,

confirmed cases decline, but the confidence intervals are wide and overlap zero in all weeks. A

potential explanation for the increasing trend in both excess and COVID-19 deaths and flat

COVID-19 cases is that SIP policies prevented transmission among people with low COVID-19

mortality risks.

Sixth, a related concern is that SIP policies may be implemented in anticipation of the

trajectory of the COVID-19 pandemic, which would potentially contaminate the pre-

implementation trends. We estimate a test that treats SIP implementation week and the

subsequent four weeks as the policy implementation window, which allows us to account for

anticipation of excess mortality for the four weeks following SIP implementation. We chose four

weeks as the anticipation period following CDC ensemble models, which are likely to be used by

policymakers to forecast the number of COVID-19 deaths (Ray et al. 2020). We then repeat the

event study comparing excess deaths pre and post implementation. Our results remain

qualitatively similar and we continue to find no protective effect of SIP policies on excess

deaths. We repeat this analysis with a longer eight week policy implementation window and find

similar results (Appendix Figures A9 to A10). In addition, we control for lagged cases up to four

and eight weeks prior to SIP implementation. Policymakers might implement SIP policies based

on the trajectory of lagged COVID-19 cases. As shown in Appendix Figures A11 and A12, we

find that our results are robust to including controls for lagged new COVID-19 cases per 100,000

population.

Seventh, we examine changes in related policies following the implementation of SIP

policies. While SIP policies are the most widespread form of COVID-19 risk mitigation, other

17
similar policies, such as school closures and mask mandates, exist. If these policies are

concurrently implemented with SIP policies, then our estimates represent the combined effects of

these policies. Alternatively, SIP policies may substitute for other, potentially more effective,

policies. In such a case, we should expect the stringency index to decline following SIP

implementation and the removal of other policies following SIP implementation may explain the

observed increase in excess mortality. To test this scenario, we use the Oxford stringency index,

which measures the number of implemented risk mitigation policies, and measure changes in

related policy implementation following SIP policies. We measure changes in the COVID-19

stringency index for policies, both including and excluding SIP policies, following SIP

implementation. Appendix Figures A13 and A14 show increases in the stringency index

following SIP implementation, but the confidence intervals are wide.

Finally, we examine changes in excess deaths per 100,000 population following the

implementation of more stringent SIPs. We restrict the analysis to policies that “required” the

population to not leave the house with exceptions for daily exercise, grocery shopping, and

essential trips. Event studies with this sample produce results consistent with the main results

(Appendix Figure A15). In the international models, we observe an increase in excess deaths

after the implementation of these stronger SIPs. In the within-U.S. comparison, we find a small

increase in excess deaths which trends towards below zero post-week 20, however, the

confidence intervals overlap zero in all weeks.

5. DISCUSSION

In response to the COVID-19 pandemic, many countries implemented social distancing and

SIP policies. The effects of these policies on public health are a-priori ambiguous and complex.

We used data from 43 countries and all U.S. states and find the introduction of SIP policies did

18
not lead to reductions in excess deaths. In fact, we find that difference in COVID-19 cases,

COVID-19 deaths, and excess mortality between countries that implemented SIP versus

countries that did not implement SIP was trending downwards in the weeks prior to SIP

implementation. Had this pre-existing difference in trends in outcomes continued, we would

expect a protective effect in the weeks following SIP implementation in countries that

implemented SIP policies relative to countries that did not implement policies. However, we find

that the pre-existing trend in excess mortality reversed following implementation of SIP policies.

This suggest that our estimates of the effects of SIP on excess mortality are conservative as pre-

existing trends are biased towards finding a protective effect of SIP. We emphasize that our

results should be interpreted using an “intent-to-treat” framework. We do not estimate the effect

of “ideal” SIP policies or of improved compliance with SIP policies, but rather evaluate the

impact of SIP policies that were implemented.

This study is not without limitations. First, our primary outcome of interest was all-cause

mortality as it is more comprehensive and less prone to measurement error compared to cause

specific deaths such as COVID-19 deaths. However, total mortality can also suffer from

measurement error (COVID-19 Excess Mortality Collaborators 2022). Registering deaths may

have lags and upward revision of mortality data is common in many countries. Deaths may be

undercounted, especially in developing countries or rural areas due to lack of resources. Second,

we are unable to measure the impacts of SIP policies on quality of life. SIP policies likely

adversely affected quality of life and a full welfare analysis of SIP policies should account for

these effects. Third, SIP policies were endogenously implemented in response to the COVID-19

pandemic. We find similar results when stratifying based on pre-SIP exposure to the pandemic

and that controlling for exposure to the pandemic does not show a protective effect of SIP

19
policies, but these sensitivity tests may not fully account for policy endogeneity. Fourth, it might

be challenging to collect data on SIP policies, across many countries and states. Although the

data source we use for describing the policy environment has been used in many peer reviewed

publications, we cannot rule out measurement error. Fifth, enforcement and implementation of

SIP policies could vary across countries or U.S. states. However, the country-by-country and

state-by-state event studies suggest that the findings were qualitatively similar across countries

and U.S. states other than for a few island countries and Hawaii. We also found increases in

COVID-19 deaths when using a dynamic treatment effect event study approach. Sixth, it is

possible that SIP policies were implemented with other policies related to the pandemic and we

cannot completely isolate the causal effects of SIP policies alone. Seventh, there is substantial

variation in the excess deaths within an individual country (Michelozzi et al. 2020; Kontopantelis

et al. 2021), and within an individual state (Ackley et al. 2021). Future work should examine

whether the effect of SIP policies differs at smaller geographic levels. Finally, our results are

limited to 2020. The number of excess deaths in 2021 exceeded the number of excess deaths in

2020 in the U.S. and globally (Heuveline 2022; COVID-19 Excess Mortality Collaborators

2022).

Overall, the results suggest that the implementation of SIP policies does not appear to have

met the aim of reducing excess mortality or COVID-19 deaths. There are several potential

explanations for this finding. First, it is possible that SIP policies do not slow COVID-19

transmission. As discussed earlier, prior studies find only a modest effect of SIP policies on

mobility. A potential reason for the modest impact on mobility may be that individuals have a

private response to change behavior to avoid COVID-19 risk, even in the absence of SIP

policies. Effectively preventing COVID-19 transmission may require more stringent policies.

20
While stronger SIP policies may have been more effective in reducing COVID-19 transmission,

they would have also likely increased the adverse consequences of SIP policies. On the other

hand, SIP policies could have substituted for other potentially more targeted policies, such as

more robust testing or contact tracing. Given the substantial age-gradient of COVID-19

mortality, policies more focused on high-risk individuals may have been more effective than the

blunt impacts of SIP policies (Shruti Gupta et al. 2020). In addition, mobility across US states

was not restricted during the pandemic (Dave, McNichols, and Sabia 2021). The efficacy of a

SIP policy in a given region could have been diminished by travel between different regions. It is

possible that if regions were to physically isolate, then SIP policies could have been more

effective, consistent with our findings of effective SIP policies in islands.

Second, many SIP policies were focused on formal gatherings, such as work, education,

transportation, and dining restrictions. SIP policies likely reduced formal gatherings but might

have increased informal gatherings within the household. However, other work finds increases in

COVID-19 infection following events associated with informal social gatherings, even among

households in counties with active SIP policies (Whaley et al. 2021). In some cases, the risk of

COVID-19 transmission might be higher at home or in informal settings with family and friends

than at workplaces with risk mitigation protocols in place (Mulligan 2021).

Third, it is possible that SIP policies increased deaths of despair due to economic and social

isolation effects of SIP policies. Recent estimates in the U.S between March and August 2020

show that drug overdoses, homicides, and unintentional injuries increased in 2020, while

suicides declined (Faust et al. 2021). Relatedly, existing studies suggest that SIP policies led to a

reduction in non-COVID-19 health care, which might have contributed to an increase in non-

COVID-19 deaths (Cantor et al. 2022; Ziedan, Simon, and Wing 2020).

21
In light of this evidence, continued reliance on SIP policies to slow COVID-19 transmission

may not be optimal. Instead, the best policy response may be pharmaceutical interventions in the

form of vaccinations and therapeutics when they become available. Early evidence suggests that

initial vaccination efforts have led to large reductions in COVID-19 incidence and mortality

(Christie 2021; X. Chen et al. 2021; Haas et al. 2021; Sumedha Gupta et al. 2021). Policy efforts

to promote vaccination are thus likely to have much larger positive impacts.

22
6. REFERENCES

Abaluck, Jason, Laura H. Kwong, Ashley Styczynski, Ashraful Haque, Md. Alamgir Kabir,
Ellen Bates-Jefferys, Emily Crawford, et al. 2021. “Normalizing Community Mask-
Wearing: A Cluster Randomized Trial in Bangladesh.” Working Paper 28734. Working
Paper Series. National Bureau of Economic Research. https://doi.org/10.3386/w28734.
Ackley, Calvin A., Dielle J. Lundberg, Irma T. Elo, Samuel H. Preston, and Andrew C. Stokes.
2021. “County-Level Estimates of Excess Mortality Associated with COVID-19 in the
United States.” MedRxiv, May, 2021.04.23.21255564.
https://doi.org/10.1101/2021.04.23.21255564.
Aparicio Fenoll, Ainoa, and Shoshana Grossbard. 2020. “Intergenerational Residence Patterns
and Covid-19 Fatalities in the EU and the US*.” Economics and Human Biology 39
(December): 100934. https://doi.org/10.1016/j.ehb.2020.100934.
Askitas, Nikolaos, Konstantinos Tatsiramos, and Bertrand Verheyden. 2021. “Estimating
Worldwide Effects of Non-Pharmaceutical Interventions on COVID-19 Incidence and
Population Mobility Patterns Using a Multiple-Event Study.” Scientific Reports 11 (1):
1972. https://doi.org/10.1038/s41598-021-81442-x.
Babin, J. Jobu, Marine Foray, and Andrew Hussey. 2021. “Shelter-in-Place Orders, Loneliness,
and Collaborative Behavior.” Economics and Human Biology 43 (August): 101056.
https://doi.org/10.1016/j.ehb.2021.101056.
Banerjee, Abhijit, Marcella Alsan, Emily Breza, Arun G. Chandrasekhar, Abhijit Chowdhury,
Esther Duflo, Paul Goldsmith-Pinkham, and Benjamin A. Olken. 2020. “Messages on
COVID-19 Prevention in India Increased Symptoms Reporting and Adherence to
Preventive Behaviors Among 25 Million Recipients with Similar Effects on Non-
Recipient Members of Their Communities.” Working Paper 27496. Working Paper
Series. National Bureau of Economic Research. https://doi.org/10.3386/w27496.
Baral, Stefan David, Sharmistha Mishra, Daouda Diouf, Nittaya Phanuphak, and David Dowdy.
2020. “The Public Health Response to COVID-19: Balancing Precaution and Unintended
Consequences.” Annals of Epidemiology 46 (June): 12–13.
https://doi.org/10.1016/j.annepidem.2020.05.001.
Bendavid, Eran, Bianca Mulaney, Neeraj Sood, Soleil Shah, Rebecca Bromley-Dulfano, Cara
Lai, Zoe Weissberg, et al. 2021. “COVID-19 Antibody Seroprevalence in Santa Clara
County, California.” International Journal of Epidemiology 50 (2): 410–19.
https://doi.org/10.1093/ije/dyab010.
Berry, Christopher R., Anthony Fowler, Tamara Glazer, Samantha Handel-Meyer, and Alec
MacMillen. 2021. “Evaluating the Effects of Shelter-in-Place Policies during the
COVID-19 Pandemic.” Proceedings of the National Academy of Sciences of the United
States of America 118 (15). https://doi.org/10.1073/pnas.2019706118.
Bisbee, James, and Diana Lee. 2020. “Mobility and Elite Cues: Partisan Responses to Covid-19,”
August. https://doi.org/10.33774/apsa-2020-76tv9.
Cantor, Jonathan, Neeraj Sood, Dena M. Bravata, Megan Pera, and Christopher Whaley. 2020.
“The Impact of the COVID-19 Pandemic and Policy Response on Health Care
Utilization: Evidence from County-Level Medical Claims and Cellphone Data.” Working
Paper 28131. National Bureau of Economic Research.

23
———. 2022. “The Impact of the COVID-19 Pandemic and Policy Response on Health Care
Utilization: Evidence from County-Level Medical Claims and Cellphone Data.” Journal
of Health Economics, January, 102581. https://doi.org/10.1016/j.jhealeco.2022.102581.
Cappa, Claudia, and Isabel Jijon. 2021. “COVID-19 and Violence against Children: A Review of
Early Studies.” Child Abuse & Neglect, April, 105053.
https://doi.org/10.1016/j.chiabu.2021.105053.
Chen, Sophia, Deniz O. Igan, Nicola Pierri, and Andrea F. Presbitero. 2020. “Tracking the
Economic Impact of COVID-19 and Mitigation Policies in Europe and the United
States.” 2020/125. IMF Working Papers. IMF Working Papers. International Monetary
Fund. https://ideas.repec.org/p/imf/imfwpa/2020-125.html.
Chen, Xiao, Hanwei Huang, Jiandong Ju, Ruoyan Sun, and Jialiang Zhang. 2021. “Impact of
Vaccination on the COVID-19 Pandemic: Evidence from U.S. States.” MedRxiv, May,
2021.05.08.21256892. https://doi.org/10.1101/2021.05.08.21256892.
Cheng, Yafang, Nan Ma, Christian Witt, Steffen Rapp, Philipp S. Wild, Meinrat O. Andreae,
Ulrich Pöschl, and Hang Su. 2021. “Face Masks Effectively Limit the Probability of
SARS-CoV-2 Transmission.” Science 372 (6549): 1439–43.
https://doi.org/10.1126/science.abg6296.
Christie, Athalia. 2021. “Decreases in COVID-19 Cases, Emergency Department Visits, Hospital
Admissions, and Deaths Among Older Adults Following the Introduction of COVID-19
Vaccine — United States, September 6, 2020–May 1, 2021.” MMWR. Morbidity and
Mortality Weekly Report 70. https://doi.org/10.15585/mmwr.mm7023e2.
Chu, Derek K., Elie A. Akl, Stephanie Duda, Karla Solo, Sally Yaacoub, Holger J. Schünemann,
Derek K. Chu, et al. 2020. “Physical Distancing, Face Masks, and Eye Protection to
Prevent Person-to-Person Transmission of SARS-CoV-2 and COVID-19: A Systematic
Review and Meta-Analysis.” The Lancet 395 (10242): 1973–87.
https://doi.org/10.1016/S0140-6736(20)31142-9.
Correia, Sergio, Stephan Luck, and Emil Verner. 2020. “Pandemics Depress the Economy,
Public Health Interventions Do Not: Evidence from the 1918 Flu.” SSRN Scholarly
Paper ID 3561560. Rochester, NY: Social Science Research Network.
https://doi.org/10.2139/ssrn.3561560.
Courtemanche, Charles, Joseph Garuccio, Anh Le, Joshua Pinkston, and Aaron Yelowitz. 2020.
“Strong Social Distancing Measures In The United States Reduced The COVID-19
Growth Rate.” Health Affairs, May, 10.1377/hlthaff.2020.00608.
https://doi.org/10.1377/hlthaff.2020.00608.
COVID-19 Excess Mortality Collaborators. 2022. “Estimating Excess Mortality Due to the
COVID-19 Pandemic: A Systematic Analysis of COVID-19-Related Mortality, 2020-
21.” Lancet (London, England), March, S0140-6736(21)02796-3.
https://doi.org/10.1016/S0140-6736(21)02796-3.
Dave, Dhaval, Drew McNichols, and Joseph J. Sabia. 2021. “The Contagion Externality of a
Superspreading Event: The Sturgis Motorcycle Rally and COVID-19.” Southern
Economic Journal 87 (3): 769–807. https://doi.org/10.1002/soej.12475.
Faust, Jeremy M., Chengan Du, Katherine Dickerson Mayes, Shu-Xia Li, Zhenqiu Lin, Michael
L. Barnett, and Harlan M. Krumholz. 2021. “Mortality From Drug Overdoses,
Homicides, Unintentional Injuries, Motor Vehicle Crashes, and Suicides During the
Pandemic, March-August 2020.” JAMA. https://doi.org/10.1001/jama.2021.8012.

24
Ge, Yong, Wen-Bin Zhang, Haiyan Liu, Corrine W. Ruktanonchai, Maogui Hu, Xilin Wu,
Yongze Song, et al. 2021. “Effects of Worldwide Interventions and Vaccination on
COVID-19 between Waves and Countries.” MedRxiv, April, 2021.03.31.21254702.
https://doi.org/10.1101/2021.03.31.21254702.
Giattino, Charlie, Hannah Ritchie, Max Roser, Esteban Ortiz-Ospina, and Joe Hasell. 2021.
“Excess Mortality during the Coronavirus Pandemic (COVID-19) - Statistics and
Research.” Our World in Data. 2021. https://ourworldindata.org/excess-mortality-covid.
Gilligan, Megan, J. Jill Suitor, Marissa Rurka, and Merril Silverstein. 2020. “Multigenerational
Social Support in the Face of the COVID-19 Pandemic.” Journal of Family Theory &
Review 12 (4): 431–47. https://doi.org/10.1111/jftr.12397.
Goodman-Bacon, Andrew. 2021. “Difference-in-Differences with Variation in Treatment
Timing.” Journal of Econometrics, June. https://doi.org/10.1016/j.jeconom.2021.03.014.
Gupta, Shruti, Salim S. Hayek, Wei Wang, Lili Chan, Kusum S. Mathews, Michal L. Melamed,
Samantha K. Brenner, et al. 2020. “Factors Associated With Death in Critically Ill
Patients With Coronavirus Disease 2019 in the US.” JAMA Internal Medicine 180 (11):
1436–47. https://doi.org/10.1001/jamainternmed.2020.3596.
Gupta, Sumedha, Jonathan Cantor, Kosali Simon, Ana Bento, Coady Wing, and Christopher M.
Whaley. 2021. “COVID-19 Mass Vaccination Associated with 140,000 Averted COVID-
19 Deaths in the United States.” Health Affairs.
Haas, Eric J., Frederick J. Angulo, John M. McLaughlin, Emilia Anis, Shepherd R. Singer, Farid
Khan, Nati Brooks, et al. 2021. “Impact and Effectiveness of MRNA BNT162b2 Vaccine
against SARS-CoV-2 Infections and COVID-19 Cases, Hospitalisations, and Deaths
Following a Nationwide Vaccination Campaign in Israel: An Observational Study Using
National Surveillance Data.” The Lancet 397 (10287): 1819–29.
https://doi.org/10.1016/S0140-6736(21)00947-8.
Hale, Thomas, Noam Angrist, Rafael Goldszmidt, Beatriz Kira, Anna Petherick, Toby Phillips,
Samuel Webster, et al. 2021. “A Global Panel Database of Pandemic Policies (Oxford
COVID-19 Government Response Tracker).” Nature Human Behaviour 5 (4): 529–38.
https://doi.org/10.1038/s41562-021-01079-8.
Hale, Thomas, Laura Hallas, Ariq Hatibie, Saptarshi Majumdar, and Monika Pyarali. 2020.
“VARIATION IN US STATES’ RESPONSES TO COVID-19.” BLAVATNIK
SCHOOL WORKING PAPER.
https://www.bsg.ox.ac.uk/research/publications/variation-us-states-responses-covid-19.
Heuveline, Patrick. 2022. “The Dramatic Surge of Excess Mortality in the United States between
2017 and 2021.” medRxiv. https://doi.org/10.1101/2022.03.21.22272722.
Holland, Kristin M., Christopher Jones, Alana M. Vivolo-Kantor, Nimi Idaikkadar, Marissa
Zwald, Brooke Hoots, Ellen Yard, et al. 2021. “Trends in US Emergency Department
Visits for Mental Health, Overdose, and Violence Outcomes Before and During the
COVID-19 Pandemic.” JAMA Psychiatry 78 (4): 372–79.
https://doi.org/10.1001/jamapsychiatry.2020.4402.
Honein, Margaret A. 2020. “Summary of Guidance for Public Health Strategies to Address High
Levels of Community Transmission of SARS-CoV-2 and Related Deaths, December
2020.” MMWR. Morbidity and Mortality Weekly Report 69.
https://doi.org/10.15585/mmwr.mm6949e2.

25
Hsu, Lin-Chi, and Alexander Henke. 2021. “COVID-19, Staying at Home, and Domestic
Violence.” Review of Economics of the Household 19 (1): 145–55.
https://doi.org/10.1007/s11150-020-09526-7.
Jay, Jonathan, Jacob Bor, Elaine O. Nsoesie, Sarah K. Lipson, David K. Jones, Sandro Galea,
and Julia Raifman. 2020. “Neighbourhood Income and Physical Distancing during the
COVID-19 Pandemic in the United States.” Nature Human Behaviour 4 (12): 1294–
1302. https://doi.org/10.1038/s41562-020-00998-2.
Johns Hopkins Coronavirus Resource Center. 2022. “COVID-19 Map.” Johns Hopkins
Coronavirus Resource Center. 2022. https://coronavirus.jhu.edu/map.html.
Kaufman, Brystana G., Rebecca Whitaker, Nirosha Mahendraratnam, Sophie Hurewitz, Jeremy
Yi, Valerie A. Smith, and Mark McClellan. 2021. “State Variation in Effects of State
Social Distancing Policies on COVID-19 Cases.” BMC Public Health 21 (1): 1239.
https://doi.org/10.1186/s12889-021-11236-3.
Kaufman, Harvey W., Zhen Chen, Justin Niles, and Yuri Fesko. 2020. “Changes in the Number
of US Patients With Newly Identified Cancer Before and During the Coronavirus Disease
2019 (COVID-19) Pandemic.” JAMA Network Open 3 (8): e2017267–e2017267.
https://doi.org/10.1001/jamanetworkopen.2020.17267.
Kontopantelis, Evangelos, Mamas A. Mamas, John Deanfield, Miqdad Asaria, and Tim Doran.
2021. “Excess Mortality in England and Wales during the First Wave of the COVID-19
Pandemic.” J Epidemiol Community Health 75 (3): 213–23. https://doi.org/10.1136/jech-
2020-214764.
Krelle, Holly, Claudia Barclay, and Charles Tallack. 2020. “Understanding Excess Mortality.”
The Health Foundation. May 6, 2020. https://www.health.org.uk/news-and-
comment/charts-and-infographics/understanding-excess-mortality-the-fairest-way-to-
make-international-comparisons.
Leslie, Emily, and Riley Wilson. 2020. “Sheltering in Place and Domestic Violence: Evidence
from Calls for Service during COVID-19.” Journal of Public Economics 189
(September): 104241. https://doi.org/10.1016/j.jpubeco.2020.104241.
Maringe, Camille, James Spicer, Melanie Morris, Arnie Purushotham, Ellen Nolte, Richard
Sullivan, Bernard Rachet, and Ajay Aggarwal. 2020. “The Impact of the COVID-19
Pandemic on Cancer Deaths Due to Delays in Diagnosis in England, UK: A National,
Population-Based, Modelling Study.” The Lancet Oncology 21 (8): 1023–34.
https://doi.org/10.1016/S1470-2045(20)30388-0.
Mason, Maryann, Sarah B. Welch, Ponni Arunkumar, Lori Ann Post, and Joseph M. Feinglass.
2021. “Notes from the Field: Opioid Overdose Deaths Before, During, and After an 11-
Week COVID-19 Stay-at-Home Order — Cook County, Illinois, January 1, 2018–
October 6, 2020.” MMWR. Morbidity and Mortality Weekly Report 70.
https://doi.org/10.15585/mmwr.mm7010a3.
McBain, Ryan K., Jonathan H. Cantor, Anupam B. Jena, Megan F. Pera, Dena M. Bravata, and
Christopher M. Whaley. 2021. “Decline and Rebound in Routine Cancer Screening Rates
During the COVID-19 Pandemic.” Journal of General Internal Medicine, March.
https://doi.org/10.1007/s11606-021-06660-5.
Michelozzi, Paola, Francesca de’Donato, Matteo Scortichini, Patrizio Pezzotti, Massimo
Stafoggia, Manuela De Sario, Giuseppe Costa, et al. 2020. “Temporal Dynamics in Total
Excess Mortality and COVID-19 Deaths in Italian Cities.” BMC Public Health 20 (1):
1238. https://doi.org/10.1186/s12889-020-09335-8.

26
Mulligan, Casey B. 2021. “The Backward Art of Slowing the Spread? Congregation Efficiencies
during COVID-19.” Working Paper 28737. Working Paper Series. National Bureau of
Economic Research. https://doi.org/10.3386/w28737.
Nguyen, Thuy D., Sumedha Gupta, Martin Andersen, Ana Bento, Kosali I. Simon, and Coady
Wing. 2020. “Impacts of State Reopening Policy on Human Mobility.” 27235. NBER
Working Papers. NBER Working Papers. National Bureau of Economic Research, Inc.
https://ideas.repec.org/p/nbr/nberwo/27235.html.
Patrick, Stephen W., Laura E. Henkhaus, Joseph S. Zickafoose, Kim Lovell, Alese Halvorson,
Sarah Loch, Mia Letterie, and Matthew M. Davis. 2020. “Well-Being of Parents and
Children During the COVID-19 Pandemic: A National Survey.” Pediatrics, August.
https://doi.org/10.1542/peds.2020-016824.
Petterson, Stephen, John M. Westfall, and Benjamin F. Miller. 2020. “Projected Deaths of
Despair During COVID-19.” Well Being Trust. https://wellbeingtrust.org/areas-of-
focus/policy-and-advocacy/reports/projected-deaths-of-despair-during-covid-19/.
Rannan-Eliya, Ravindra Prasan, Nilmini Wijemunige, J. R. N. A. Gunawardana, Sarasi N.
Amarasinghe, Ishwari Sivagnanam, Sachini Fonseka, Yasodhara Kapuge, and Chathurani
P. Sigera. 2021. “Increased Intensity Of PCR Testing Reduced COVID-19 Transmission
Within Countries During The First Pandemic Wave.” Health Affairs (Project Hope) 40
(1): 70–81. https://doi.org/10.1377/hlthaff.2020.01409.
Ray, Evan L., Nutcha Wattanachit, Jarad Niemi, Abdul Hannan Kanji, Katie House, Estee Y.
Cramer, Johannes Bracher, et al. 2020. “Ensemble Forecasts of Coronavirus Disease
2019 (COVID-19) in the U.S.” medRxiv. https://doi.org/10.1101/2020.08.19.20177493.
Ritchie, Hannah, Esteban Ortiz-Ospina, Diana Beltekian, Edouard Mathieu, Joe Hasell, Bobbie
Macdonald, Charlie Giattino, Cameron Appel, Lucas Rodés-Guirao, and Max Roser.
2020. “Coronavirus Pandemic (COVID-19).” Our World in Data, March.
https://ourworldindata.org/coronavirus.
Sanmarchi, Francesco, Davide Golinelli, Jacopo Lenzi, Francesco Esposito, Angelo Capodici,
Chiara Reno, and Dino Gibertoni. 2021. “Exploring the Gap Between Excess Mortality
and COVID-19 Deaths in 67 Countries.” JAMA Network Open 4 (7): e2117359.
https://doi.org/10.1001/jamanetworkopen.2021.17359.
Singh, Surya, Mujaheed Shaikh, Katharina Hauck, and Marisa Miraldo. 2021. “Impacts of
Introducing and Lifting Nonpharmaceutical Interventions on COVID-19 Daily Growth
Rate and Compliance in the United States.” Proceedings of the National Academy of
Sciences of the United States of America 118 (12).
https://doi.org/10.1073/pnas.2021359118.
Sonu, Stan, Sharon Post, and Joe Feinglass. 2019. “Adverse Childhood Experiences and the
Onset of Chronic Disease in Young Adulthood.” Preventive Medicine 123 (June): 163–
70. https://doi.org/10.1016/j.ypmed.2019.03.032.
Sood, Neeraj, Paul Simon, Peggy Ebner, Daniel Eichner, Jeffrey Reynolds, Eran Bendavid, and
Jay Bhattacharya. 2020. “Seroprevalence of SARS-CoV-2–Specific Antibodies Among
Adults in Los Angeles County, California, on April 10-11, 2020.” JAMA 323 (23): 2425–
27. https://doi.org/10.1001/jama.2020.8279.
Stokes, Jeffrey E., and Sarah E. Patterson. 2020. “Intergenerational Relationships, Family
Caregiving Policy, and COVID-19 in the United States.” Journal of Aging & Social
Policy 32 (4–5): 416–24. https://doi.org/10.1080/08959420.2020.1770031.

27
Sun, Liyang, and Sarah Abraham. 2020. “Estimating Dynamic Treatment Effects in Event
Studies with Heterogeneous Treatment Effects.” Journal of Econometrics.
The Economist. 2020. “The Economist’s Tracker for Covid-19 Excess Deaths (Online).” 2020.
https://github.com/TheEconomist/covid-19-excess-deaths-tracker.
Thierry, Alain R., Brice Pastor, Ekaterina Pisareva, Francois Ghiringhelli, Olivier Bouché,
Christelle De La Fouchardière, Julie Vanbockstael, et al. 2021. “Association of COVID-
19 Lockdown With the Tumor Burden in Patients With Newly Diagnosed Metastatic
Colorectal Cancer.” JAMA Network Open 4 (9): e2124483.
https://doi.org/10.1001/jamanetworkopen.2021.24483.
Velazquez, Ana I., Jessica H. Hayward, Blake Gregory, and Niharika Dixit. 2021. “Trends in
Breast Cancer Screening in a Safety-Net Hospital During the COVID-19 Pandemic.”
JAMA Network Open 4 (8): e2119929.
https://doi.org/10.1001/jamanetworkopen.2021.19929.
Weill, Joakim, Mathieu Stigler, Olivier Deschenes, and Michael Springborn. 2021. “COVID-19
Mobility Policies Impacts: How Credible Are Difference-in-Differences Estimates?”
SSRN Scholarly Paper ID 3896512. Rochester, NY: Social Science Research Network.
https://doi.org/10.2139/ssrn.3896512.
Whaley, Christopher M., Jonathan Cantor, Megan Pera, and Anupam B. Jena. 2021. “Assessing
the Association Between Social Gatherings and COVID-19 Risk Using Birthdays.”
JAMA Internal Medicine, June. https://doi.org/10.1001/jamainternmed.2021.2915.
Whaley, Christopher M., Megan F. Pera, Jonathan Cantor, Jennie Chang, Julia Velasco, Heather
K. Hagg, Neeraj Sood, and Dena M. Bravata. 2020. “Changes in Health Services Use
Among Commercially Insured US Populations During the COVID-19 Pandemic.” JAMA
Network Open 3 (11): e2024984.
Xie, Xinyan, Qi Xue, Yu Zhou, Kaiheng Zhu, Qi Liu, Jiajia Zhang, and Ranran Song. 2020.
“Mental Health Status Among Children in Home Confinement During the Coronavirus
Disease 2019 Outbreak in Hubei Province, China.” JAMA Pediatrics 174 (9): 898–900.
https://doi.org/10.1001/jamapediatrics.2020.1619.
Xu, Dafeng. 2021. “Physical Mobility under Stay-at-Home Orders: A Comparative Analysis of
Movement Restrictions between the U.S. and Europe.” Economics and Human Biology
40 (January): 100936. https://doi.org/10.1016/j.ehb.2020.100936.
Yanez, N. David, Noel S. Weiss, Jacques-André Romand, and Miriam M. Treggiari. 2020.
“COVID-19 Mortality Risk for Older Men and Women.” BMC Public Health 20 (1):
1742. https://doi.org/10.1186/s12889-020-09826-8.
Ziedan, Engy, Kosali I Simon, and Coady Wing. 2020. “Effects of State COVID-19 Closure
Policy on NON-COVID-19 Health Care Utilization.” Working Paper 27621. Working
Paper Series. National Bureau of Economic Research. https://doi.org/10.3386/w27621.

28
Figures

Figure 1: Total excess deaths per 100,000 population after 24 weeks from the first COVID-19 death, compared
to number of weeks countries/states took to implement first SIP policy from the week of first COVID-19
death

(a) International Comparison

(b) United States Comparison

Note: These figures compare excess deaths per 100,000 population in the 24 weeks following the first COVID-
19 death (y-axis) against the number of weeks between the first death and implementation of shelter-in-place
policies using data from 43 countries (Panel A) and U.S. states (Panel B).

29
Figure 2: Total excess deaths per 100,000 population after 24 weeks from the first COVID-19 death, compared
to number of weeks SIP policy was implemented by countries/states from the week of first COVID-19 death
until twenty fifth week from the first week

(a) International Comparison

(b) United States Comparison

Note: These figures compare excess deaths per 100,000 population in the 24 weeks following the first COVID-
19 death (y-axis) against the number of weeks that a shelter-in-place policy was implemented between the week
of the first death and 24 weeks following the week of the first death using data from 43 countries (Panel A)
and U.S. states (Panel B).
30
Figure 3: Event study estimates of changes in excess deaths per 100,000 population, before and after shelter-
in-place policies

(a) International Comparison

(b) United States Comparison

31
Figure 4: Event study estimates of changes in excess deaths per 100,000 population, before and after shelter-
in-place policies, and based on COVID-19 deaths prior to policy implementation

(a) International Comparison

(b) United States Comparison

Note: These figures estimate changes in excess deaths per 100,000 population before and after implementa-
tion of shelter-in-place policies in 43 countries (Panel A) and U.S. states (Panel B). Event studies estimated
using the specification in equation (2), but separated between policies implemented in countries/states be-
low (left hand side) and above (right hand side) the median number of COVID-19 deaths prior to policy
implementation.

32
Appendix Figures

Figure A.1: Event study estimates of changes in excess deaths per 100,000 population, before and after
shelter-in-place policies with fixed eﬀects for number of weeks since the first COVID-19 death

(a) International Comparison

(b) United States Comparison

Note: These figures estimate changes in excess deaths per 100,000 population before and after implementation
of shelter-in-place policies in 43 countries (Panel A) and U.S. states (Panel B). Event studies estimated using
the specification in equation (2) but include fixed eﬀects for the number of weeks since the first COVID-19
death in that country or state.

33
Figure A.2: Event study estimates of changes in log of excess deaths per 100,000 population, before and
after shelter-in-place policies

(a) International Comparison

(b) United States Comparison

Note: These figures estimate changes in log-transformed excess deaths per 100,000 population before and
after implementation of shelter-in-place policies in 43 countries (Panel A) and U.S. states (Panel B). Event
studies estimated using the specification in equation (2).

34
Figure A.3: Sun Abraham event study estimates of changes in excess deaths per 100,000 population, before
and after shelter-in-place policies

(a) International Comparison

(b) United States Comparison

Note: These figures estimate changes in excess deaths per 100,000 population before and after implementation
of shelter-in-place policies in 43 countries (Panel A) and U.S. states (Panel B). Event studies estimated using
the specification in equation (2) but estimated using the Sun-Abraham dynamic event study approach.

35
Figure A.4: Event study estimates of changes in COVID-19 deaths per 100,000 population, before and after
shelter-in-place policies