What scientists have learnt from COVID lockdowns

In March 2021, a doctor in Brazil named Ricardo Savaris published a now-discredited research paper that went viral on social media¹.

It had been a year since the first wave of the COVID-19 pandemic forced governments to apply the desperate measures collectively known as lockdowns — cancelling sporting and cultural events, closing retail outlets, restaurants, schools and universities, and ordering people to stay at home. At the time, countries were once again dialling lockdown policies up or down, as the Alpha variant of the coronavirus SARS-CoV-2 surged in different places.

Lockdown measures did what they were supposed to. When they were enforced rigorously enough to reduce people’s social contacts sharply, they shrank COVID-19 outbreaks; several studies had demonstrated this.

But Savaris, an obstetrician and gynaecologist at the Federal University of Rio Grande do Sul in Porto Alegre, tried a fresh analysis together with three colleagues (who worked in statistics, computer science and informatics). They compared 87 locations around the world, in pairs, to see whether a lower rate of COVID-19 deaths correlated with greater time spent at home, assessed using anonymized cellphone data released by Google. In most cases, their paper in Scientific Reports concluded, it didn’t.

The paper was highlighted by prominent lockdown sceptics and some news sites and swiftly gained notoriety. “The findings were quite remarkable, on the face of it,” says Gideon Meyerowitz-Katz, an epidemiologist at the University of Wollongong, Australia. As he and others would show, the results were wrong, because of errors in the paper’s choice of statistical methods.

Within a week, Scientific Reports added an ‘editor’s note’ to the paper, alerting readers to criticisms. Nine months later, the journal published two letters^2,3 that laid out the paper’s errors. A week after that, it retracted the work, although neither Savaris nor his co-authors agreed with the retraction. (Scientific Reports is published by Springer Nature; Nature’s reporting is editorially independent of its publisher.)

The retracted paper is not the only one to contend that lockdowns failed to save lives. But these analyses are out of step with the majority of studies. Most scientists agree that lockdowns did curb COVID-19 deaths and that governments had little option but to restrict people’s social contacts in early 2020, to stem SARS-CoV-2’s spread and avert the collapse of health-care systems. “We needed to buy ourselves some time,” says Lauren Meyers, a biological data scientist at the University of Texas at Austin.

At the same time, it’s clear that lockdowns had huge costs, and there is debate about the utility of any subsequent lockdown measures. School and university closures disrupted education. Closing businesses contributed to financial and social hardship, mental ill health and economic downturns. “There’s costs and benefits,” says Samir Bhatt, a public-health statistician at Imperial College London and the University of Copenhagen.

Scientists have been studying the effects of lockdowns during the pandemic in the hope that their findings could inform the response to future crises. They have reached some conclusions: countries that acted quickly to bring in stringent measures did best at preserving both lives and their economies, for instance. But researchers have also encountered difficulties. Analysing competing harms and benefits often comes down not to scientific calculations, but to value judgements, such as how to weigh costs that fall on some sections of society more than others. That is what makes lockdowns so hard to study — and can lead to bitter disagreement.

Tricky calculation

There’s a fundamental difficulty with analysing the effects of COVID-19 lockdowns: it is hard to know what would have happened in their absence.

Lockdowns do reduce viral transmission, as the shutdown of Wuhan, China, showed when SARS-CoV-2 first emerged. Even in countries that didn’t emulate China’s all-in approach of closing borders, ordering citizens to stay at home and isolating people with COVID in central facilities, lockdown measures still cut disease spread. In May 2020, for instance, Bhatt and others analysed lockdowns in 11 European countries and extrapolated from the fall in viral transmission that these measures alone had saved more than 3 million lives⁴.

That paper’s methodology has also been questioned, however. One issue is that it could have overstated the size of the benefit because it assumes that without lockdown mandates, people wouldn’t have reduced their social contacts. In reality, rising deaths would probably have changed people’s behaviour.

That happened in Florida, for instance, where data show a reduction in mobility during the first wave about two weeks before lockdowns, says health-policy researcher Thomas Tsai at the Harvard T.H. Chan School of Public Health in Boston, Massachusetts. “People were watching the news in New York and Boston and seeing how severe COVID could be,” he says.

One analysis⁵ by political scientist Christopher Berry and his colleagues at the University of Chicago, Illinois, supports this. It suggests that US states’ shelter-in-place orders did little to further reduce COVID-19 cases and deaths, not because social distancing doesn’t work, but because people were already avoiding contact before the orders were imposed.

Other researchers have tried instead to compare whether countries with stricter lockdown policies performed better than those with more relaxed ones on measures such as disease transmission rates or deaths. This isn’t simple, either: enforcement, levels of government aid and compliance with official policies differed from one region to another — as did cultural context and a host of other factors, including population densities, levels of social contact and viral prevalence.

Take Sweden, for instance, which imposed relatively light restrictions in early 2020, keeping schools open for all but the oldest students. It experienced a lower rate of excess deaths in 2020 than did many other western European nations. But it is also a country where many people live alone (the average household size in Sweden is the lowest in the European Union), and where people have high trust in government, making it much easier for official recommendations alone — rather than mandates — to reduce social contacts and slow disease spread. Far from carrying on life as normal, Swedes reduced their mobility, as shown by mobile-phone data. Even so, its Nordic neighbours that imposed lockdowns performed better in 2020: age-standardized mortality rates show that Denmark, Finland and Norway experienced fewer deaths than normal that year, whereas Sweden experienced slightly more than usual. (As in other countries, Sweden also failed to prevent the most vulnerable people, such as those in elder-care homes, from dying of COVID-19.)

“It wasn’t really clear what is the best way of estimating the effectiveness of [lockdown] measures,” says Peter Klimek, a data scientist at the Medical University of Vienna. Still, by tracking the stringency and timing of government policies in more than 100 countries, researchers at the University of Oxford, UK, and their colleagues did find⁶ that the more stringent a nation’s containment policies, the more successful it was at averting deaths from COVID-19.

It is even harder to tease out subtler insights, such as which of the grab bag of lockdown policies — from closing schools to ordering people to stay at home — had the most effect, especially because policies were often announced in quick succession.

After the first wave of COVID-19, Klimek’s team analysed thousands of government interventions. The group noted that some measures seemed effective according to one modelling approach, but not according to others, and that their effectiveness estimates came with wide uncertainty ranges. But the researchers were able to produce an overall ranking (see ‘How effective were COVID-19 interventions?’). The most effective measures were policies banning small gatherings and closing businesses and schools, closely followed by land-border restrictions and national lockdowns. Less-intrusive measures — such as government support for vulnerable populations, and risk-communication strategies — also had an impact. Airport health checks, however, had no discernible benefit⁷.

Other studies have tried to put more precise figures on the effects of lockdown policies, but their findings differ. An analysis⁸ of 41 countries in Europe and elsewhere found that stay-at-home orders had a relatively small impact on transmission, reducing R — the average number of people that one person with COVID-19 will go on to infect — by just 13% beyond what could be achieved by closing schools and universities (38%) or limiting gatherings to 10 people or fewer (42%). Yet Bhatt’s analysis⁴ of 11 countries suggested that stay-at-home orders cut R by 81%, with school closures, public-event bans and other measures being less important. Klimek warns against generalizing about the effectiveness of lockdown policies on the basis of figures such as these. “The effectiveness of each intervention is highly context dependent,” he says. What several analyses suggest is that no single intervention can reduce R to below 1 (signifying that infections are declining): multiple measures achieve this by working in concert.

Go hard, go fast

The pre-vaccine period of the pandemic does show that countries that acted harshly and swiftly — the ‘go hard, go fast’ approach — often fared better than those that waited to implement lockdown policies. China’s harsh lockdowns eliminated COVID-19 locally, for a time. Successful countries that learnt from this were “proactive”, according to a May 2021 report⁹ by the Independent Panel on Pandemic Preparedness & Response, established by the World Health Organization in September 2020 to review the global response. Examples include island nations such as Iceland, Australia and New Zealand, which also benefited from being able to close their borders and take action before many people with the virus arrived.

Others have echoed this. Epidemiologist Edward Knock and other members of the Imperial College COVID-19 response team concluded that nationwide lockdown was the only measure that consistently took R below 1 in England. And the earlier that strict measures were imposed, the better. Knock estimated that had England introduced a nationwide lockdown one week earlier in March 2020, it would have halved deaths during the first wave¹⁰. A study¹¹ of government responses in Asia also suggested that a ‘go hard, go fast’ approach was best.

But harsher lockdowns aren’t always more effective by themselves, especially in countries where it is difficult for people to stay at home. Peru is an example. It imposed early and strict lockdown measures, but experienced an excess death rate much higher than did other countries in the region that used less draconian measures. Peru is still held up as evidence that lockdowns don’t work — but in fact it struggled to enforce them. The country has a large informal workforce, combined with expensive and inadequate health infrastructure. Despite lockdowns, many Peruvians continued to venture out to shop and to work, and so transmission remained stubbornly high, says Camila Gianella Malca, a public-policy researcher at the Pontifical Catholic University of Peru in Lima.

Second wave

The impacts of lockdowns also differed from one pandemic wave to the next. By the time second waves emerged, so much had been learnt about the virus that people’s behaviour was quite different. By October 2020, population-wide masking policies had become common. Schools and other settings established physical-distancing measures to keep people apart, and people took more precautions when local transmission increased. Hospitals also learnt quickly how best to treat COVID-19: death rates after the first wave decreased by 20% owing to improved treatment alone.

Together, these changes dampened the extent to which countries benefited from lockdowns. For example, several studies found that school closures during the first wave reduced the spread of COVID-19. Yet Bhatt’s analysis¹² suggests that second-wave school closures had a much smaller effect. “We were surprised by that, to be honest,” he says.

Only a handful of countries continued to take a ’go hard, go fast’ approach after the first wave. Countries that had aimed for elimination — China, Australia, New Zealand and Vietnam, for example — saw that it worked and then went harder and faster, according to research by Anna Petherick, a public-policy researcher at the University of Oxford’s Blavatnik School of Government and her colleagues, who tracked government policies in more than 180 countries¹³. But in countries where belated first-wave lockdowns had merely lessened transmission, governments became less likely to take early action, tolerating higher case numbers in subsequent waves before ordering lockdowns (see ‘Thresholds for lockdown’).

Blunt tool?

Some researchers argue that countries could have avoided blunt all-of-society lockdowns, especially after the measures taken early in 2020. Among them is Mark Woolhouse, an infectious-disease epidemiologist at the University of Edinburgh, UK, who advised the Scottish government during the pandemic. He argues that it might have been possible to avoid the closing of schools and cooping-up of younger people — who were at lower risk of COVID-19 — while focusing efforts on protecting vulnerable and older people as soon as high-risk individuals and settings were identified. “This pandemic was crying out for [a] precision public-health response, because the risks associated with the public-health threat with the virus were so focused on a small minority, and the harms done by things like lockdown were not focused on the same people,” he says.

But many researchers have pushed back against the idea that a more targeted approach was ever possible. Klimek says that roughly one-third of the population in wealthy nations was vulnerable because of underlying health conditions, so targeted measures would have been difficult to implement. And the virus has caused not only deaths but also post-infection illnesses such as long COVID — which has emerged as a health burden even for people who had mild disease.

Another targeted option for governments considering how to reopen societies might have been to keep only high-risk locations closed — restaurants and bars, say, or even neighbourhoods with high population mobility, says Serina Chang at Stanford University in California, who worked with colleagues to identify such places using cellphone data¹⁴. But shutting down neighbourhoods would probably disproportionately affect socially disadvantaged communities. “Fairness is such an important question here,” she says.

Benefits versus harms

Woolhouse says there was scant effort to debate the scale of potential harms caused by lockdowns, meaning that policymakers were unable to weigh up costs and benefits properly. Indeed, early on, many countries adopted a ‘save lives at any cost’ approach, he says.

And lockdown policies did bring costs. Although they delayed outbreaks, saving lives by allowing countries to hang on for vaccines and drugs, they also brought significant social isolation and associated mental-health problems, rising rates of domestic violence and violence against women, cancelled medical appointments and disruption to education for children and university students. And they were often (although not always) accompanied by economic downturns.

But the common refrain that lockdowns involved a choice — saving lives versus livelihoods, or lives versus the economy — is a false dichotomy, says Stuart McDonald, an actuary and founder of the UK-based COVID-19 Actuaries Response Group, a community of specialists who have carried out regular analyses of mortality during the pandemic. If the UK government had not imposed belated lockdowns in 2020, hospital systems would have been overrun, death rates for all kinds of illnesses would have rocketed and economies and livelihoods would have collapsed anyway, he says. One analysis¹⁵ up to November 2021 estimated that the United States lost US$65.3 billion a month during lockdowns. But another¹⁶ estimated that US lockdowns from early March until the end of July 2020 added between $632.5 billion and $765 billion to the economy, compared with the alternative of no lockdowns. Unsurprisingly, the countries that did best in terms of saving lives and protecting the economy were those that acted fast with strict lockdowns.

What is more, some governments did at least try to consider various harms, McDonald says. In July 2020, for instance, McDonald attended a meeting of the UK government’s COVID-19 advisory group to discuss efforts to model the direct and indirect health impacts of lockdowns, measured by the preservation or loss of quality-adjusted life years — QALYs. (This measure gives more weight to younger lives than the lives of older people, who are judged to have lost fewer QALYs if they die.) Fewer deaths from road accidents were tallied as a benefit of lockdown, for instance; QALYs lost owing to delayed cancer diagnoses, or poorer health owing to loss of income, were harms. In August 2020, the report discussed at the meeting was publicly released¹⁷: it argued that QALYs lost would have been three times higher had there been no mitigation measures, such as lockdowns, in place. (McDonald was not involved in writing it, but contributed to subsequent versions of the report.)

Not all harms can be accounted for in this way. Loss of education because of school closures might indirectly harm children in the long run, potentially decreasing their future earnings and placing them at greater risk of poorer health outcomes. McDonald says that such harms are so far off — decades, in some cases — that they can’t readily be factored into a QALY ledger.

Value judgements

Pure economic analyses of whether lockdowns were worth it generally try to estimate the value of lives saved and compare that with the costs of economic downturns. But there is no consensus on how to make this comparison. Tweaks to the value placed on human life in these analyses can alter conclusions about whether lockdowns were worth it, found Lisa Robinson, a public-policy analyst at the Harvard T.H. Chan School of Public Health, and her colleagues¹⁸. If older lives are assigned a lower monetary value than younger ones, for instance, then — because COVID-19 disproportionately affected older people — lockdowns might be judged to have provided less benefit than if all lives are valued equally.

Jonathan Aldred, an economist at the University of Cambridge, UK, says that cost–benefit calculations of this kind are ill-suited to decision-making during an emergency, given the numerous sources of uncertainty. Putting monetary values on everything — from lives lost to the economic hit caused by shop closures — can give the impression that decisions about lockdowns are objective. But, says Aldred, the upshot is that “you’re hiding the fact that there are ethical judgements that have been made”. Policymakers should instead have a transparent discussion about the ethics of weighing costs and benefits, says Aldred, rather than suggest there is a ‘scientific’ answer. Without this kind of reckoning, we could be “back to square one” in a future pandemic, he says, with the same contentious debates about whether to close schools and at what harm to other sections of society.

The next pandemic

Now that COVID-19 vaccines and treatments for severe disease are widely available, most countries that have taken full advantage of them are unlikely to return to lockdowns. So what have researchers learnt that can inform decisions when another viral pandemic arrives?

One lesson that Klimek takes from lockdown studies is that there was an early window of opportunity when the virus could have been eliminated — as it was, in effect, in countries such as China, Australia and New Zealand. Had harsher measures been adopted sooner, and more widely, the pandemic might have played out very differently. “I think this is the big learning that we need to take away,” he says.

The paradox is that a successful early clampdown, or hard and fast action against a virus that turns out to be milder than initial indications suggest, could lead to complaints of overreaction.

A future threat might, of course, also spread in a completely different way from COVID-19. Ethical choices could look very different if the next pandemic is caused by an influenza virus that predominantly affects and is spread by young children.

Lockdowns hold another clear lesson: they exacerbate inequalities that already exist in society. Those already living in poverty and insecurity are hit hardest. Guarding against these unequal impacts requires improved health access and financial safeguards when times are good.

And transparency is key, too: the public needs to know more about how pandemic-control policies are decided, says Tsai. “That makes public-health policymaking seem less capricious,” he says, “because it’s reactive to both the science and values.”