Skip to main content
SearchLogin or Signup

Review 3: "The infection fatality rate of COVID-19 inferred from seroprevalence data"

This study finds substantial heterogeneity in the infection fatality rate across different locations. Data are useful and add to the emerging picture on IFR, however, substantial conclusions cannot be drawn.

Published onSep 06, 2020
Review 3: "The infection fatality rate of COVID-19 inferred from seroprevalence data"
1 of 2
key-enterThis Pub is a Review of
The infection fatality rate of COVID-19 inferred from seroprevalence data
Description

Objective To estimate the infection fatality rate of coronavirus disease 2019 (COVID-19) from data of seroprevalence studies. Methods Population studies with sample size of at least 500 and published as peer-reviewed papers or preprints as of July 11, 2020 were retrieved from PubMed, preprint servers, and communications with experts. Studies on blood donors were included, but studies on healthcare workers were excluded. The studies were assessed for design features and seroprevalence estimates. Infection fatality rate was estimated from each study dividing the number of COVID-19 deaths at a relevant time point by the number of estimated people infected in each relevant region. Correction was also attempted accounting for the types of antibodies assessed. Secondarily, results from national studies were also examined from preliminary press releases and reports whenever a country had no other data presented in full papers of preprints. Results 36 studies (43 estimates) were identified with usable data to enter into calculations and another 7 preliminary national estimates were also considered for a total of 50 estimates. Seroprevalence estimates ranged from 0.222% to 47%. Infection fatality rates ranged from 0.00% to 1.63% and corrected values ranged from 0.00% to 1.31%. Across 32 different locations, the median infection fatality rate was 0.27% (corrected 0.24%). Most studies were done in pandemic epicenters with high death tolls. Median corrected IFR was 0.10% in locations with COVID-19 population mortality rate less than the global average (<73 deaths per million as of July 12, 2020), 0.27% in locations with 73-500 COVID-19 deaths per million, and 0.90% in locations exceeding 500 COVID-19 deaths per million. Among people <70 years old, infection fatality rates ranged from 0.00% to 0.57% with median of 0.05% across the different locations (corrected median of 0.04%). Conclusions The infection fatality rate of COVID-19 can vary substantially across different locations and this may reflect differences in population age structure and case-mix of infected and deceased patients as well as multiple other factors. Estimates of infection fatality rates inferred from seroprevalence studies tend to be much lower than original speculations made in the early days of the pandemic.

RR:C19 Evidence Scale rating by reviewer:

  • Misleading. Serious flaws and errors in the methods and data render the study conclusions misinformative. The results and conclusions of the ideal study are at least as likely to conclude the opposite of its results and conclusions than agree. Decision-makers should not consider this evidence in any decision.

***************************************

Review of Ioannidis, “The infection fatality rate of COVID-19 inferred from seroprevalence data”

This paper:

  • addresses an important topic, the estimation of the infection-to-fatality ratio (IFR) of COVID-19;

  • reports results of a systematic review that identified 50 rows of data on the seroprevalence of COVID-19 in different locations at different times;

  • links the serosurveys to COVID deaths counts and uses these as numerators and estimated infection counts as denominators to estimate the IFR for each location/time;

  • presents these IFRs as evidence in support of the author's hypothesis (1) that COVID-19 is not as dangerous as people might believe and that non-pharmaceutical interventions (NPIs) like state-mandated social distancing orders are not warranted.

In the production of scientific knowledge, the systematic review and meta-analysis is often considered the most reliable method, and described as the top of the pyramid of evidence-based medicine.(2) Conducting such a study is not easy, and efforts to do so are valuable and should be encouraged. Unfortunately, the method used here to estimate IFR is flawed due to systematic bias introduced by internal migration (as I shall describe in detail below). This bias produces misleading conclusions when used to argue that IFR is lower in locales with lower-than-average cumulative death rates. The discussion section is therefore also fatally flawed. Even if the sources of bias in the IFR estimates were somehow addressed, the author's thesis, that NPIs are unneeded, ignores the emerging evidence of the long-term non-fatal health burden caused by COVID-19.

This systematic review has been useful in identifying 50 rows of data on population seroprevalence, which is more than previous work (3) (but less than subsequent (4)), but because of its flawed methods and misleading conclusions, I must recommend rejecting it as unsalvageable.

Insurmountable methodological flaws

The method of IFR estimation employed in this paper uses an estimated infection count derived from a serosurveys together with the cumulative death count a week after the survey to approximate the IFR with the formula

IFR=cumulativedeathcount÷(seroprevalenceratepopulationcount)IFR = cumulative death count \div (seroprevalence rate*population count)

There are two major challenges in this approach: (1) misclassification bias and (2) migration bias. Both are most damaging when the cumulative infection count is low, which makes the estimates of IFR in locations with low COVID-19 burden differentially less accurate than the locations with high burden.

Like the author, I think of the IFR as a parameter about an individual: if a person selected uniformly at random is infected with the SARS-CoV-2 virus, what is the probability that they die from COVID-19? If we assume that all members of the population are equally likely to get infected, then this can be approximated accurately by the ratio of the cumulative death count to the cumulative infection count (however, as society learns who is most at risk of severe disease and acts to reduce infections in these vulnerable populations, this approximation will become less accurate).

Although not stated directly, an implicit assumption of the IFR estimation method in this paper is that

cumulativeinfectioncountseroprevalenceratepopulationcount.cumulative infection \approx countseroprevalencerate*population count.

This assumption may fail for at least two reasons. First, as acknowledged by the author, seroprevalence testing is not perfect, and even if a test has specificity of 99%, it will still yield a non-negligible number of false positives in locations with low cumulative infections. It is possible for serosurvey papers to adjust for the sensitivity and specificity of the test statistically, but, as the author acknowledged, the evidence base for such adjustment is still emerging, which means that adjusting for the sensitivity of tests on individuals who have had asymptomatic or mildly symptomatic infections more of an art than a science at this point.

There is a second, and perhaps less familiar, reason that the seroprevalence rate might be very different than the cumulative infection rate: out-migration from COVID hotspots. Even with a perfectly accurate test, the seroprevalence count for a population is not necessarily close to the cumulative infection count, because of population change. Rather, 

seroprevalencecountcumulativeinfectioncountseropositivedeaths+netmigrationofseropositives.seroprevalence count \approx cumulativeinfection count-seropositive deaths+net migration of seropositives.

If a substantial number of seropositive individuals have relocated from a COVID hotspot to a location with low seroprevalence, ignoring the net migration term will produce a substantial overestimate of cumulative infections. A hypothetical example (inspired by data from a New York Times analysis from May (5)) positing that 10% of residents of New York City have migrated out (including individuals who are seropositive and seronegative) and 1% of them relocated to Vermont illustrates this point:

Location

Cumulative

infection count

COVID

deaths

Net migration

of seropositives

New York City

6,000,000

30,000

-600,000

Vermont

12,000

60

6,000

The inaccuracy of estimating the cumulative infection count with seroprevalence count in NYC is not substantial: seroprevalence count = 6,000,000 – 30,000 – 600,000 = 5,370,000. The total error in a serosurvey here would likely be dominated by the sampling strategy.

In Vermont, on the other hand, an influx of seropositive people fleeing New York, even if it is only 1% of the total population leaving New York, would bias the seroprevalence count to be nearly 50% higher than the cumulative infection count: seroprevalence count = 12,000 - 60 + 6,000 = 17,940.  

Although it is possible to identify recent in-migrants during data collection in a serosurvey, it appears that this has not been done in the surveys identified in this systematic review. Without including some protection for migration bias, IFR estimates from locations with low seroprevalence must be considered inaccurate.

In conclusion

This important work must be rejected as unsalvageable. Future data collection in serosurveys should ask participants about their recent migration history, to permit developing more precise estimates of cumulative infection count.

In addition to the major concerns described above, I must comment that this paper does not follow the norms of systematic review. For example, it does not include a PRISMA checklist, it does not include a flow diagram, and the description of the search strategy appears to be incomplete and insufficient for replication.(6)

References

1. An Elite Group Of Scientists Tried To Warn Trump Against Lockdowns In March [Internet]. BuzzFeed News. [cited 2020 Aug 28]. Available from: https://www.buzzfeednews.com/article/stephaniemlee/ioannidis-trump-white-house-coronavirus-lockdowns

2. Murad MH, Asi N, Alsawas M, Alahdab F. New evidence pyramid. BMJ Evid-Based Med. 2016 Aug 1;21(4):125–7. 

3. A systematic review and meta-analysis of published research data on COVID-19 infection-fatality rates | medRxiv [Internet]. [cited 2020 Aug 28]. Available from: https://www.medrxiv.org/content/10.1101/2020.05.03.20089854v4

4. Arora RK, Joseph A, Wyk JV, Rocco S, Atmaja A, May E, et al. SeroTracker: a global SARS-CoV-2 seroprevalence dashboard. Lancet Infect Dis [Internet]. 2020 Aug 4 [cited 2020 Aug 28];0(0). Available from: https://www.thelancet.com/journals/laninf/article/PIIS1473-3099(20)30631-9/abstract

5. Quealy K. The Richest Neighborhoods Emptied Out Most as Coronavirus Hit New York City. The New York Times [Internet]. 2020 May 15 [cited 2020 Aug 28]; Available from: https://www.nytimes.com/interactive/2020/05/15/upshot/who-left-new-york-coronavirus.html

6. Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med. 2009 Jul 21;6(7):e1000097. 


Comments
0
comment

No comments here