Skip to main content
SearchLogin or Signup

Review 2: "Preprinting a pandemic: the role of preprints in the COVID-19 pandemic"

This study proves that COVID-19 has led to the unprecedented role of preprints and preprint servers in the dissemination of COVID-19 science. Findings are robust and informative, though there are some errors and misinterpretations.

Published onAug 11, 2020
Review 2: "Preprinting a pandemic: the role of preprints in the COVID-19 pandemic"
1 of 2
key-enterThis Pub is a Review of
Preprinting a pandemic: the role of preprints in the COVID-19 pandemic
Preprinting a pandemic: the role of preprints in the COVID-19 pandemic

Abstract The world continues to face an ongoing viral pandemic that presents a serious threat to human health. The virus underlying the COVID-19 disease, SARS-CoV-2, has caused over 3.2 million confirmed cases and 220,000 deaths between January and April 2020. Although the last pandemic of respiratory disease of viral origin swept the globe only a decade ago, the way science operates and responds to current events has experienced a paradigm shift in the interim. The scientific community has responded rapidly to the COVID-19 pandemic, releasing over 16,000 COVID-19 related scientific articles within 4 months of the first confirmed case, of which at least 6,000 were hosted by preprint servers. We focused our analysis on bioRxiv and medRxiv, two growing preprint servers for biomedical research, investigating the attributes of COVID-19 preprints, their access and usage rates, characteristics of their sharing on online platforms, and the relationship between preprints and their published articles. Our data provides evidence for increased scientific and public engagement (COVID-19 preprints are accessed and distributed at least 15 times more than non-COVID-19 preprints) and changes in journalistic practice with reference to preprints. We also find evidence for changes in preprinting and publishing behaviour: COVID-19 preprints are shorter, with fewer panels and tables, and reviewed faster. Our results highlight the unprecedented role of preprints and preprint servers in the dissemination of COVID-19 science, and the likely long-term impact of the pandemic on the scientific publishing landscape.

RR:C19 Evidence Scale rating by reviewer:

  • Potentially informative: The main claims made are not strongly justified by the methods and data, but may yield some insight. The results and conclusions of the study may resemble those from the hypothetical ideal study, but there is substantial room for doubt. Decision-makers should consider this evidence only with a thorough understanding of its weaknesses, alongside other evidence and theory. Decision-makers should not consider this actionable, unless the weaknesses are clearly understood and there is other theory and evidence to further support it.



This paper compares COVID-related and non-COVID-related preprints published on bioRxiv and medRxiv. Specifically, they calculate how often preprints are accessed, how the articles are shared on online platforms, and how preprints compare to their published articles. The data clearly demonstrates COVID-19 preprints are accessed, shared on social media, and referenced by journalists more often than non-COVID preprints. The study data, methods, and analysis do support the claim that: “Our results highlight the unprecedented role of preprints and preprint servers in the dissemination of COVID-19 science”. The authors also conclude that there is a “likely long-term impact of the pandemic on the scientific publishing landscape”, although there is not any specific data to support the second claim. The study is potentially informative.

Detailed Review

Introduction. The authors made a compelling case highlighting the gap between traditional publication processes and the urgent needs in disseminating scientific information regarding COVID. Current literature and context is well-described. The objective of this study is clearly stated.

Revision suggestion: Suggest removing the results and conclusion from the introduction(i.e., “We found that preprint servers... in this endeavour.”)

Results. Interesting and extensive analysis of the preprint repositories. It was particularly relevant to consider the issue of social media and misinformation as part of the analysis. The impact of the study could be enhanced by specifically investigating whether COVID-19 preprints address issues of health disparities or diversity, equity, and inclusion regarding study authors, study participants, or media headlines.

Revision suggestion: use sub-headings to organize content and help the readers navigate the many analyses.

Discussion. Thorough discussion. The main study finding is that preprints are playing a newly important role in science communication during the COVID-19 pandemic. This is well-supported by the research and clearly articulated in the discussion. However, the authors should specifically mention: what are the implications or recommendations to improve science communication that flow from this study finding? The paragraph about misinformation on twitter is interesting, but the authors could enhance the utility of the study by linking these results to the discussion about the speed at which new findings are shared, whether they are published, and how the preprint papers change by the time of publication.

Revision Suggestion: Two other study claims in the discussion are not well-supported by the data and methods. First, the discussion says “the pandemic has left what is likely to be a lasting imprint on the preprint and science publishing landscape”. None of the data speaks to changes on the publishing impact outside of the pandemic. They also conclude that rapid dissemination through preprints has not affected “quality of preprints that are subsequently published.” However, this study does not have data that speaks directly to the quality of the evidence.

Methods. The authors should be applauded for the major effort to build the study dataset from multiple information sources in order to comment on many aspects of the preprint lifecycle including eventual publication and mentions in social and traditional media. However, odds ratios are not the appropriate effect estimate to use. The risk / relative risk can be calculated instead and is preferred.

Figures & Tables.

Figure 1D. Would it be possible to align the timeline of panel D with panels A - C? Or enlarge the grey knots for dates.

Figure 2C & 2D. Spell out country name abbreviations in footnote; Or, if the goal is to show clustering by continent (2D), remove the country IDs and enlarge the knots.

1 of 1

No comments here