RR:C19 Peer Review Guidelines and Desired Content

I. Instructions for Peer Reviewers

We are aiming for reviews to be about 500 words (one page, double-spaced), but we will accept reviews of up to 1,000 words and reviews have the opportunity to upload supplemental materials (such as an annotated manuscript). We would like the focus to be on whether the overall conclusions and main claims should be taken seriously or not. Editorial suggestions (i.e., grammar, style, structure) are less important for this review. We anticipate that most of the meritorious preprints will eventually be published through a formal editing process.

II. RR:C19 Strength of Evidence Scale

We will ask reviewers to consider the following options for strength of evidence in the preprint and to choose one of the following that best represents their opinion of the manuscript. 

• Strong: The main study claims are very well-justified by the data and analytic methods used. There is little room for doubt that the study produced has very similar results and conclusions as compared with the hypothetical ideal study. The study’s main claims should be considered conclusive and actionable without reservation.

• Reliable: The main study claims are generally justified by its methods and data. The results and conclusions are likely to be similar to the hypothetical ideal study. There are some minor caveats or limitations, but they would/do not change the major claims of the study. The study provides sufficient strength of evidence on its own that its main claims should be considered actionable, with some room for future revision.

• Potentially informative: The main claims made are not strongly justified by the methods and data, but may yield some insight. The results and conclusions of the study may resemble those from the hypothetical ideal study, but there is substantial room for doubt. Decision-makers should consider this evidence only with a thorough understanding of its weaknesses, alongside other evidence and theory. Decision-makers should not consider this actionable, unless the weaknesses are clearly understood and there is other theory and evidence to further support it.

• Not informative: The flaws in the data and methods in this study are sufficiently serious that they do not substantially justify the claims made. It is not possible to say whether the results and conclusions would match that of the hypothetical ideal study. The study should not be considered as evidence by decision-makers.

• Misleading: Serious flaws and errors in the methods and data render the study conclusions misinformative. The results and conclusions of the ideal study are at least as likely to conclude the opposite of its results and conclusions than agree. Decision-makers should not consider this evidence in any decision.

  
  

III. The RR:C19 Strength of Evidence scale 

IV. Other areas to consider in your review:

We ask that reviewers consider a few key evaluation questions.

• Does the manuscript confirm previous work or refute the current understanding? Do the findings contribute to broader research understandings? Can the evidence and arguments presented support advancement of COVID-19 understanding within society? The degree of novelty can be considered here but is not the main driver of this indicator.

• How well does the manuscript position the work within the current literature/understanding? Does the manuscript cite current literature and discuss limitations? Is it steeped in reality with the potential to impact the implementation of policy and programs? Would you recommend this manuscript for publishing? 

• Is there clarity regarding the recommended actions that result from the findings? Is the work clearly and accurately presented. That is, is it well-structured and well-written, with an ability to speak to key audiences?

• Do authors pay attention to ethics, diversity, and inclusion? Have the authors adequately discussed ethical concerns? When appropriate, have they been inclusive and taken into account equity, rights, and diversity?

  
  

V. Should this preprint be published?

As a measure, we will ask whether the reviewer recommends that the author of the manuscript should be given the opportunity to publish with the RR:C19 Journal. Our aim is for very positively reviewed preprints to be given the option of having their manuscript published in RR:C19, if the authors have not been offered publication elsewhere. [Note: the reviews that will be published are a separate activity from the publishing of selected preprints!] With that in mind, reviewers are asked to make a recommendation on the preprint: 

• Accept: Accept manuscript without modification.

• Minor Revise: This implies intended acceptance, assuming the requested revisions are made appropriately, which are usually reviewed only by the Editor.

• Major Revise: This does not imply acceptance, but rather indicates that the revised manuscript will likely require re-review by the original Reviewers.

• Reject: Reject manuscript without recommendation for re-review