Skip to main content
SearchLogin or Signup

Review 1: "Development of a Codebook of Online Anti-Vaccination Rhetoric to Manage COVID-19 Vaccine Misinformation"

Reviewers find this a well-executed effort to characterize the nature of vaccine hesitancy in online forums, while also noting that vaccine hesitancy arises through other channels, and that the study might be enriched by engaging other theoretical and empirical literature.

Published onJun 09, 2021
Review 1: "Development of a Codebook of Online Anti-Vaccination Rhetoric to Manage COVID-19 Vaccine Misinformation"

RR:C19 Evidence Scale rating by reviewer:

  • Reliable. The main study claims are generally justified by its methods and data. The results and conclusions are likely to be similar to the hypothetical ideal study. There are some minor caveats or limitations, but they would/do not change the major claims of the study. The study provides sufficient strength of evidence on its own that its main claims should be considered actionable, with some room for future revision.



This manuscript reports the results from a purposive content analysis of narratives, rhetorical styles, and tropes in online anti-vaccination stories related to COVID-19 vaccines. The goal is to identify the most common and compelling types of vaccine misinformation with the hope of better informing both science communicators and public health practitioners about ways to counter such messages. This descriptive effort to identify “common narratives” related to misinformation might help better inform those seeking to combat vaccine hesitancy.   The evidence involved a systematic identification of hashtags and keyword searches to identify different “channels” or sources of content that would be selected for coding/content analysis – focusing on English language media that met specific predetermined criteria. The initial round of coding was used for identification through software of a codebook that was applied to a second round of corpora with an inductive method and team of researchers. The results of this analysis led to the identification of the universe of narrative tropes and rhetorical strategies surrounding online anti-vaccination rhetoric. Importantly, the narrative tropes identified four major types of antagonists, or four “key common narrative tropes,” as well as four key common rhetorical strategies. 

The main study claims are generally justified by the methodology and data presented. The manuscript might connect this work directly with the voluminous body of research on how the media frames scientific misinformation and anti-vaccine “conspiracy rhetoric” (or “media frames”) present a threat to public health.  The large cross-disciplinary research on media framing effects as a “process” that begins as “frames emerge” and “frames are advocated” by actors who promote different types of “individual level framing effects” is relevant (e.g., Levin, Schneider and Gaeth 1998 – Typology of Framing). Also relevant is the related work on COVID and framing effects that promote vaccine hesitancy or resistance (e.g., COVID States Project – e.g., for a definition of “vaccine hesitancy,” “vaccine resistance,” and “uptake” as a an attitude that varies along a continuum for individuals – and also the importance of conspiracy rhetoric and other types of scientific misinformation as a determinant of these attitudes (as well as “motivation” and “partisan/ideological identity”) (e.g., Van Baval et al. 2021).

The manuscript is generally well-written and free from errors: I noticed one typo in the sentence on “an innocent put disempowered….” Change “put” to “but.”  The manuscript is informative, and the codebook of misinformation generates a better understanding of the most common, and therefore potentially “strongest,” types of ant-vaccination misinformation that people may be attending to that generates hesitancy/resistance.  The Tables at the end of the paper are nice but it might be better to have those referenced and discussed more in the text.  In the discussion section, the lengthy review of inoculation as an approach to combating misinformation might be reduced to include discussion of other potential antidotes – e.g., Levy, Bayes, et al. 2021.


No comments here