RR:C19 Evidence Scale rating by reviewer:
Potentially informative. The main claims made are not strongly justified by the methods and data, but may yield some insight. The results and conclusions of the study may resemble those from the hypothetical ideal study, but there is substantial room for doubt. Decision-makers should consider this evidence only with a thorough understanding of its weaknesses, alongside other evidence and theory. Decision-makers should not consider this actionable, unless the weaknesses are clearly understood and there is other theory and evidence to further support it.
***************************************
Review:
This manuscript by Wu and colleagues describes a large-scale wastewater sampling study that took place during the early stages of the COVID-19 pandemic in the United States. The group had previously published seminal work in the area and this paper is largely tied into the techniques pioneered by the same team using wastewater samples from multiple locations around the country. The work is certainly novel as there are few organizations that have worked across such as large geographic scale to monitor levels of SARS-CoV-2 in wastewater. There are several limitations noted by the author, which I feel are all accurate and can be addressed in some future work. I have made note of several minor comments for the authors to consider addressing.
Timing of study:
Why was the study only conducted for roughly 3 months? It is clear that the pandemic was still ongoing as of June 2, 2021, but there was no rationale as to why the data stopped at that particular point.
Characteristics of sites:
The characteristics of the individual wastewater treatment sites were not clearly explained in the methods or text. The differences from one site to another have a major effect on how data should be considered, and also informs the most appropriate normalization approach. The sites should be described.
Connection between WWTP and county-level data:
Additionally, there seems to be some oversimplification of how much each site reflects each county it may be situated in. Often, WWTP sites served multiple counties, in addition to counties having multiple WWTP sites serving their populations. This means that comparing viral case incidence and WW viral titers comes across as a big assumption, as we do not have any information in the text that tells us how much county coverage the WWTP sites have for each county. Something similar to this was done in a MedRxiv preprint by Melvin and colleagues.
Line 176, states that viral titers seemed to precede rise and fall in ICU admissions, but couldn’t this be determined mathematically? Using vague language when it appears so much data is at your disposal seems like a waste of an opportunity to answer a very important question.
Line 191, states that a linear relationship exists between total viral load and catchment size-normalized daily new cases. The author also states this is consistent with the hypothesis that average shedding rates are similar across catchment areas. 1) Does this linear relationship negate the idea of wastewater being predictive of daily caseload? 2) Where is the reference for this hypothesis mentioned in the text?, 3) Does the aforementioned similarity across catchment areas, in fact, mean to say catchment areas of similar size, or is this all catchment areas regardless of size?
Line 205, speaks on the capability of wastewater-based surveillance to detect SARS-CoV-2 at approximately 1/27,000. Wouldn’t this be a likely overestimate since no matter the county-level data is always larger ( although understated) than the viral load within the catchment area? So the catchment area would likely have a higher concentration of virus than an entire county in many cases, especially in rural locations. I think the data of disease incidence needs to be restricted to catchment area for detection sensitivity to be determined accurately.
Line 216, The author states that the results are consistent with a previous model that shows that viral detection increases with population size. What happens to the trend with large or small populations with unequal incidence rates?
In figure 2, it is not clearly stated what “Wx.Cx” is referring to. Could that be clearly explained in the text?
Line 227, Does “W0.C1” have a catchment area that is particularly small, and is there a measurable proportion that is rural?
Questions on methods:
Were wastewater sites told to take composite samples exclusively or were they left to sample any way they want?
What was the average transit time of samples and were they kept cold?
How long were filtered supernatants kept at 4 degrees before analysis?
Line 317, the reference to Deer Island plant appears out of nowhere. Why was this one site mentioned here?
How many sites are sampled per county?