RR:C19 Evidence Scale rating by reviewer:
Reliable. The main study claims are generally justified by its methods and data. The results and conclusions are likely to be similar to the hypothetical ideal study. There are some minor caveats or limitations, but they would/do not change the major claims of the study. The study provides sufficient strength of evidence on its own that its main claims should be considered actionable, with some room for future revision.
Summary: This is interesting work that describes a novel, sensitive and quantitative device for objective testing of COVID-19 patients for olfactory dysfunction. This could be a useful new approach, although it has serious limitations, because the new device is not commercially available and it has not been compared with already existing devices.
The authors designed and tested a novel device for objective and quantitative measurements of olfactory deficits that can be used in the coronavirus pandemic to identify asymptomatic carriers of the virus. Many COVID-19 patients have olfactory dysfunction, but most studies rely on the patient’s subjective impressions rather than objective tests, and therefore miss a significant fraction of cases with subtle olfactory deficits. Although several previous studies have applied objective measures in COVID-19 patients, the new approach has some advantages and appears to produce interesting new results. The conclusions are generally supported by the data presented. However, there are some major deficiencies that need to be addressed for the final publication, including that age was not considered as a confounding variable, that the authors do not compare their novel device with those already in use and validated, and that they do not consider the extensive previous literature on this topic (objective measurements of olfactory deficits in COVID-19). The authors do not consider previous reports on alterations of threshold vs. intensity in odor detection, and they do not comment on the apparent heterogeneity of results in their patient cohort.
The strengths of this work are that it describes a novel quantitative approach, that it includes a relatively large number of odorants that are tested at different concentrations, that the testing has produced interesting results, and that the new device has been optimized for use in a pandemic.
Weaknesses of the work are that the authors did not provide age information. Age is a potentially confounding variable, and if the ages in the healthy control cohort are much younger than those in the COVID-19 cohort, then the reported differences may be misleading. Another major weakness is that the extensive literature on objective testing in COVID-19 was not considered, neither were (at least) 14 original studies (1-14), nor the two reviews on this topic (15, 16). The cohorts that have been tested are small; tests on larger cohorts are desirable. There are some discrepancies between data and interpretation: The data presentation appears to show that the intermediate or stronger odor concentrations are most suitable to identify olfactory dysfunctions in COVID-19, but the text says that lower concentrations and thresholds are more informative. A major limitation of the proposed device is that most hospitals and research centers will not have the technical expertise and technology to manufacture this type of apparatus; unless the device can be made commercially available for a reasonable cost, the use of the novel technology will be rather limited. There is no discussion of this aspect. A discussion of the interesting topic of deficits in threshold vs intensity of odor perception is desirable, and such a discussion should include recent relevant reports on this topic (17-22). There are no ethical concerns.
1. Chung TW, Sridhar S, Zhang AJ, et al. Olfactory Dysfunction in Coronavirus Disease 2019 Patients: Observational Cohort Study and Systematic Review. Open Forum Infect Dis. 2020;7(6):ofaa199. doi: 10.1093/ofid/ofaa199.
2. Qiu C, Cui C, Hautefort C, et al. Olfactory and Gustatory Dysfunction as an Early Identifier of COVID-19 in Adults and Children: An International Multicenter Study. Otolaryngol Head Neck Surg. 2020 Jun 16:194599820934376. doi: 10.1177/0194599820934376. Epub ahead of print.
3. Lechien JR, Saussez S, Cabaraux, et al. Psychophysical olfactory findings of mild-to-moderate COVID-19 patients: Preliminary report. MedRxiv [Preprint]. 2020 May 06. doi: 10.1101/2020.05.02.20070581.
4. Hintschich CA, Wenzel JJ, Hummel T, et al. Psychophysical tests reveal impaired olfaction but preserved gustation in COVID-19 patients. Int Forum Allergy Rhinol. 2020 Jul 1:10.1002/alr.22655. doi: 10.1002/alr.22655. Epub ahead of print.
5. Hornuss D, Lange B, Schröter N, et al. Anosmia in COVID-19 patients. Clin Microbiol Infect. 2020 May 22:S1198-743X(20)30294-9. doi: 10.1016/j.cmi.2020.05.017. Epub ahead of print.
6. Tsivgoulis G, Fragkou PC, Delides A, et al. Quantitative evaluation of olfactory dysfunction in hospitalized patients with Coronavirus  (COVID-19). J Neurol. 2020;267(8):2193-2195. doi: 10.1007/s00415-020-09935-9.
7. Moein ST, Hashemian SM, Mansourafshar B, et al. Smell dysfunction: a biomarker for COVID-19. Int Forum Allergy Rhinol. 2020;10(8):944-950. doi: 10.1002/alr.22587.
8. Moein ST, Hashemian SM, Tabarsi P, et al. Prevalence and reversibility of smell dysfunction measured psychophysically in a cohort of COVID-19 patients. Int Forum Allergy Rhinol. 2020 Aug 6:10.1002/alr.22680. doi: 10.1002/alr.22680. Epub ahead of print.
9. Petrocelli M, Ruggiero F, Baietti AM, et al. Remote psychophysical evaluation of olfactory and gustatory functions in early-stage coronavirus disease 2019 patients: the Bologna experience of 300 cases. J Laryngol Otol. 2020;134(7):571-576. doi: 10.1017/S0022215120001358.
10. Vaira LA, Deiana G, Fois AG, et al. Objective evaluation of anosmia and ageusia in COVID-19 patients: Single-center experience on 72 cases. Head Neck. 2020;42(6):1252-1258. doi: 10.1002/hed.26204.
11. Vaira LA, Salzano G, Petrocelli M, et al. Validation of a self-administered olfactory and gustatory test for the remotely evaluation of COVID-19 patients in home quarantine. Head Neck. 2020;42(7):1570-1576. doi: 10.1002/hed.26228.
12. Vaira LA, Hopkins C, Salzano G, et al. Olfactory and gustatory function impairment in COVID-19 patients: Italian objective multicenter-study. Head Neck. 2020;42(7):1560-1569. doi: 10.1002/hed.26269.
13. Lechien JR, Cabaraux P, Chiesa-Estomba CM, et al. Psychophysical Olfactory Tests and Detection of COVID-19 in Patients with Sudden Onset Olfactory Dysfunction: A Prospective Study. Ear Nose Throat J. 2020 May 29:145561320929169. doi: 10.1177/0145561320929169. Epub ahead of print.
14. Lechien JR, Cabaraux P, Chiesa-Estomba CM, et al. Psychophysical Olfactory Tests and Detection of COVID-19 in Patients with Sudden Onset Olfactory Dysfunction: A Prospective Study. Ear Nose Throat J. 2020 May 29:145561320929169. doi: 10.1177/0145561320929169. Epub ahead of print.
15. Hannum ME, Ramirez VA, Lipson SJ, et al. Objective sensory testing methods reveal a higher prevalence of olfactory loss in COVID-19 positive patients compared to subjective methods: A systematic review and meta-analysis. MedRxiv [Preprint]. 2020 Jul 06. doi: 10.1101/2020.07.04.20145870.
16. von Bartheld CS, Hagen MM, Butowt R. Prevalence of Chemosensory Dysfunction in COVID-19 Patients: A Systematic Review and Meta-analysis Reveals Significant Ethnic Differences. ACS Chem Neurosci. 2020;10.1021/acschemneuro.0c00460. doi:10.1021/acschemneuro.0c00460
17. Le Bon SD, Pisarski N, Verbeke J, et al. Psychophysical evaluation of chemosensory functions 5 weeks after olfactory loss due to COVID-19: a prospective cohort study on 72 patients. Eur Arch Otorhinolaryngol. 2020 Aug 4:1–8. doi: 10.1007/s00405-020-06267-2. Epub ahead of print.
18. Dell'Era V, Farri F, Garzaro G, et al. Smell and taste disorders during COVID-19 outbreak: Cross-sectional study on 355 patients. Head Neck. 2020 Jul;42(7):1591-1596. doi: 10.1002/hed.26288.
19. Iravani B, Arshamian A, Ravia A, et al. Relationship between odor intensity estimates and COVID-19 prevalence prediction in a Swedish population. Chem Senses. 2020 May 22:bjaa034. doi: 10.1093/chemse/bjaa034. Epub ahead of print.
20. Rodriguez S, Cao L, Rickenbacher GT, et al. Innate immune signaling in the olfactory epithelium reduces odorant receptor levels: modeling transient smell loss in COVID-19 patients. medRxiv [Preprint]. 2020 Jun 16:2020.06.14.20131128. doi: 10.1101/2020.06.14.20131128.
21. Vaira LA, Deiana G, Fois AG, et al. Objective evaluation of anosmia and ageusia in COVID-19 patients: Single-center experience on 72 cases. Head Neck. 2020 Jun;42(6):1252-1258. doi: 10.1002/hed.26204.
22. Walsh-Messinger J, Sahar K, Manis H, et al. 2020. Standardized Testing Demonstrates Altered Odor Detection Sensitivity and Hedonics in Asymptomatic College Students as SARS-CoV-2 Emerged Locally. Preprint medRxiv. doi: 10.1101/2020.06.17.20106302.