Observed negative vaccine effectiveness could be the canary in the coal mine for biases in observational COVID-19 studies.

Since the emergence of the SARS-CoV-2 Omicron variant, multiple observational studies have reported negative vaccine effectiveness (VE) against infection, symptomatic infection, and even severity (hospitalization), potentially leading to an interpretation that vaccines were facilitating infection and disease. However, current observations of negative VE likely stem from the presence of various biases (e.g., exposure differences, testing differences). Although negative VE is more likely to arise when true biological efficacy is generally low and biases are large, positive VE measurements can also be subject to the same mechanisms of bias. In this perspective, we first outline the different mechanisms of bias that could lead to false-negative VE measurements and then discuss their ability to potentially influence other protection measurements. We conclude by discussing the use of suspected false-negative VE measurements as a signal to interrogate the estimates (quantitative bias analysis) and to discuss potential biases when communicating real-world immunity research.