US officials knew their monitoring method might be missing some harmful effects of COVID vaccines, claims an investigative journalism article in a medical journal.
Rare heart inflammation side effects were identified overseas in early 2021 but the US monitoring failed to link COVID vaccination with this and other potential harms.
The article claims that US officials suppressed warnings from experts that their safety monitoring was not fit-for-purpose.
Associate Professor Helen Petousis-Harris, vaccinologist at the Faculty of Medical and Health Sciences, University of Auckland, comments:
“The BMJ investigation raises legitimate and concerning questions about how one part of US COVID-19 vaccine safety surveillance was managed. Transparency about these systems is vital, because confidence in vaccine safety should come from rigorous surveillance, not from protecting the surveillance system from inconvenient findings.
“The statistical problem at the centre of the investigation is actually quite simple. Systems such as VAERS look for unusual patterns among reports of health events following vaccination. One FDA method asked, in effect, whether a particular event was being reported unusually often for a particular vaccine compared with what would be expected from the reporting database as a whole.
“During the COVID rollout, however, the database became dominated by COVID vaccine reports. Pfizer and Moderna, therefore, contributed heavily to the background against which each was being assessed. If both vaccines increased reporting of the same event, that could raise the expected level as well as the observed level, making the difference harder to see. This is a recognised statistical problem called masking.
“Here is an analogy: Imagine trying to spot whether red cars have a specific brake defect by comparing them to all other cars in the repair shop. If suddenly almost every car brought to the shop is red, they are essentially being compared against each other, making a shared problem much harder to spot against the background noise. That is essentially the statistical problem here.
“If regulators understood that limitation and nevertheless discouraged scientists from testing alternative approaches, that deserves serious scrutiny.
“But a failure of a single signal-detection method is not evidence that the broader vaccine safety system collapsed, nor that hundreds of conditions reported to VAERS were caused by vaccination. VAERS is a passive early-warning system. Reports can identify patterns that need investigation, but by themselves they cannot tell us whether a vaccine caused an event or how often that event occurs.
“Myocarditis is a useful example. This particular FDA algorithm failed to flag the association. But myocarditis was nevertheless identified during 2021 through other analyses of VAERS and other vaccine-safety systems, investigated, and ultimately established as a rare adverse effect of mRNA vaccination.
“That distinction is critical. A statistical signal means ‘look at this carefully’; it does not mean ‘the vaccine caused this.’ Equally, the absence of a signal from one particular algorithm does not prove that no risk exists.
“Good pharmacovigilance requires both vigilance and discipline: we should design systems capable of finding bad news, investigate signals wherever they lead, and then use appropriate epidemiological studies to determine which signals represent genuine causal risks.”
Conflict of interest statement: Helen has led several local and international studies assessing the safety of COVID-19 vaccines.
Dr Michael Lydeamore, Department of Econometrics and Business Statistics, Monash University, comments:
“No statistical methodology is perfect, especially when new reports are coming in quickly, and in very large volumes. The fact that the mRNA COVID-19 vaccines were rolled out so quickly and in such great numbers means that some of the assumptions behind the methods we use to estimate risk don’t hold true. This is the case for both the PRR [proportional reporting ratios] and the Empirical Bayes method explored in the BMJ article.
“The issues about ‘masking’ in the methods used by the FDA are well known, and there are ways to mitigate these statistical effects, although it is not an easy process. Whether the FDA used these approaches or not, we can’t tell from the BMJ article. Of course, PRR is not protected from problems either: when reports flood in rapidly for a new vaccine, the PRR can overestimate risk by a large amount. This is because the PRR can only operate on reports of adverse events, not on the actual number of people who got the vaccine. If the number of people who got the COVID-19 vaccine is drastically different to the number of people being vaccinated for other reasons, then it is well known that PRR can overestimate the risk of adverse events.
“This is not to say the risk of an adverse event following a COVID-19 vaccine is zero. Research into rates of adverse events has been going on for years since the worst of the pandemic, and a systematic review in Nature Medicine has shown that the rate of vaccine-related adverse events is far lower than the same complications caused by the infection itself. Perhaps I can best summarise it as: it is important that those who ‘understand the statistics’ remember that limitations apply to all of our methods.”
Conflict of interest statement: Dr Lydeamore has not declared any conflicts of interest.
Dr Christopher Rudge, Lecturer in Health Law, Sydney Law School; and Deputy Director of Sydney Health Law, The University of Sydney, comments:
“VAERS – the US counterpart to Australia’s Database of Adverse Event Notifications – is an early-warning system. It identifies possible patterns, or ‘signals’, for closer scrutiny; it cannot establish that a vaccine caused an event.
“This externally peer-reviewed BMJ investigation draws on government emails released under US freedom-of-information laws and to US Senate investigators, together with interviews. It alleges that officials knew their chosen method for scanning VAERS could miss such signals, yet continued to treat the absence of an automated alert as reassuring.
“If substantiated, the allegations would not show that any reported condition was caused by vaccination. They would, however, raise serious questions about whether the absence of an automated alert was presented as a ‘true negative’ – that is, as reliable evidence that no safety signal existed – when the method could not support that conclusion.
“When governments make vaccination a condition of work or participation in public life, justification depends on safety monitoring that is fit for purpose, candid about its limits, and responsive when evidence changes. Public confidence depends on regulators being rigorous and transparent about the limits of their tools.”
Conflict of interest statement: Not yet received.