Vaccine hesitancy continues to rear its ugly head
Dr. Edsel Maurice T. Salvana
The columns from the past few weeks discussed vaccine platforms and vaccine efficacy. With the recent release of the FDA Emergency Use Authorization (EUA) for Sinovac, several questions cropped up. There is a lot of confusion as to why Sinovac is not being recommended for health care workers and seniors. These two groups are known to be at the highest risk for dying, and logically should receive the vaccine first. If Sinovac prevents severe disease, then it follows that these vulnerable groups should be vaccinated as soon as possible.
With vaccine hesitancy continuing to rear its ugly head, it is important to understand the process by which experts evaluate evidence. Phase 3 clinical trials remain the gold standard for evaluating efficacy and safety. Results, however, are not always straightforward. Clinical trial design, sample size, and choice of primary outcome will have a major effect on the interpretation of the results and applicability in the real world. The choice of study subjects is very important since this will affect the generalizability of results to the intended end user. For example, testing a vaccine on elderly Caucasian males may not yield results applicable to pediatric Asian females.
The recommendation against use in medical frontline workers, seniors, and those with comorbid conditions does not mean the vaccine will not work for these groups. It just means that the level of evidence that was presented may not have been enough to elicit enough confidence in the efficacy of the vaccine in these groups. Clinical trial results must meet prespecified numerical outcomes for the data to be considered strong enough to make a recommendation.
The recommendation not to use does not exclude the possibility that the vaccine will work. In fact, it probably will work. The studies that were done, however, showed some uncertainty in the overall effect. This is how science works. Data is often messy, and can only be evaluated within strict set limits. The best way to reach stronger conclusions is to continue to collect more data. Recommendations may change as more data emerges. More Sinovac trials are ongoing to answer these very questions. Clinical trials in children are just starting and so it cannot be used in this age group just yet. The same is true with those above 60 years old. It doesn’t mean these vaccines won’t work in children and the elderly, only that there isn’t enough data at this time. The World Health Organization (WHO) will decide on whether to grant an Emergency Use Listing for Sinovac. This may come as early as March, and will likely provide more clarity on the use of this vaccine.
Three clinical trials evaluated Sinovac’s efficacy. The Brazil trial looked at frontline healthcare workers 18 years and above. The Turkey trial looked at 15 to 59 year olds, 10 percent of whom were healthcare workers. The Indonesia trial had 15 to 59 year old subjects. There were important differences in the way these trials were conducted. Some tested for prior disease, and inclusion and exclusion criteria differed.
The methods and results of the three trials were very different. For example, the Brazil trial did not screen for previous Covid-19 infection among the healthcare workers. Antibody testing was not done in the Brazil subjects. This is important because some immunity is expected from previous Covid-19 infection. At the time of the study, Brazil had the fastest growing Covid-19 numbers.This means that many infections were possibly missed. If a significant number of subjects in the placebo group were already immune from previous infection, the vaccine effect would appear less than it really was. The difference in protection between the placebo and the vaccine group would be smaller, and the vaccine would not appear to be much better than placebo. The other two trials in Indonesia and Turkey did exclude patients with antibody evidence of previous infection, and this may have contributed to their better numbers.
Clinical trials measure effects of a medication or treatment. These effects are expressed as a point estimate, in this case as a percent efficacy.This point estimate is set within a confidence interval. The confidence interval is the range of numbers where the estimate would fall 95 percent of the time, if the study were done repeatedly. In a way, the width of the confidence interval is a measure of the certainty (or uncertainty) of the results of a study. Narrow confidence intervals mean a stronger probability that the point estimate is accurate.
Confidence intervals are affected by the size of the study population, and the strength of the effect of the intervention. Two clinical trials measuring the effect of the same drug can have very different confidence intervals. The following example illustrates this point. One clinical trial enrolls 1,000 people each in the treatment and placebo arms. Another clinical trial enrolls 10,000 people in each arm. Both studies show that the drug is 50 percent effective in curing the disease. The incidence or new cases in the population is 10 percent.
The smaller clinical trial has 100 people who get sick in the placebo group, and 50 people who get sick in the treatment group. A number called relative risk is calculated. Relative risk is the number of people who got sick in the treatment group divided by the number of people who got sick in the placebo group. In this case that number is 0.5, or 50 percent. The treatment reduces the risk of getting sick by 50 percent. The 95 percent confidence interval calculated by a statistical program for this group would be 0.326 to 0.681.
The larger clinical trial has 1,000 people who get sick in the placebo group, and 500 people who get sick in the treatment group. The relative risk is still 0.5, or 50 percent. The treatment in this larger clinical trial, just like in the smaller clinical trial, reduces the risk of getting sick by 50 percent. The 95 percent confidence interval for this group, however, would be much narrower at 0.423 to 0.530. This would be a more precise estimate of the true effect of the treatment.
So the same types of trials, but with more subjects, can produce narrower, more precise confidence intervals, even if both have exactly the same effect and both are statistically significant. This is why bigger trials are better in general. If there is a particularly strong effect, larger trials may not be necessary and the drug can already be used.
If the effect is smaller, the same trials will not be as precise. If the same examples above instead had a relative risk of 0.9, or the risk was reduced by only 10 percent, the calculations would be different. That scenario would have the smaller trial with 100 people who get sick in the placebo group, and 90 people who get sick in the treatment group. The 95 percent confidence interval would be 0.652 to 1.214. The larger trial would have 1,000 people who get sick in the placebo group, and 500 people who get sick in the treatment group. The 95 percent confidence interval is now 0.809 to 0.980.
With the weaker effect of this treatment or drug, the smaller trial becomes a lot less certain and loses statistical significance. That happens when the numbers of the confidence interval cross one. Though the smaller study shows the same reduction in risk of getting sick as the larger trail, the data from the smaller trial cannot be used to make a recommendation in favor of the drug or treatment. The bigger trial shows statistical significance and supports use of the drug, even with a smaller effect.
In summary, evidence used to support the use of a vaccine or drug are affected by the size of the trial population, and the magnitude or strength of the effect. The recommendations that experts make are affected by the precision of available data.
For Sinovac, the magnitude of the effect in the Brazil trial (50 percent) is less compared to the other trials (91 percent for Turkey, 65 percent for Indonesia) and so the degree of certainty of effect is less, especially since WHO recommends a cutoff of 50 percent. The 95 percent confidence interval for the Brazil study will include numbers below the 50 percent recommended cutoff, and so there is a chance a repeat trial will fall below threshold.
Since the Brazil trial was exclusively with healthcare workers, the evidence for healthcare workers is less certain due to the decreased magnitude of the effect despite the similar numbers of subjects enrolled. In addition, there were only a few patients above age 60 enrolled in the Brazil trial, so the subgroup analysis for the over 60 subjects would have a really wide, imprecise confidence interval.
The efficacy rates from publicly available data for Sinovac are as follows:
Vaccine efficacy for general population 15 to 59 years old (included 10 percent healthcare workers): 65 to 91 percent (mild disease as the primary outcome)
Vaccine efficacy for health care workers: 50.4 percent (mild disease as the primary outcome), 78 percent (moderate disease as a secondary outcome), 100 percent (severe disease as a secondary outcome).
Vaccine efficacy for >60 years old: unknown
Vaccine efficacy for children: unknown
The Philippine FDA has clarified that the vaccine can be given to healthcare workers, as long as the healthcare worker understands the limits of the data. The FDA stated that there may be better vaccines out there for frontline healthcare workers, but if there is no availability of an alternative, then Sinovac can be considered. Healthcare workers will still benefit from Sinovac, including those with direct exposure to Covid-19 patients in the workplace. Even if the protection from mild disease is modest, the protection from severe disease will prevent many healthcare workers from dying of Covid-19.
A third vaccine with an EUA is a good thing. It isn’t perfect, but it can prevent infections and deaths when used judiciously. The EUA recommendations may be modified when more data are available, but even now it is a very useful addition in the fight against Covid-19.