WHO Ad Hoc Expert Group on the Next Steps for Covid-19 Vaccine Evaluation
Recent announcements that some Covid-19 vaccines are estimated to have high short-term efficacy provide new hope that vaccination will soon contribute to controlling the pandemic. The initial roll-out of limited quantities of vaccines that are still investigational will provide the opportunity to ethically obtain pivotal data to improve regulatory and public health decision making, thereby increasing public and professional confidence in these and other vaccines.
After relatively short follow-up in phase 3 trials, even when vaccine efficacy appears to be high, reliable information will still be needed on longer-term safety and duration of protection. Other information gaps will include more comprehensive assessments of short-term safety, knowledge of whether waning of vaccine-induced protection may lead to vaccine-enhanced disease if a vaccinee becomes infected on exposure to SARS-CoV-2, information on protection against clinically severe forms of Covid-19, and knowledge of any associations between the degree of protection and the recipient’s age or coexisting conditions. Even after the first vaccines become available, it will still be important to evaluate additional vaccines to meet worldwide needs.1
On November 6, 2020, we, as participants in a World Health Organization (WHO) ad hoc consultation on the next steps for Covid-19 vaccine evaluation, discussed what critical additional data should be sought to inform regulatory and policy recommendations for the first successful vaccines and subsequent optimization. Consensus emerged that while it is still feasible and ethical, ongoing studies and others that are about to start should continue to collect high-quality information using directly randomized comparisons against placebo to address as many of the data requirements as possible. While vaccine supplies are limited, available vaccines are still investigational, or public health recommendations to use those vaccines have not been made, we believe it is ethically appropriate to continue blinded follow-up of placebo recipients in existing trials and to randomly assign new participants to vaccine or placebo. Moreover, under these conditions, we believe that trial sponsors are not ethically obligated to unblind treatment assignments for participants who desire to obtain a different investigational vaccine. People who enroll in clinical trials for altruistic reasons would probably understand the value of gathering data that will further elucidate the safety and efficacy of these vaccines and their appropriate use.
Conversely, there was concern that observational data obtained from nonrandomized studies after vaccine deployment could yield unreliable answers. Observational studies are subject to substantial biases and are much less amenable to unambiguous interpretation. Their limitations are of particular concern during this public health emergency, because vaccinated and unvaccinated people will differ in their risk of exposure to infection and of serious disease, partly because of fluctuating attack rates and because during early phases of vaccine deployment, vaccinees may well be at particular risk of infection. In these circumstances, even carefully analyzed observational studies can yield misleading answers about safety and efficacy.2,3 In addition, unrelated events that occur by chance after vaccination may be incorrectly attributed to the vaccine, and such anecdotes may be deliberately promulgated by groups opposed to vaccination.
Large, placebo-controlled, phase 3 efficacy trials could provide much of the needed information if they have appropriately prolonged follow-up while random assignments are still blinded. Such continuation would yield unbiased evidence on the duration of protection and on longer-term safety, including assessment of any evidence of the vaccine eventually enhancing the risk of severe disease (as was recently detected by continued follow-up of placebo recipients in dengue vaccine studies4). If there are hazards, they need to be identified; conversely, longer follow-up might reassuringly demonstrate continued protection with few or no adverse consequences, reducing vaccine hesitancy.
This opportunity to obtain reliable evidence about longer-term effects would be destroyed by early unblinding and immediate vaccination of participants assigned to placebo. Although each participant has the option to pursue any available intervention, if substantial numbers of participants choose not to do so, continuation of blinded follow-up in a population in which no licensed vaccine is being deployed could yield important and unexpected findings that would be difficult to obtain reliably any other way.
Thus, early deployment of scarce doses of still-investigational vaccines (under Emergency Use Listing [EUL] or similar regulatory mechanisms) could bring additional public health benefits if accompanied by firm commitments to maintaining blinded follow-up of participants in ongoing or future placebo-controlled trials until a licensed vaccine is fully deployed in the population. In some settings, early deployment could instead use the Expanded Access/Compassionate Use (EA/CU) mechanism, under which recipients are unambiguously informed of the vaccine’s investigational nature. For example, under trying conditions, the WHO deployed an Ebola vaccine under EA/CU during the recent outbreak in the Democratic Republic of Congo, ensuring that hundreds of thousands of prioritized persons received an investigational vaccine that was in limited supply.4
Because hundreds of millions of people in some priority groups will eventually be vaccinated against Covid-19, the world needs highly reliable evidence of vaccine safety that can be straightforwardly and convincingly explained to the public. Indeed, the ultimate impact of Covid-19 vaccines in a population may depend more on the prevalence of hesitancy or strong disinclination to receive a Covid-19 vaccine than on whether the vaccine has 95%, 80%, or 70% efficacy. Current phase 3 studies typically provide controlled data on about 20,000 vaccine recipients and 20,000 placebo recipients. Although these numbers should suffice for detecting relatively common adverse events, there is a risk of missing or exaggerating less common but clinically important events. Because large numbers of people will rapidly be vaccinated, vaccination will inevitably seem to be temporally associated with some uncommon adverse events. A large, simple trial5 to evaluate serious safety outcomes, in which many participants (even hundreds of thousands) are randomly assigned to vaccine or placebo and those who receive placebo are vaccinated only about 2 months later could identify any rare but serious short-term side effects or show that there were none. Such a trial could be conducted either during a period of emergency use or immediately after licensure and could be viewed as a fair way of allocating initially limited vaccine supplies.
What about vaccine candidates that do not become available for phase 3 study until after effective vaccines have already been deployed in some locations? Additional vaccines with worthwhile efficacy would still be desirable, especially if they could be readily deployed on a large scale or if safety concerns emerge with the first vaccines. For example, a 70% effective single-dose vaccine may be more valuable than a two-dose regimen with 90% efficacy and greater implementation challenges. It is noteworthy that such a vaccine could not be identified without using placebo controls. Participants in trials of such vaccines should have access to the standard of care in their location3 and, if the trial is successful, their communities should share in the benefit. Countries with limited or no access to a known effective vaccine could thus ethically permit placebo-controlled trials of vaccines of potential relevance to them even if effective vaccines were already being marketed elsewhere.
Randomized, placebo-controlled trials are the bedrock of modern clinical decision making and remain the most efficient way to obtain reliable results. If successful, focused attempts to ascertain correlates of protection could materially accelerate acceptance of second-generation vaccines but alone cannot provide an adequate basis for assessing safety and efficacy. Early clinical trial results may offer promise, but they cannot provide all the reliable data required. Randomized, noninferiority trials can provide clinically relevant data in some cases, but at a considerable cost to efficiency.
We can address important needs with continued follow-up of placebo recipients in phase 3 trials, use of placebo controls in large, simple safety trials, and clinical data from placebo-controlled, randomized trials evaluating new vaccines. A concerted global effort to collect such data while it’s still possible would increase the likelihood of reliably identifying multiple vaccines with favorable benefit–risk profiles. These studies would go far toward earning the broad public confidence required for widespread vaccine acceptance so that we can bring this pandemic to an end.