When to use vaccine effectiveness studies

Abstract

Studies on the effectiveness of seasonal influenza vaccine can affect an individual’s perception of the ability of this vaccine to protect against influenza. However, vaccine effectiveness studies are designed to inform public health decisions rather than for individual decision-making. This overview explains what vaccine effectiveness means and why vaccine effectiveness estimates can vary. Individual variation in the response to seasonal influenza vaccine is based upon risk factors such as age, underlying health conditions, immune status and risk of infection and complications. Therefore, an individual’s decision to get vaccinated should be primarily informed by their risk of influenza illness and their risk of transmitting influenza to vulnerable people.

Introduction

Influenza is a vaccine-preventable disease that causes significant morbidity and mortality every year. Annual influenza epidemics in Canada result in approximately 12,200 hospitalizations^{Footnote 1} and 3,500 deaths^{Footnote 2}. To reduce this burden of illness, Canada’s National Advisory Committee on Immunization (NACI) recommends influenza vaccination every year for everyone six months and older who does not have contraindications to the vaccine, especially those at high risk of complications of influenza^{Footnote 3}.

Influenza viruses continually undergo genetic changes. Influenza vaccines are reformulated annually, but due to a lag between when the vaccine strains are decided on and when the vaccine becomes available on the market, antigens within the vaccine may no longer provide the desired protection against the viruses circulating in the community. Because these factors can affect the antigenic match between the vaccine and circulating influenza strains, influenza surveillance networks monitor how well the influenza vaccine is working during the current season each year.

In Canada and elsewhere, surveillance networks typically calculate their jurisdiction’s estimates of influenza vaccine effectiveness twice in a season – in the middle and again at end of the season. While routine annual estimation of vaccine effectiveness is a valuable public health tool, it does not directly translate into how well the vaccine may protect an individual against influenza. Nevertheless, an individual’s awareness of the effectiveness of the influenza vaccine in a given season can affect their perception of the protection offered by the vaccine and their decision to get vaccinated^{Footnote 4}.

Therefore, someone considering influenza vaccination may ask, “Will the vaccine protect me from getting influenza?” To help answer this question, this article provides a brief explanation of what vaccine effectiveness means as a measure of influenza vaccine performance and how it relates to individual-level decisions to vaccinate.

Efficacy versus effectiveness

Two distinct terms describe how well a vaccine performs: vaccine efficacy and vaccine effectiveness. These terms are often used interchangeably although what they each refer to is quite different. Both efficacy and effectiveness describe how well the vaccine works at protecting against influenza infection and resulting complications (e.g. hospitalization). Vaccine efficacy studies are conducted under optimal conditions, such as a highly controlled clinical trial. Vaccine effectiveness studies, the focus of this article, are conducted under “real world” conditions, such as in outpatient settings (e.g. a primary care clinic).

What does influenza vaccine effectiveness mean?

Influenza vaccine effectiveness is the relative benefit of vaccination in preventing influenza cases compared to no vaccination. In other words, influenza vaccine effectiveness equals the percentage of cases of influenza that could be prevented in a vaccinated group compared with an unvaccinated group. How the vaccine effectiveness estimate was generated, which takes into account the influenza strain and measured clinical outcome, is important in the interpretation of the estimate. When the Canadian Sentinel Practitioner Surveillance Network (SPSN) reported that the influenza vaccine had a vaccine effectiveness of 72% against influenza A(H1N1)pdm09 for the 2018–2019 season among individuals presenting to outpatient clinics with influenza-like illness, it means that the vaccinated individuals in the study were 72% less likely to be infected with medically attended influenza A(H1N1)pdm09 illness than unvaccinated individuals^{Footnote 5}.

It is also important to note that a vaccine effectiveness of 72% does not mean that a vaccinated individual has a 72% chance of not getting the clinical outcome measured in the study. Rather, it is the vaccinated group that is 72% less likely to get the outcome. To put this in discrete numbers, one needs to know that about 10% of unvaccinated adults are infected with influenza each season^{Footnote 6}. This means that out of a group of 100 unvaccinated adults, 10 would become infected. If an influenza vaccine has a vaccine effectiveness of 72%, out of 100 vaccinated adults only three adults, rather than 10, would become infected. In this scenario, the vaccine would prevent seven out of 10 (or approximately 72%) adults from being infected.

In scientific reporting, vaccine effectiveness estimates are often reported as adjusted values. This means that the estimate accounts for potential confounding factors such as age group, sex, race/ethnicity, study site and time from illness onset to study enrolment. Vaccine effectiveness estimates come with a confidence interval that provides information about the certainty of the estimate. Generally, the narrower the confidence interval, the more likely that the estimate is similar to the true vaccine effectiveness. If the confidence interval includes zero, the vaccine may provide no additional protection compared to no vaccination for that outcome, despite the vaccine effectiveness estimate being greater than zero.

Why do vaccine effectiveness estimates vary?

No single vaccine effectiveness estimate can sum up how well influenza vaccines work, even within a given influenza season, as each study’s vaccine effectiveness estimate is specific to the conditions of that study. The vaccine effectiveness will vary depending on a multitude of factors, including how closely related the vaccine virus strains are to the circulating viruses in a given influenza season, the population studied, when and where the study was conducted and differences in the methodology of studies assessing vaccine effectiveness (e.g. study design, sample size, influenza vaccines used, outcomes measured).

An example of the heterogeneity of vaccine effectiveness estimates is the SPSN’s seasonal influenza vaccine effectiveness point estimates from 2004–2005 to 2018–2019. These ranged widely, from 9% to 93%, against any type of influenza. They were similarly wide ranging for specific influenza strains^{Footnote 7}.

Should vaccine effectiveness estimates inform individual decision-making?

Influenza vaccine effectiveness studies are designed to estimate the relative benefits of influenza vaccination at a population level, not at an individual level. Population-level vaccine effectiveness estimates represent the protection offered by the vaccine in a study population of differing ages, underlying health conditions, influenza vaccines used and influenza viruses causing infection.

An individual’s risk of influenza depends not only on how well the influenza vaccine works, as estimated by vaccine effectiveness studies, but also by the individual’s risk of being exposed to influenza, their susceptibility to infection and their risk of complications from influenza. How well an individual responds to the vaccine depends on their age, underlying health conditions and immune system status. Therefore, the utility of vaccine effectiveness estimates best serves to inform public health policy decisions, such as signalling use of adjunct protective measures including antiviral drugs in a potentially low vaccine effectiveness season^{Footnote 8}, and guide vaccine virus strain selection for the future seasons^{Footnote 9}.

An individual deciding whether to get vaccinated should consider their risk of influenza-related complications. Pregnant women, children and adults with chronic health conditions, young children 6–59 months old, adults 65 years and older, people residing in nursing homes and other chronic care facilities and Indigenous peoples are at high risk of influenza-related complications and hospitalization. Individuals should also consider their capability of transmitting influenza to those at high risk, for example, care providers of those at high risk of influenza-related complications or hospitalization and their occupation, such as those who provide essential community services or who are in direct contact with poultry infected with avian influenza during culling operations. Further details on groups who are at increased risk of influenza-related complications and groups who can transmit influenza to those at high risk are detailed in the NACI Statement on Seasonal Influenza Vaccine for 2019–2020^{Footnote 3}.

Conclusion

Influenza vaccine effectiveness monitoring is an important population-level public health tool, but the findings are not designed to drive an individual’s decision whether to get vaccinated. An individual’s decision to get vaccinated should be primarily informed by their risk of influenza complications as well as their risk of transmitting influenza virus to vulnerable individuals^{Footnote 3}.

Authors’ statement

• LZ — Conceptualization, writing – original draft, writing – review and editing
• RS — Conceptualization, writing – review and editing
• KY — Conceptualization, writing – review and editing

Conflict of interest

None.

Acknowledgements

The authors would like to thank C Bancej, M Dubuc, J Guertin, L Whitmore and the peer reviewers for their valuable comments.

Funding

This work was supported by the Public Health Agency of Canada.

This work is licensed under a Creative Commons Attribution 4.0 International License