Recommendations on the Duration of the Post-vaccination Observation Period for Influenza Vaccination during the COVID-19 Pandemic

An Advisory Committee Statement (ACS)
National Advisory Committee on Immunization (NACI)

Preamble

The National Advisory Committee on Immunization (NACI) provides the Public Health Agency of Canada (hereafter referred to as PHAC) with ongoing and timely medical, scientific, and public health advice relating to immunization.

In addition to burden of disease and vaccine characteristics, PHAC has expanded the mandate of NACI to include the consideration of programmatic factors in developing evidence-based recommendations to facilitate timely decision-making for publicly funded vaccine programs at provincial and territorial levels.

The additional factors to be considered by NACI include: economics, ethics, equity, feasibility, and acceptability. Over the coming years NACI will be refining methodological approaches to include these factors. Not all NACI Statements will require in-depth analyses of all programmatic factors. As NACI works towards full implementation of the expanded mandate, select Statements will include varying degrees of programmatic analyses for public health programs.

PHAC acknowledges that the advice and recommendations set out in this statement are based upon the best current available scientific knowledge and is disseminating this document for information purposes. People administering the vaccine should also be aware of the contents of the relevant product monograph(s). Recommendations for use and other information set out herein may differ from that set out in the product monograph(s) of the Canadian manufacturer(s) of the vaccine(s). Manufacturer(s) have sought approval of the vaccine(s) and provided evidence as to its safety and efficacy only when it is used in accordance with the product monographs. NACI members and liaison members conduct themselves within the context of PHAC's Policy on Conflict of Interest, including yearly declaration of potential conflict of interest.

Download the alternative format
(PDF format, 41 pages)

Organization: Public Health Agency of Canada

Publication Date: 2020-10-15

Related topics

Table of contents

Summary of the Information Contained in this NACI Statement

The following highlights key information for immunization providers. Please refer to the remainder of this statement for details.

1. What

2. Who

3. How

  1. NACI recommends that the current post-vaccination observation period, as specified in the Canadian Immunization Guide Reference a, should be maintained for influenza vaccination settings that can adhere to appropriate public health and infection prevention and control measures Reference b to reduce SARS-CoV-2 transmission, particularly physical distancing (Strong NACI recommendation).
    • NACI concludes that there is fair evidence to maintain a 15-minute post-vaccination observation period during the COVID-19 pandemic for individuals with no known history of severe allergic reactions (including anaphylaxis) to any component of the influenza vaccine being considered for administration or any history of other immediate post-vaccination reactions (e.g., syncope with or without seizure) (Grade B Evidence).
  2. NACI recommends that a shorter post-vaccination observation period, between 5 to 15 minutes after influenza immunization, may be considered during the COVID-19 pandemic, but only during times when appropriate physical distancing in post-vaccination waiting areas cannot otherwise be maintained due to the volume of individuals seeking immunization, and only when specific conditions are metEndnotes * (Discretionary NACI recommendation).
    • NACI concludes that there is insufficient evidence from retrospective, passive adverse event following immunization (AEFI) case series reports to support a reduced post-vaccination observation period. However, other factors may be taken into consideration to support a reduced post-vaccination observation period, if necessary for public health and infection prevention and control purposes (Grade I Evidence).

    Endnotes *

    Endnotes *

    A shorter observation period may be considered only if the vaccine recipient meets the following conditions Reference c:

    • Past history of receipt of influenza vaccine and no known history of severe allergic reactions (including anaphylaxis) to any component of the influenza vaccine being considered for administration (note that novel technology vaccine recipients should be exempt from reduced post-observation period eligibility) Reference d.
    • No history of other immediate post-vaccination reactions (e.g., syncope with or without seizure) after receipt of any vaccines.
    • The vaccine recipient is accompanied by a parent/guardian (in the case of a child) or responsible adult who will act as a chaperone to monitor the vaccine recipient for a minimum of 15 minutes post-vaccination. In the case of two responsible adults, both can be vaccine recipients for the purposes of this criterion, if both agree to monitor the other post-vaccination.
    • The vaccine recipient will not be operating a motorized vehicle or self-propelled or motorized wheeled transportation (e.g., bicycle, skateboard, rollerblades, scooter), or machinery for a minimum of 15 minutes after vaccination.
    • The vaccine recipient and the parent/guardian or responsible adult chaperone are aware of when and how to seek post-vaccination advice and given instructions on what to do if assistance and medical services are required.
    • The vaccine recipient and the parent/guardian/responsible adult agree to remain in the post-vaccination waiting area for the post-vaccination observation period and to notify staff if the recipient feels or looks at all unwell before leaving. They should be informed that an individual exhibiting any symptom suggestive of an evolving AEFI at the end of the shortened post-observation period necessitates a longer period of observation in the clinic.

    Return to Reference * referrer

4. Why

I. Introduction

A cluster of cases of pneumonia of unknown origin was reported from Wuhan, Hubei Province, China in December 2019. These cases were determined to be due to a novel coronavirus (SARS-CoV-2) that causes a disease now referred to as coronavirus disease 2019 (COVID-19). The World Health Organization (WHO) characterized COVID-19 as a pandemic on March 11, 2020Footnote 1.

Canadian provinces and territories instituted a series of public health measures to limit SARS-CoV-2 transmission and to reduce the burden on the healthcare system, including the temporary deferral of non-essential medical services. Recognizing the need to continue routine immunization programs during the pandemic, the Public Health Agency of Canada (PHAC), with consultation from the National Advisory Committee on Immunization (NACI) and the Canadian Immunization Committee, released interim guidance on priority populations for immunization programs in Canada,Footnote 2 as well as guidance on strategies and modifications of usual immunization clinic processes for the upcoming influenza season to reduce the risk of SARS-CoV-2 transmission in a variety of immunization clinic settingsFootnote 3.

There are a variety of process modifications (administrative, environmental and engineering controls, and personal protective measures) that can be implemented in the vaccination clinic setting to prevent SARS-CoV-2 transmission, such as promoting physical distancing (at least two metres) among vaccine recipients, and staff and their co-workers. Another possible measure may be to reduce the post-vaccination observation period for rare, but serious AEFIs. Reducing the amount of time vaccine recipients spend in the vaccination clinic post-vaccination could reduce the opportunity for SARS-CoV-2 transmission from an unrecognized, infectious COVID-19 case. However, any consideration of a reduction in post-vaccination observation period must weigh the potential risk of delayed identification of an adverse event that may require immediate medical intervention against the potential benefits of reducing the risk of SARS-CoV-2 transmission and allowing more individuals to be vaccinated (increase vaccination clinic "throughput") in a given time period.

In Canada, it is recommended that vaccine recipients remain under observation for at least 15 minutes after vaccination, unless there is a specific concern about a possible vaccine allergy in which case a 30-minute post-vaccination observation period is considered a safer intervalFootnote 4. Guidelines for post-vaccination observation in other countries vary Footnote 5Footnote 6Footnote 7Footnote 8. In April 2020, the Australian Technical Advisory Group on Immunization (ATAGI) released a recommendation that the post-vaccination observation period could be reduced from 15 minutes to 5 minutes in immunization clinics where adequate physical distancing was not possible, as long as specific criteria were metFootnote 9. The ATAGI guidance was based on a targeted review of the literature and expert opinion.

Guidance objective

The objectives of this NACI statement are to review the evidence on the timing of onset of serious AEFIs (anaphylaxis, syncope with or without seizure). These data will be used to assess the potential impact of a reduction in the post-vaccination observation period on the risk of delayed identification of a serious adverse event that may require immediate medical intervention, balanced against the potential benefits of reducing SARS-CoV-2 transmission in influenza vaccination settings and allowing more individuals to be vaccinated in a given time period. The analysis will be used to provide guidance to provincial and territorial immunization programs and frontline clinicians on the duration of the post-vaccination observation period for influenza vaccination during the COVID-19 pandemic.

II. Methods

Rapid literature review

In preparation for influenza vaccination clinics in fall 2020, a rapid review of literature was conducted to determine the likelihood of missing serious AEFIs (anaphylaxis, syncope with or without seizure) with shorter post-vaccination observation periods. The rapid review's search strategy was developed in conjunction with a federal Reference Librarian and was used to search a single electronic database for studies on the timing of serious AEFIs in children and adults (all ages and populations).

Research question

What is the minimum observation period needed post-vaccination to observe serious AEFI?

P (population):
Individuals 6 months of age and older
I (intervention):
Observation period post-vaccination, any vaccine
C (comparison):
N/A
O (outcomes):
Time to onset of serious AEFI

The electronic database (EMBASE) was searched from inception to May 22, 2020 using the following search terms: seizure, syncope, anaphylaxis, adverse event, immunization, vaccination reaction, allergic reaction, and post-vaccination reaction. Searches were restricted to articles published in English and French. Two reviewers (NF, PDP) independently screened the titles and abstracts and full-text of eligible articles, but not in duplicate. The PRISMA Flow Diagram is shown in Appendix A. The complete search strategy is available upon request.

Studies were included if they met the following criteria:

Studies were excluded if they met one or more of the following criteria:

The eligible studies that were identified were divided amongst three reviewers (NF, PDP, GC). The reviewers independently extracted data from their assigned studies into evidence tables and appraised the risk of bias of the studies. The data extraction and risk of bias assessments were not done in duplicate for any of the eligible studies.

The extracted data were used to summarize key study characteristics and to analyze the studies for risk of bias, using a modified Institute of Health Economics Tool (IHE)Footnote 10, as all included studies were case reports or case series. A narrative synthesis of the extracted data was created to summarize the outcomes of interest (number of AEFIs, severity, time to onset, and incidence, if reported). As not all of the identified articles evaluated outcomes by severity, the number of serious reports identified were compiled from reports that were explicitly classified as serious, as well as from inference from stated sequelae of individual cases that would meet usual serious outcome definitions (e.g., death, life-threatening illness, hospitalization, prolonged hospitalization, permanent disability).

Canadian Adverse Events Following Immunization Surveillance System

The Canadian Adverse Events Following Immunization Surveillance System (CAEFISS) database was searched for AEFI reports of anaphylaxis, syncope, and afebrile seizure associated with a date of vaccine administration between 2014 and 2018. The Medical Dictionary for Regulatory Activities (MedDRA) terms used to code CAEFISS reports were used to search CAEFISS for reports for the three outcomes of interest. Reports of anaphylaxis had to meet the Brighton Collaboration definition for anaphylaxis Footnote 11 and the individual in the report managed as such. CAEFISS uses the World Health Organization definition of a serious AEFIFootnote 12: an event that is life threatening, results in hospitalization or a prolongation of hospitalization, persistent or significant disability or where the outcome is a birth defect or death. Reports that did not include a time to onset of symptoms were excluded.

The time to symptom onset for each AEFI was summarized by 5-minute intervals within the first 15 minutes post-vaccination, if available; and the time of onset that includes 25%, 50%, 75%, 95%, and 100% of all reports. These summary measures were analyzed by age groups: less than 2 years, 2 to less than 7 years, 7 to less than 18 years, 18 to less than 50 years, 50 to less than 65 years, 65 years and older, all ages, and by all reports versus serious AEFI.

Qualitative assessment of reducing the post-vaccination observation period

It is challenging to accurately quantify the impact of a reduction in the post-vaccination observation period on the risk for SARS-CoV-2 transmission in a vaccination setting. This is particularly the case because the clinic settings in Canada are diverse; community prevalence of SARS-CoV-2 varies between communities and fluctuates over time within communities; and because this measure would be one of several public health and infection prevention and control measures implemented to reduce the risk of transmission in these settings. Consequently, a qualitative assessment of this intervention was undertaken to weigh the potential risk of any potential delay in identifying serious AEFIs that may require immediate medical intervention against the potential benefit of reducing the risk of SARS-CoV-2 transmission and enhancing the ability to vaccinate more individuals during a given time period (increase vaccination clinic "throughput").

Recommendation Development

The draft statement was developed through the NACI Influenza Working Group and in consultation with the NACI Vaccine Safety Working Group. These three analyses, outlined above, were then used by NACI to draft recommendations on whether to reduce the duration of post-vaccination observation period in the context of the COVID-19 pandemic. NACI critically appraised the available evidence and approved the final recommendation on August 13, 2020.

III.Onset of adverse events following immunization

Twenty-two studies on time to onset of serious AEFI were identified through the rapid review. All studies were evaluated for risk of bias using a modified IHE tool and all were deemed to be at serious risk of bias, except two studies, which were deemed to be at unclear risk of biasFootnote 14Footnote 19. Both studies with unclear risk of bias described the incidence of anaphylaxis after vaccine administration using a restricted set of ICD-9 codes to search a health care database, a process deemed to be more sensitive at identifying anaphylaxis episodes than relying on reports submitted to passive AEFI reporting systems. McNeil et al.Footnote 19 used the same code set used by Bohlke et al.Footnote 14, but supplemented their search with allergy codes combined with epinephrine-dispensing codes. The majority of the included articles used data from passive surveillance systems, and had limitations such as underreporting, insufficient information on cases reported, lack of denominators and inconsistent case definitions. For measurement of whether the intervention of interest was described, articles were considered to be of low bias if they reported the vaccine name and manufacturer and whether this was the first or subsequent immunization dose, if relevant. A visual summary of the risk of bias assessments is provided in Figure 2.

The findings from the rapid review must be interpreted with caution in light of the findings of the risk of bias assessment for the articles identified in the rapid review and the inherent limitations of data obtained from retrospective case series studies that rely on passive AEFI surveillance systems. For example, the voluntary nature of reporting in most of these systems can lead to underreporting of AEFIs; the retrospective nature of these studies may make it difficult to obtain the necessary clinical data to verify outcomes and to determine that outcomes meet case definitions; the studies may also not have access to data on the sequelae for individuals in case reports, which may lead to an underestimation of the seriousness of some outcomes. There is often no access to denominator data (i.e., the number of vaccine doses distributed or ideally the number of doses administered) to estimate the incidence of outcomes. Also estimates of incidence based on relatively few case reports may be unreliable (i.e., relatively few additional case reports can significantly alter the calculated rate) and rates generated in heterogeneous populations may not be comparable.

III.1 Anaphylaxis

Rapid literature review

Time to symptom onset of anaphylaxis

There were 16 articles (12 retrospective case series Footnote 13 Footnote 14 Footnote 15 Footnote 16 Footnote 17 Footnote 18 Footnote 19 Footnote 20 Footnote 21 Footnote 22 Footnote 23 Footnote 24, 4 case reportsFootnote 25 Footnote 26 Footnote 27 Footnote 28) identified in the rapid review that provide data on the timing of reports of anaphylaxis following vaccine administration. The retrospective case series primarily use AEFI surveillance systems to examine reports of anaphylaxis in vaccine recipients. Across all of the identified articles, 981 reports of anaphylactic reactions were described, with 816 (83%) reports assessed against the Brighton Collaboration definition of anaphylaxis and the criteria for the three levels of diagnostic certainty (level 1: 413 reports; level 2: 372; level 3: 14; other/level not reported: 17). Of the 981 reports of anaphylaxis, 729 (74%) of the reports were classified as serious.

Overall, 873 (89%) of the anaphylaxis reports had some time to symptom onset recorded, of which the majority (492/873, 56%) occurred within 30 minutes of vaccination. The majority of the data (735/873, 84%) came from a single study using reports from the US Department of Health and Human Service's Vaccine Adverse Events Reporting System (VAERS) database, which categorized onset times into relatively broad time intervals (within 30 minutes, 30-119 minutes, 2-4 hours, 4-8 hours and 8-24 hours)Footnote 23.

There were 8 retrospective case series Footnote 13 Footnote 14 Footnote 15 Footnote 16 Footnote 17 Footnote 19 Footnote 22 Footnote 24 and 3 case reports Footnote 25 Footnote 26 Footnote 28 that identified 55 reports that provided descriptions of or actual times to anaphylaxis symptom onset in intervals within the first 15 minutes post-vaccination. Of these 55 reports, 35 (64%) occurred between 0 and 5 minutes post-vaccination, 4 (7%) "within minutes" or "within a few minutes," 9 (16%) between 5 and 10 minutes, 1 (2%) between 10 and 15 minutes, and 6 (11%) within 5 to 15 minutes.

Three of the retrospective case series studies Footnote 13 Footnote 20 Footnote 23 provided summary measures of the time to anaphylaxis symptom onset after vaccination. Baxter et al. evaluated the Australian Surveillance of Adverse Events following Vaccination in the Community (SAEFVIC) database for reports of anaphylaxis occurring within the first 60 minutes following immunization in preschool aged (<5 years) children. The study found all 12 identified anaphylaxis reports had a time of symptom onset of 0-40 minutes, with an average of 7.5 minutes and a median of 5 minutesFootnote 13. Su et al., using anaphylaxis case reports from VAERS from 1990-2016, found 828 case reports that occurred within 1 day post-vaccination. These reports had a median time of onset after vaccination of 20 minutes (ranging from <1 minute to 24 hours)Footnote 23. And finally Pahud et al., examining serious, non-fatal adverse events reported to VAERS following receipt of 2009 H1N1 vaccine in children (<18 years of age) from October 2009-January 2010 found 12 true allergic reactions (i.e., not just reports of anaphylaxis) occurred within 3 hours of vaccination with a median of 30 minutes (range: 5 minutes-3 hours)Footnote 20.

Estimates of anaphylaxis incidence based on literature

There were 6 retrospective case series Footnote 13 Footnote 14 Footnote 15 Footnote 16 Footnote 18 Footnote 23 that had access to the data on the number of vaccine doses either distributed or administered during the period of the study; these data were used to calculate either overall or vaccine-specific estimates of anaphylaxis incidence during the study period.

The incidence of anaphylaxis was fairly consistent across studies, although there was some variation. The variation could be due to the small number of anaphylaxis case reports and the different denominators used to calculate incidence (doses distributed or doses administered). McNeil et al. reported an overall incidence of anaphylaxis in children and adults of 1.31 per 1,000,000 doses administeredFootnote 18. Two other studies calculated the incidence of anaphylaxis in children <18 years of age; estimates ranged from 0.65 to 1.3 per 1,000,000 dosesFootnote 13 Footnote 14.

Four studies reported incidence estimates for specific vaccines that varied significantly by vaccine and by study (0.1 to 26 per 1,000,000 doses)Footnote 15 Footnote 16 Footnote 18 Footnote 23. The highest reported incidence was from a study based on anaphylaxis reports submitted to the New South Wales Health Immunization Unit for human papillomavirus (HPV) vaccineFootnote 15 and the lowest reported incidence was for influenza vaccine based on reports submitted to VAERSFootnote 23. Incidence of anaphylaxis estimates after administration of influenza vaccine was 0.1 to 1.8 per 1,000,000 doses (administered or distributed)Footnote 18 Footnote 23.

Canadian Adverse Events Following Immunization Surveillance System

Time to symptom onset of anaphylaxis

Of the 136 reports of anaphylaxis identified, 112 (82%) were classified as serious. Just over half of all the anaphylaxis reports occurred in individuals 7 to less than 18 years of age (n=30, 22%) and 18 to less than 50 years of age (n=46, 34%). There were fewer numbers of AEFI reports in each of the other age groups (range: 9-20 AEFI reports per age group).

For all ages combined, 25% of anaphylaxis reports had onset within 5 minutes of vaccination, increasing to 50% of reported cases by 15 minutes post-vaccination, with an overall median time to symptom onset of 15 minutes (range: 1 minute to 48 hours). The proportion of individuals with symptom onset within the first 15 minutes post-vaccination did not vary significantly by age grouping, with the exception of children less than 2 years of age (6 minutes vs 15 minutes for all ages) and individuals 50 to less than 65 years of age (45 minutes vs 15 minutes) (Table 1). When comparing the time to symptom onset for all anaphylaxis reports compared to reports classified as serious, similar trends were found. For all ages combined, 29% of case reports had onset of symptoms by 5 minutes post-vaccination, 46% by 10 minutes post-vaccination and 54% by 15 minutes post-vaccination. These proportions did not differ significantly by age grouping, with the exception of children less than 2 years of age (50% by 5 minutes, 67% by 15 minutes) and individuals 50 to less than 65 years of age (8% by 5 minutes, 31% by 15 minutes) (data not shown), reflecting the differences in median time to symptom onset in these age groups (Table 1).

Table 1. Onset times for anaphylaxis for all vaccines combined and administered from 2014-2018 by age group.
Age group (years) Number of cases (%) Percentage of case reports by time to symptom onset
25% of cases 50% of cases 75% of cases 95% of cases 100% of cases Median (range)
Less than 2 15 (11%) 3 minutes 6 minutes 40 minutes 3 hours 3 hours 6 minutes
(1 minute to 3 hours)
2 to less than 7 20 (15%) 5 minutes 13 minutes 32 minutes 25 hours 48 hours 13 minutes
(1 minute to 48 hours)
7 to less than 18 30 (22%) 5 minutes 15 minutes 30 minutes 60 minutes 2 hours 15 minutes
(1 minute to 2 hours)
18 to less than 50 46 (34%) 7 minutes 15 minutes 30 minutes 6 hours 19 hours 15 minutes
(1 minute to 19 hours)
50 to less than 65 16 (12%) 17 minutes 45 minutes 7 hours 48 hours 48 hours 45 minutes
(1 minute to 48 hours)
65 and older 9 (7%) 6 minutes 18 minutes 25 minutes 2 hours 2 hours 18 minutes
(5 minutes to 2 hours)
All Ages 136 5 minutes 15 minutes 31 minutes 7 hours 48 hours 15 minutes
(1 minute to 48 hours)

III.2 Syncope and Seizure

As seizures may occur secondary to syncopal episodes, the two outcomes are considered together in this section.

Rapid literature review

Time to symptom onset of syncope or seizure

There were 8 articles (7 retrospective case series Footnote 19 Footnote 21 Footnote 29 Footnote 30 Footnote 31 Footnote 32 Footnote 33, 1 case reportFootnote 34) identified in the rapid review that provide data on the outcomes of either syncope or seizure following vaccine administration. The retrospective case series studies all utilized AEFI surveillance systems to evaluate reports of syncope or seizure in vaccine recipients. In the 8 articles that identified reports of syncope or seizure, there were 1,180 reports of syncope alone (i.e., not associated with any reported seizure activity), 199 reports of seizures associated with syncopal episodes ("syncopal seizures"), and 16 reports of seizures alone (either afebrile or febrile) for a total of 1,395 reports. Of the 1,395 reports, 103 (7%) were considered serious (97 syncope alone, 1 syncopal seizure, and 5 febrile seizures).

Of the 1,180 reported cases of syncope alone, 453 (38%) had some indication of time to symptom onset. The range of reported onset times varied from within 5 minutes to 2 days; however, 340 (75%) of the syncopal case reports with a recorded time to symptom onset occurred within the first 15 minutes post-vaccination. Twenty-four (5%) of these reports were from four studies Footnote 19 Footnote 21 Footnote 31 Footnote 32 using retrospective case series data: 15 (63%) cases had onset from 0-5 minutes post-vaccination and 1 (4%) case with onset within 10-15 minutes. The time of onset of the remaining 8 cases were not provided in the same 5-minute time intervals: 4 (17%) occurred within 10 minutes of vaccination and 4 (17%) within 15 minutes of vaccination.

Among the studies identified, a fifth studyFootnote 29 also using retrospective case series data provided the largest number of syncopal case reports with symptom onset within the first 15 minutes post-vaccination. This study by Braun et al evaluated reports of syncope occurring within 12 hours of immunization submitted to VAERS and injury reports for syncope-related falls from the US National Vaccine Injury Compensation Program in individuals of any age from 1990-October 1995. Of the 571 reports that had a recorded time to symptom onset (81.9% of total reports), 323 (63.2%) occurred within 5 minutes of vaccination, increasing to 416 (81.4%) within 10 minutes, and 454 (88.8%) within 15 minutes of vaccination. Of the 454 syncopal episodes, 153 were associated with "tonic or clonic movements" ("syncopal seizures"): 138 (30. 4%) within 15 minutes of vaccination and 15 (12.8%) greater than 15 minutes after vaccination. Of note, 67 (9.6%) cases reported subsequent hospitalization. Six individuals sustained head injuries after syncope-induced falls, which included skull fractures, cerebral contusions, cerebral hematomata and one case of major cerebral hemorrhage. Three of the injuries necessitated neurosurgery to alleviate cerebral swelling or bleeding and 2 individuals still had significant neurologic deficits up to 2 years after the incidents. All 6 of these head injuries occurred from syncopal events within 15 minutes after vaccination.

In the study by Sutherland et al., there were 10 secondary injuries recorded among the 26 serious syncopal reports: head injuries after syncopal falls (n=9), including one death of a 15-year-old boy due to intracranial hemorrhage, and a motor vehicle incident after a loss of consciousness while driving (n=1). Seven (70%) of the injuries occurred within 15 minutes of vaccinationFootnote 32.

A description of the time to symptom onset was available for 154 (77%) of the 199 reports of seizures associated with syncopal episodes ("syncopal seizures") and for all 16 (100%) reports of seizures alone. Two articles provided the data on syncopal seizure following vaccine administration Footnote 29 Footnote 31. Braun et al.Footnote 29 reported 153 syncopal reports with associated "tonic or clonic movements": 138 (90.2%) occurred within 15 minutes of vaccination and 15 (9.8%) occurred greater than 15 minutes after vaccination. The remaining case which was reported in the second study occurred at 0 minutes post-vaccination (i.e., immediately)Footnote 31. The 16 cases of seizure alone were identified in two studiesFootnote 30 Footnote 19. Crawford et al analyzed detailed clinical information on SAEFVIC case reports of syncope and seizures after quadrivalent HPV vaccination in females 12-26 years of age vaccinated as part of the Australian National HPV Vaccination Program from May 2007-April 2009.Footnote 30 The study identified 31 episodes of syncope with associated seizures, but the time to seizure onset was reported only for 3 reports of afebrile seizure, all in individuals with confirmed underlying epilepsy. There were no deaths reported in the study population, but there were 7 injuries reported: head injuries (n=5), mouth bleeding (n=1) and a T5/T6 vertebral fracture (n=1). The study by Milstien et al.Footnote 19 identified 13 reports that met the US Food and Drug Administration (FDA) case definition for a potentially vaccine-associated convulsion, four afebrile convulsions and nine febrile convulsions. The time of onset for the afebrile convulsions was 30 minutes, 2 hours, 24 hours and 2 days after vaccination.

Estimates of syncope and/or seizure incidence based on literature

There were 3 retrospective case series studies that had access to the data on the number of vaccine doses either distributed or administered during the period of the study, allowing for calculation of either overall or vaccine-specific syncope or seizure incidence during the study periodFootnote 30 Footnote 31 Footnote 32.

In an analysis of SAEFVIC case reports of syncope and seizures after quadrivalent HPV vaccination as part of the Australian National HPV Vaccination Program, Crawford et al. estimated an incidence of 7.8 and 2.6 per 100,000 doses of vaccine distributed for syncope and syncopal seizures respectivelyFootnote 30.Subelj et al. estimated the incidence of syncope and seizures associated with syncope to be 13.4 and 3.4 per 100,000 doses, respectively, based on 4 years of data from mandatory reports to the National Institute of Public Health of a school-based four-valent HPV (HPV4) vaccine program targeting girls 11-14 years of age in SloveniaFootnote 31. Sutherland et al evaluated reports of "syncope" or "syncope vasovagal" occurring in individuals ≥5 years of age on the same day as vaccination submitted to the VAERS surveillance system from January 1, 2005-July 31, 2007 and compared the rates of syncopal events to the rate of VAERS reports received from 2002-2004Footnote 32. The rates of syncopal events per 1,000,000 doses of vaccines distributed in the US during the respective study periods was increased in 2005-2006 (0.31-0.54) compared to 2002-2004 (0.28-0.35), as were the proportion of reports from females (77.5% vs 61.6%) and persons 11-18 years of age (62.0% vs 47.3%).

Canadian Adverse Events Following Immunization Surveillance System

Time to symptom onset of syncope or seizure

Of the 52 reports of syncope, 20 (38%) were classified as serious. Individuals 7 to less than 18 years of age (n=33, 65%) and 18 to less than 50 years of age (n=8, 16%) accounted for the majority of all 52 reports. There were very few AEFI reports in each of the remaining age groups (N is between 2 and 4 for each).

Table 2. Onset times for syncope for all vaccines combined and administered between 2014-2018

Age group (years)
Number of cases (%) Percentage of case reports by time to symptom onset
25% of cases 50% of cases 75% of cases 95% of cases 100% of cases Median
(range)
Less than 2 2 (4%) 5 days 12 days 18 days 18 days 18 days 12 days
(5 to 18 days)
2 to less than 7 2 (4%) 4 minutes 7 minutes 10 minutes 10 minutes 10 minutes 7 minutes
(4 to 10 minutes)
7 to less than 18 33 (65%) 1 minute 2 minutes 6 minutes 20 minutes 35 minutes 2 minutes
(1 to 35 minutes)
18 to less than 50 8 (16%) 2 minutes 63 minutes 48 hours 29 days 29 days 63 minutes
(1 minute to 29 days)
50 to less than 65 2 (4%) 1 minute 31 minutes 60 minutes 60 minutes 60 minutes 31 minutes
(1 to 60 minutes)
65 and older 4 (8%) 2 hours 11 hours 33 hours 48 hours 48 hours 11 hours
(1 minute to 48 hours)
All Ages 52 1 minute 3 minutes 15 minutes 5 days 29 days 3 minutes
(1 minute to 29 days)

For all ages combined, 25% of syncope reports had onset within 1 minute of vaccination, increasing to 50% of reported cases by 3 minutes post-vaccination and 75% within 15 minutes, with an overall median time to symptom onset of 3 minutes (range: 1 minute to 29 days) (Table 2). The proportions were similar for syncope reports classified as serious (data not shown).

There were 61 reports of afebrile seizure between 2014 and 2018, 50 (82%) of which were classified as serious. The majority (74%) of reports were in individuals less than 2 years of age. There were very few AEFI reports in the other age groups. For all ages combined, the median time to onset of afebrile seizure was 21 hours (range: 1 minute to 29 days). Only for the group of individuals 7 to less than 18 years were a significant proportion (25%) of afebrile seizures reported to have occurred within 2 minutes of vaccination, based on a small number of case reports (n=8).

III.3 Risk-benefit assessment

The rationale for potentially considering a reduction in the post-vaccination observation period in influenza vaccination settings during the COVID-19 pandemic is to reduce the duration of time vaccine recipients spend together with others in post-vaccination areas and to potentially reduce the number of contacts recipients encounter in vaccination settings. This could then reduce the opportunity for SARS-CoV-2 transmission from an unrecognized, infectious COVID-19 case and allow more individuals to be vaccinated (increase vaccination clinic "throughput") in a given time period. However, it is very challenging to quantify the impact of a reduction in the post-vaccination observation period on the risk for SARS-CoV-2 transmission, independent of other public health and infection prevention and control measures (engineering, environmental and administrative controls, and personal protective equipment) that are likely to be implemented in influenza vaccination settings during the COVID-19 pandemic. In addition, there will be differences in the risk of transmission in various settings due to variability in disease prevalence at different locations and at different points in time. Any consideration of a reduction in post-vaccination observation period would be a deviation from the current standard practice in Canadian provinces and territories. The current standard of a 15-minute post-vaccination observation period is intended to identify serious AEFIs that may require immediate medical intervention. This must be weighed against the potential benefits of reducing risk of SARS-CoV-2 transmission and allowing more individuals to be vaccinated (increase vaccination clinic "throughput") in a given time period.

Potential benefits of a reduced post-vaccination observation period

Potential risks of a reduced post-vaccination observation period

IV. Recommendations

Following a thorough review of the evidence summarized above, NACI has made two recommendations for public health program decision-making. These recommendations are based on currently available scientific evidence and expert opinion and are relevant only during the COVID-19 pandemic, after which the current guidance for the post-vaccination observation period as identified in Part 2 of the Canadian Immunization Guide should be followedFootnote 4.

In considering these recommendations and for the purposes of publicly funded program implementation, provinces and territories may take into account economic factors and other local operational factors (e.g. current immunization programs, resources). Recognizing that there are differences in operational contexts across Canada, jurisdictions are advised to refer to the local epidemiology of COVID-19 and additional PHAC guidanceFootnote 3 to determine the relative merits of modifying the post-vaccination observation period. NACI will continue to monitor the scientific literature for developments related to post-vaccination observation period and will update recommendations as evidence evolves, if required.

Please note:

Refer to Table 3 for a more detailed explanation of strength of NACI recommendations and grade of the body of evidence.

IV.1 Recommendations for Public Health Program Level Decision-Making

  1. NACI recommends that the current post-vaccination observation period, as specified in the Canadian Immunization GuideEndnote a, should be maintained for influenza vaccination settings that can adhere to appropriate public health and infection prevention and control measures Endnote bto reduce SARS-CoV-2 transmission, particularly physical distancing (Strong NACI recommendation).
    • NACI concludes that there is fair evidence to maintain a 15-minute post-vaccination observation period during the COVID-19 pandemic for individuals with no known history of severe allergic reactions (including anaphylaxis) to any component of the influenza vaccine being considered for administration or any history of other immediate post-vaccination reactions (e.g., syncope with or without seizure) (Grade B Evidence).

    Evidence and Rationale

    • A 15-minute post-vaccination observation period is the Canadian standard for individuals with no known history of prior severe allergic reaction or any other immediate post-vaccination reactions.
    • A vaccination clinic setting that is able to maintain recommended public health and infection prevention and control measures, including physical distancing in post-vaccination areas, is unlikely to achieve significant additional reductions in the risk of SARS-CoV-2 transmission by implementing a shorter post-vaccination observation period.
  2. NACI recommends that a shorter post-vaccination observation period, between 5 to 15 minutes after influenza immunization, may be considered during the COVID-19 pandemic, but only during times when appropriate physical distancing in post-vaccination waiting areas cannot otherwise be maintained due to the volume of individuals seeking immunization, and only when specific conditions are met Endnotes * (Discretionary NACI recommendation).
    • NACI concludes that there is insufficient evidence from retrospective, passive adverse event following immunization (AEFI) case series reports to support a reduced post-vaccination observation period. However, other factors may be taken into consideration to support a reduced post-vaccination observation period, if necessary for public health and infection prevention and control purposes (Grade I Evidence).

    Endnotes *

    Endnotes *

    A shorter observation period may be considered only if the vaccine recipient meets the following conditions Reference c:

    • Past history of receipt of influenza vaccine and no known history of severe allergic reactions (including anaphylaxis) to any component of the influenza vaccine being considered for administration (note that novel technology vaccine recipients should be exempt from reduced post-observation period eligibility)Reference d.
    • No history of other immediate post-vaccination reactions (e.g., syncope with or without seizure) after receipt of any vaccines.
    • The vaccine recipient is accompanied by a parent/guardian (in the case of a child) or responsible adult who will act as chaperone to monitor the vaccine recipient for a minimum of 15 minutes post-vaccination. In the case of two responsible adults, both can be vaccine recipients for the purposes of this criterion, if both agree to monitor the other post-vaccination.
    • The vaccine recipient will not be operating a motorized vehicle or self-propelled or motorized wheeled transportation (e.g., bicycle, skateboard, rollerblades, scooter), or machinery for a minimum of 15 minutes after vaccination.
    • The vaccine recipient and the parent/guardian or responsible adult chaperone are aware of when and how to seek post-vaccination advice and given instructions on what to do if assistance and medical services are required.
    • The vaccine recipient and the parent/guardian/responsible adult agree to remain in the post-vaccination waiting area for the reduced post-vaccination observation period and to notify staff if the recipient feels or looks at all unwell before leaving the clinic. They should be informed that an individual exhibiting any symptom suggestive of an evolving AEFI at the end of the shortened post-observation period necessitates a longer period of observation in the clinic.

    Return to endnote * referrer

Evidence and Rationale

Tables and figures

Table 3. NACI Recommendations: Strength of Recommendation and Grade of Evidence
STRENGTH OF NACI RECOMMENDATION GRADE OF EVIDENCE
Based on factors not isolated to strength of evidence (e.g. public health need) Based on assessment of the body of evidence

Strong

"should/should not be offered"

  • Known/Anticipated advantages outweigh known/anticipated disadvantages ("should"),

    OR Known/Anticipated disadvantages outweigh known/anticipated advantages ("should not")

  • Implication: A strong recommendation applies to most populations/individuals and should be followed unless a clear and compelling rationale for an alternative approach is present

A - good evidence to recommend
B - fair evidence to recommend
C - conflicting evidence, however other factors may influence decision-making
D - fair evidence to recommend against
E - good evidence to recommend against
I - insufficient evidence (in quality or quantity), however other factors may influence decision-making

Discretionary

"may be considered"

  • Known/Anticipated advantages closely balanced with known/anticipated disadvantages, OR uncertainty in the evidence of advantages and disadvantages exists
  • Implication: A discretionary recommendation may be considered for some populations/individuals in some circumstances. Alternative approaches may be reasonable
A - good evidence to recommend
B - fair evidence to recommend
C - conflicting evidence, however other factors may influence decision-making
D - fair evidence to recommend against
E - good evidence to recommend against
I - insufficient evidence (in quality or quantity), however other factors may influence decision-making
Table 4. Ranking Individual Studies: Levels of Evidence Based on Research Design
Level Description
I Evidence from randomized controlled trial(s).
II-1 Evidence from controlled trial(s) without randomization.
II-2 Evidence from cohort or case-control analytic studies, preferably from more than one centre or research group using clinical outcome measures of vaccine efficacy.
II-3 Evidence obtained from multiple time series with or without the intervention. Dramatic results in uncontrolled experiments (such as the results of the introduction of penicillin treatment in the 1940s) could also be regarded as this type of evidence.
III Opinions of respected authorities, based on clinical experience, descriptive studies and case reports, or reports of expert committees.
Table 5. Ranking Individual Studies: Quality (internal validity) Rating of Evidence
Quality Rating Description
Good A study (including meta-analyses or systematic reviews) that meets all design- specific criteriaTable 5 tablenote * well.
Fair A study (including meta-analyses or systematic reviews) that does not meet (or it is not clear that it meets) at least one design-specific criterionTable 5 tablenote * but has no known "fatal flaw".
Poor A study (including meta-analyses or systematic reviews) that has at least one design-specificTable 5 tablenote * "fatal flaw", or an accumulation of lesser flaws to the extent that the results of the study are not deemed able to inform recommendations.

Table 5 Tablenotes

Table 5 tablenote *

General design specific criteria are outlined in Harris et al. (2001).Footnote 24.

Return to Table 5 tablenote * referrer

Table 6. Summary of evidence on the safety of post-vaccination observation time
Study Vaccine Study Design Participants Summary of Key Findings Level of Evidence Quality
Case Series

Baxter et al., 2018

Any vaccine

Case Series

Australia

SAEFVIC passive surveillance system

July 2007 to June 2015

Children less than 5 years of age

Anaphylaxis

The incidence of anaphylaxis was found to be 0.13 per 100,000 doses of vaccine. The time interval between vaccine dose and onset of anaphylaxis ranged from 0-40 minutes (n=12), with an average of 7.5 minutes and a median of 5 minutes. Ten cases (83%) developed symptoms of anaphylaxis within 5 minutes, two cases (17%) had delayed symptom onset with one at 20 minutes and another at 40 minutes following vaccine administration. For the majority of cases (68%), more than one vaccine was given prior to the onset of anaphylaxis.

III

Poor

Bohlke et al., 2003

Any vaccine

Case Series

US

1991 to 1997

Children 17 years of age or younger registered in one of 4 Health Maintenance Organizations participating in the US CDC Vaccine Safety Datalink project

Anaphylaxis

5 cases of potential vaccine-associated anaphylaxis were identified after administration of 7,644,049 vaccine doses. Incidence of anaphylaxis was 0.65 cases per 1,000,000 doses (95% CI: 0.21-1.53). None of the episodes resulted in death. Among the 5 cases identified, one (20%) happened within 5-10 minutes of vaccine administration and the other 4 (80%) cases happen more than 1 hour following administration. Of the 5 cases, only one case of anaphylaxis reaction happened following a single vaccine administration (MMR vaccine).

III

Poor

Braun et al., 1997

Any vaccine

Case Series

US

VAERS passive surveillance system

1990 to 1995

All children and adults (no age restriction)

Syncope

A total of 697 syncopal episodes were reported that occurred within 12 hours of vaccine administration. Hospitalization was reported in 9.6% of cases. Of the 571 (73.3%) syncope events with known time, 511 (89.5%) occurred 1 hour or less after vaccination. Of these, 323 (63.2%) occurred within 5 minutes or less, 93 (18.2%) occurred within 10 minutes and 38 (7.4%) occurred within 15 minutes of vaccine administration. Six patients suffered skull fractures, cerebral bleeding, or cerebral contusion after falls; 3 of these patients required neurosurgery. All the falls occurred 15 minutes or less after vaccination, in or near the clinic or office.

III

Poor

Brotherton et al., 2008

HPV4 vaccine

Case Series

Australia

2007

Female adults and children 12-26 years of age that received HPV4 vaccine through Australia National HPV Vaccination Program in either the school-based or primary care clinic setting

Anaphylaxis

12 cases of suspected anaphylaxis were identified. Of the 11 cases that provided consent to be included in the study, 8 (73%) cases fulfilled the Brighton case definition for anaphylaxis (Level 1; n=1 and Level 2; n=7). All cases were reported to have an onset within 15 minutes of receiving vaccination; 4 (50%) cases were within 5 minutes and 3 (37.5%) cases occurred between 5-10 minutes of vaccine administration.

III

Poor

Cheng et al., 2015

Any vaccine

Case Series

Australia

SAEFVIC passive surveillance system

May 2007 to May 2013

Children less than18 years of age

Anaphylaxis

25 cases were identified that met Brighton Collaboration criteria for anaphylaxis. Of these cases, 9 (26%) met level 1 diagnosis certainty, 15 (60%) level 2 and one level 3 (4%). The majority of cases had rapid symptom onset with 13 (52%) cases happening within 5 minutes, 18 (72%) cases within the first 15 minutes and 20 (80%) cases during the first 30 minutes of vaccination. Overall, 20% (5/25) of cases had their first symptoms of anaphylaxis developed ≥30 minutes after immunization.

III

Poor

Crawford et al., 2011

HPV4 vaccine

Case Series

Female adult and children 12-26 years of age who received HPV4 vaccine through Australia National HPV Vaccination Program

Syncope and Seizure

Overall incidence following HPV4 vaccination was 7.8 per 100,000 doses distributed for syncope and 2.6 per 100,000 for syncopal seizures.

Of the 94 syncopal episodes, 67% (63/94) had syncope alone, and 33% (31/94) had associated seizure activity, of which 23% (7/31) had urinary incontinence. The HPV4 vaccine was given alone in 85% (82/97) of reports, with concomitant vaccines including: hepatitis B (6); diphtheria-tetanus-acellular pertussis (6); varicella (1); and varicella and hepatitis B vaccine (2).

Three patients were identified with afebrile seizures without syncope and all had a confirmed underlying epilepsy disorder. One patient had a generalized seizure 4 hours after the HPV4 vaccine (dose 2). Another had an exacerbation of complex partial seizures 4 hours after the HPV4 vaccine (dose 2). Another patient experienced a generalized tonic-clonic seizure 2 days after receiving HPV4 vaccine (dose 1). This resulted in a wedge fracture of spinal vertebrae, which was treated conservatively. A generalized epilepsy disorder was confirmed, and anticonvulsant medication commenced.

III

Poor

Johann-Liang et al., 2011

Any vaccine

Case Series

US

Vaccine Injury Comensation Program

Jan 2000 to Dec 2009

Adults and children of all ages

Anaphylaxis

53 unique cases alleging "anaphylaxis or anaphylactic shock" were identified through the Vaccine Injury Compensation Program, accounting for 3% of the total of 1,819 non-autism claims for the study period. Of those, 9 (17%) were defined as anaphylaxis. One case occurred from minutes to hour, 2 cases were within 5 minutes, one case had onset at 15 minutes, 2 cases were between 20 and 30 minutes and 3 cases occurred more than one hour after vaccine administration (one at 1 hour, one at 2 hours and one at 2-3 hours).

III

Poor

McNeil et al., 2016

Any vaccine

Case Series

US

Jan 2009 to Dec 2011

Adults and children of all ages enrolled in health plans at one of 9 Vaccine Safety Datalink participating sites

Anaphylaxis

76 cases of chart-confirmed anaphylaxis were identified that met Brighton Collaboration levels 1 and 2 criteria. Of these, 33 anaphylaxis cases [Brighton Collaboration level 1 (n=12; 36%) and level 2 (n=21; 64%)] were associated with vaccination and 43 were attributed to other causes. 29 cases (87.9%) of anaphylaxis had documented time to onset: 8 cases (24%) occurred within 30 minutes, 8 (24%) between 30 and 120 minutes, 10 (30%) from 2 to 4 hours and 3 cases (10%) occurred from 4 to 20 hours following vaccine administration. Only one case had specific reaction time which was an immediate reaction after multiple vaccine administration.

III

Poor

Milstien et al., 1987

Hib vaccine

Case Series

US

Apr 1985 to May 1986

Children 18 to 23 months of age at high risk of Hib and children 2 to 5 years of age who are not at high risk

Anaphylaxis

Two cases of anaphylactic-like reactions were reported. One case was in a 3-year-old boy who became pale and hypotensive and began to wheeze five minutes after vaccination. The other case was in a 4-year-old boy who became nauseated, pale, and bradycardia; circumoral cyanosis developed 20 minutes after vaccination. Both cases responded quickly to epinephrine and oxygen.

Syncope

Seven reports of syncope were identified, three that noted treatment with epinephrine and/or Benadryl. All but three episodes occurred within ten minutes of vaccination (others occurred at 30 minutes, 2 hours, and 24 hours). All episodes were in children 3 to 5 years of age.

Seizure

13 patients whose reactions met the case definitions for seizures were identified: four afebrile convulsions and nine febrile convulsions. Five patients with febrile convulsions were hospitalized. All reactions occurred more than 2 hours after vaccine administration, except one afebrile convulsion that occurred within 30 minutes. The mean time to onset of febrile convulsions after receipt of the vaccine was 24 hours (median: 12 hours). Two of the children had a prior history of febrile convulsions, and an additional three had a history of febrile convulsions in a sibling or a parent.

III

Poor

Pahud et al., 2013

2009 H1N1 monovalent influenza vaccine

Case Series

US

VAERS passive surveillance system

Oct 2009 to Jan 2010

Children less than 18 years of age

Anaphylaxis

3928 reports and 214 (5.4%) were classified as having serious, nonfatal condition. 109 cases were referred for further review of which 99 cases had complete clinical information. Of the 99 cases, they found fifteen presumed allergic reactions. The reported diagnoses included anaphylaxis (n=5), hypersensitivity reaction (n=3), angioedema (n=3), hives (n=3) and allergic reaction (n=1). True anaphylaxis caused by a vaccine was present in 4 cases, urticaria or other skin manifestations of a true allergic reaction in 6 cases and allergic reaction to a vaccine with respiratory symptoms in 2 cases. Two cases were considered anxiety reactions and 1 was considered likely attributable to an independent viral infection. All anaphylaxis cases or true allergic reactions (n = 12) occurred within 3 hours of vaccination (median, 30 minutes; range, 5 minutes-3 hours).

III

Poor

Patja et al., 2000

MMR vaccine

Case Series

Finland

Passive surveillance system

1982 to 1996

Adults and children of all ages

Anaphylaxis

30 suspected cases of anaphylaxis were identified, all of which appeared within 20 minutes of vaccination, except one case who developed symptoms several hours after vaccination.

Seizure

52 vaccinees reported febrile seizures 12 hours to 15 days after vaccination. Apart from 3 children who were 3 to 6 years of age, all reports were in children less than 3 years of age. Undefined seizures were observed in four children 2 to 12 days post-vaccination.

III

Poor

Schumacher et al., 2010

Any vaccine

Case Series

Switzerland

Passive surveillance system

1991 to 2001

Adults and children of all ages

Anaphylaxis

There were 18 cases of non-fatal anaphylaxis reaction reported. Two occurred within minutes after immunization, five within 6 hours, four within 6-24 hours, and seven after an unknown time interval. Of the 7 (3.6%) serious AEFI that were assessed as very likely or certainly related to immunization, there were 5 cases of anaphylaxis reaction with time to symptom onset ranging from 1 minute to 1 hour post-vaccine administration. One case happened several minutes after DTP-based combination vaccine, one case after 1 minute after DTP-based combination vaccine, one case after 5 minutes of DTP-based combination, one case after 1h of DTP-based combination vaccine and one case following MMR vaccination without information on timing of onset.

III

Poor

Su et al., 2019

Any vaccine

Case Series

US

VAERS passive surveillance system

Jan 1990 to Dec 2016

Adults and children of all ages

Anaphylaxis

828 reports that either met the Brighton case definition or included a diagnosis of anaphylaxis by a physician described symptoms within 24 hours of receiving the vaccine. Of reports with time to onset of symptoms available, 77% described symptoms less than 2 hours after vaccination with a median time to onset after vaccination of 20 minutes (range; <1 minute to 24 hours). Moreover, 402 (49%) of the reports had time to onset of symptoms < 30 minutes after vaccination. Finally, the authors identified 8 reports of death. Of 7 reports with time to onset symptoms available, 3 (43%) cases happened within 5 minutes and 4 (57%) cases within 15 minutes following vaccination. The other report of death happened at 20 min, 258 min or on the same day of vaccine administration.

III

Poor

Subelj et al., 2016

HPV4 vaccine

Case Series

Slovenia

Sept 2009 to Aug 2013

Girls 11 to 14 years of age who received HPV4 vaccine as part of a school-based vaccination program

Syncope
8 syncope events, accounting for 3.8% of all AEFIs reported, were identified. Incidence was 13.4 per 100,000 HPV4 vaccine doses distributed. Two syncope episodes were deemed serious. One event had an immediate onset, and another occurred 5 minutes following vaccine administration.

Seizure

2 seizure events, accounting for 0.9% of all AEFIs reported, were identified. Incidence was 3.4 per 100,000 HPV4 vaccine doses distributed. One seizure was deemed serious and occurred immediately following vaccine administration.

III

Poor

Sutherland et al., 2008

Any vaccine

Case Series

US

VAERS passive surveillance system

Jan 2002 to Dec 2004 and Jan 2005 to July 2007

Adults and children 5 years of age or older

Syncope

26 (5.6%) of the 463 post-vaccination syncope reports during 2005-2007 were coded as serious, which was not substantially different from the 20 (9.9%) serious reports during 2002-2004. Among the 23 patients for whom times of vaccination and syncope onset were indicated, 12 (52.2%) occurred within 5 minutes of vaccination, and 16 (69.6%) occurred within 15 minutes.

III

Poor

Woo et al., 2005

Any vaccine

Case Series

US

VAERS passive surveillance system

2000 to 2005

Adults and children of all ages who experienced syncope/presyncope and accidental injury on the day of vaccination

Syncope

107 reports of syncope/presyncope and unintentional injury on the day of vaccination were identified. 100 of the reported injuries occurred within 20 minutes of vaccination. There were three reports of serious head injury, including one that was deemed fatal due to vasovagal syncope.

Following the third dose of hepatitis B vaccine, a 15-year-old boy with no antecedent of event or medical problems, experienced vasovagal syncope several minutes after vaccination. He fell backward and hit his head, momentarily lost consciousness, then complained of pain in the chest and arms. Then, he reportedly had convulsions and went into cardiopulmonary arrest.

A 21- year-old man experienced intracerebral bleeding after falling backwards onto the floor 3 minutes after hepatitis B vaccine, tetanus toxoid, and diphtheria vaccine. He reportedly recovered without sequelae. An 18-year-old woman was in a fatal motor vehicle collision after she "passed out while driving her car" approximately 7 hours after meningococcal polysaccharide vaccine; the 7-hour interval between vaccination and the collision makes vaccine-related vasovagal syncope unlikely.

III

Poor

Zafack et al., 2019

Any vaccine

Case Series

Passive surveillance system database

Jan 1998 to Dec 2016

5,600 children and adults of all ages receiving additional doses of vaccine(s) previously temporally associated with an AEFI

Anaphylaxis

Of the 5,600 individuals identified who had experienced an AEFI, 18 patients had an anaphylaxis reaction, of which 14 (77.8%) had information regarding delay of onset following immunization. Among the 18 patients with reported anaphylaxis, 3 met the Brighton Collaboration level 1 of diagnostic certainty, 8 met level 2, 6 met level 3 and one did not have a description of signs and symptoms. Among the 14 patients with timing of symptom onset, 4 (28.6%) occurred by 5 minutes, 9 (64.3%) by 10 minutes and 10 (71.4%) patients had symptom onset within 30 minutes of vaccine administration.

III

Poor

Case reports

Poddighe et al., 2014

HPV2 vaccine

Physician case report

Funding:
N/A

A 14-year old girl

Syncope

A 14-year-old girl reported an AE after the second dose of bivalent HPV vaccine. The patient experienced sudden onset of general malaise and other symptoms after the vaccine administration. She fainted around 60 minutes after intramuscular injection. The patient did not have any previous physical and/or psychiatric diseases or complaints. Upon further evaluation in the following days, the patient was diagnosed with a neuropsychiatric syndrome.

III

Poor

Stone et al., 2019

MMR, Varicella, and DTaP/ IPV vaccine

Physician case report

Funding:
N/A

A 5-year-old male with a history of alpha-gal allergy

Anaphylaxis

Five minutes after receiving the vaccines, 5 year-old developed shortness of breath, wheezing, disseminated urticaria, and angioedema of the face and oropharynx, prompting an emergency room visit where he received epinephrine, diphenhydramine, prednisone and famotidine with relief of symptoms within ten minutes. He had no history of egg, latex, dairy, or gelatin allergies and had uneventfully received all prior childhood vaccinations.

III

Poor

Stone et al., 2017

Live-attenuated herpes zoster vaccine

Physician case report

Funding:
N/A

A 71-year-old woman with documented history of allergy to red meat

Anaphylaxis

A 71-year-old women with documented allergies to red meat required emergency department treatment and epinephrine administration upon receipt of live attenuated herpes zoster virus vaccine containing the Oka VZV strain. The vaccine was administered in a local pharmacy and within minutes she had a sensation of mental clouding progressing to lightheadedness, wheezing, and throat tightness.

She self-administered 50 mg diphenhydramine five minutes after symptom onset. Thirty minutes after her vaccination, she sought emergency care at which point she was documented to be dyspneic, flushed, with facial, oral, and uvular angioedema and bilateral conjunctival infections with stable vital signs and blood pressure, without documented wheezing on pulmonary examination. Her symptoms resolved within 20-30 minutes. She was later diagnosed with alpha-gal and the event was attributed to the presence of mammalian products within the vaccine.

III

Poor

Turktas et al., 1999

Whole-cell DTP vaccine

Physician case report

Funding:
N/A

A 6-month-old infant

Anaphylaxis

A 6-month-old infant experienced anaphylaxis following the third dose of the whole-cell DTP vaccine. The infant experienced drowsiness, followed by loss of consciousness, an urticarial rash throughout the body and swelling of the eyes. He was brought into a community hospital 20 minutes after vaccine administration and was then referred to the Department of Pediatrics within 2 hours of vaccination. During follow-up, the patient had mild monthly wheezing and cough episodes in winter, and perennial rhinitis throughout the year.

III

Poor

Worm et al., 2000

Tick-borne encephalitis vaccine

Case report

Funding:
N/A

A 29-year-old woman

Anaphylaxis

Patient developed a generalized urticaria, dyspnea and hypotension a few minutes after the third immunization. She immediately received antihistamines and steroids intravenously and the symptoms resolved completely within 1 hour.

III

Poor

Abbreviations: AE: adverse events; AEFI: adverse event following immunization; DTP: Diphtheria, tetanus, pertussis; HPV: human papillomavirus; HPV4 vaccine: 4-valent human papillomavirus vaccine; MedDRA: Medical Dictionary for Regulatory Activities; MMR: Measles-mumps-rubella; SAEFVIC: Surveillance of Adverse Events following Vaccination in the Community; US: United States; VAERS: Vaccine Adverse Events Reporting System

Appendix A: Rapid review PRISMA Flow Diagram

PRISMA flow diagram
PRISMA flow diagram
PRISMA flow diagram: Text description

The PRISMA flow diagram describes the process by which articles were selected for the literature review. The process is broken down into four stages: Identification, Screening, Eligibility and Included.

Stage 1: Identification

  • 964 records were identified through the May 28, 2020 database search.
  • 945 records remained after duplicates were removed.

Stage 2: Screening

  • 945 records were then screened.
  • Of these 945 records, 548 records were excluded.

Stage 3: Eligibility

  • 397 full-text articles were assessed for eligibility.
  • Of these 397 full-text articles, 375 were excluded. The exclusion breakdown is as follows: No specific timing of AEFI: 170; Not outcome of interest: 18
  • Editorial, opinion or abstract: 40; Not in English or French: 73; Full text not available: 32; Other: 42

Stage 4: Included

  • 22 articles were included in the final synthesis: 8 case series and 5 case reports.
Figure 2. Risk of Bias Domains assessed (Modified IHE tool). Graphics were generated by robvis online risk-of-bias assessment visualization tool Footnote 36.
assessment visualization tool
Risk of Bias Domains assessed (Modified IHE tool): Text description
Figure 2 Long Description (EN)
Study Domain 1 Domain 2 Domain 3 Domain 4 Domain 5 Domain 6 Domain 7 Domain 8 Domain 9 Domain 10 Domain 11 Domain 12 Domain 13 Domain 14 Domain 15 Overall
Baxter et al., 2018 Low High High High Low High Unclear Unclear Low High High Low No information No information Low High
Bohlke et al, 2003 Low Unclear Low Low Low Low Low Low Low Low Low Low No information No information Low Unclear
Braun et al., 1997 Low High High High Low Unclear Low Unclear Low High Unclear High Low High High High
Brotherton et al., 2008 Low High High High Low Low Low No information Low High Low Low No information No information Low High
Cheng et al., 2015 Low High High High Low Low Low Low Low High Unclear Low No information Unclear Low High
Crawford et al., 2011 Low High High High Low Low Low Unclear Low Unclear Unclear Unclear No information No information Low High
Johann-Liang et al., 2011 Unclear High High High Unclear Low Unclear Low Low High Unclear Low No information No information Unclear High
McNeil et al., 2016 Low Unclear Low Low Low Low Low Low Low Low Low Low No information Unclear Low Unclear
Millstein et al., 1987 Low High High High Low Unclear Unclear No information Low High High Unclear No information No information Low High
Pahud et al., 2013 Low Unclear High High Low Unclear Unclear Unclear Low High Low Low No information No information Low High
Patja et al., 2000 Low Unclear High High Unclear Unclear Unclear No information Low High Low Low No information No information Unclear High
Schumacher et al., 2010 Low Unclear Low High Low Unclear Unclear Unclear Low High Unclear Unclear No information No information High High
Su et al., 2019 Low High High High Low Low Unclear Unclear Low Unclear Low Unclear No information Unclear Unclear High
Subelj et al., 2016 Low High High High Low Unclear Low Low Low High Unclear Low No information No information Unclear High
Sutherland et al., 2008 Low Unclear High High Low Unclear Unclear Unclear Low High Low Unclear No information No information Unclear High
Woo et al., 2005 Low High Unclear Unclear Low Unclear Unclear Unclear High Unclear High High No information No information High High
Zafack et al., 2019 Low High High High Low Low Unclear Low Low Low Unclear Low Low Low Low High
Poddighe et al., 2014 Low High High No information Low Low Low Low Low Low Low High No information No information High High
Stone et al., 2019 Low Unclear High No information Low Low Low Low Low Low Low High No information No information Low High
Stone et al., 2017 Low High High No information Low Unclear Low No information Low Low Low High No information No information Unclear High
Turktas et al., 1999 Low High High No information Low Unclear Low No information Low Low Low High No information No information High High
Worm et al. 2000 Low High Unclear Unclear Low Unclear Low No information Low Low Low High No information No information High High

List of abbreviations

Abbreviation Term
AE Adverse events
AEFI Adverse event following immunization
ATAGI Australian Technical Advisory Group on Immunisation
CAEFISS Canadian Adverse Event Following Immunization Surveillance System
DTP Diphtheria, tetanus, pertussis
FDA United States Food and Drug Administration
IHE Institute of Health Economics
HPV Human papillomavirus
HPV4 vaccine 4-valent human papillomavirus vaccine
MedDRA Medical Dictionary for Regulatory Activities
MMR Measles-mumps-rubella
NACI National Advisory Committee on Immunization
PHAC Public Health Agency of Canada
SAEFVIC Surveillance of Adverse Events following Vaccination in the Community
US United States
VAERS Vaccine Adverse Events Reporting System

Acknowledgments

This statement was prepared by: R Stirling, P Doyon-Plourde, N Forbes, K Young, and R Harrison, on behalf of the NACI Influenza Working Group and was approved by NACI.

NACI gratefully acknowledges the contribution of: G Crichlow, H Anyoti, A House, K Johnson, M Laplante, A Sinilaite, M Tunis, and E Westhaver

NACI Influenza Working Group

Members: R Harrison (Chair), N Dayneka, I Gemmill, K Klein, D Kumar, J Langley, J McElhaney, A McGeer, D Moore, S Smith, and B Warshawsky

Liaison representatives: L Grohskopf (Centers for Disease Control and Prevention [CDC], United States)

Ex-officio representatives: L Whitmore (Centre for Immunization and Respiratory Infectious Diseases [CIRID], PHAC), A Gartley (First Nations and Inuit Health Branch [FNIHB], Indigenous Services Canada [ISC]), and J Xiong (Biologic and Radiopharmaceutical Drugs Directorate [BRDD], Health Canada [HC]).

NACI

Members: C Quach (Chair), S Deeks (Vice-Chair), J Bettinger, N Dayneka, P De Wals, E Dubé, V Dubey, S Gantt, R Harrison, K Hildebrand, K Klein, J Papenburg, B Sander, S Smith and S Wilson.

Liaison representatives: LM Bucci (Canadian Public Health Association), E Castillo (Society of Obstetricians and Gynaecologists of Canada), A Cohn (CDC, United States), L Dupuis (Canadian Nurses Association), J Emili (College of Family Physicians of Canada), D Fell (Canadian Association for Immunization Research Evaluation), M Lavoie (Council of Chief Medical Officers of Health), D Moore (Canadian Paediatric Society), M Naus (Canadian Immunization Committee), and A Pham-Huy (Association of Medical Microbiology and Infectious Disease Canada).

Ex-officio representatives: D Danoff (Marketed Health Products Directorate, HC), E Henry (CIRID, PHAC), M Lacroix (Public Health Ethics Consultative Group, PHAC), J Pennock (CIRID, PHAC), R Pless (BRDD, HC), G Poliquin (National Microbiology Laboratory, PHAC), V Beswick-Escanlar (National Defense and the Canadian Armed Forces), and T Wong (FNIHB, ISC).

References

References

Reference a

Vaccine recipients should be kept under observation for at least 15 minutes after immunization; 30 minutes is a safer interval when there is a specific concern about possible vaccine allergy.

Return to Reference a referrer

Reference b

Appropriate public health and infection prevention and control measures are to be determined by the responsible public health authority in the jurisdiction of the vaccination clinic.

Return to Reference b referrer

Reference c

Adapted from the Australian Technical Advisory Group on Immunisation (ATAGI), "Statement on the duration of observation after vaccination in the context of minimizing risk of exposure to COVID-19 at health care facilities

Return to Reference c referrer

Reference d

Refer to the NACI Statement on Mammalian cell-culture based influenza vaccines

Return to Reference d referrer

Footnotes

Footnote 1

WHO. Coronavirus disease 2019 (COVID-19) Situation Report - 51. [online] Available at: https://www.who.int/docs/default-source/coronaviruse/situation-reports/20200311-sitrep-51-covid-19.pdf?sfvrsn=1ba62e57_10.

Return to footnote 1 referrer

Footnote 2

PHAC. Interim guidance on continuity of immunization programs during the COVID-19 pandemic. Updated May 13, 2020. [online] Available at: /content/canadasite/en/public-health/services/immunization/national-advisory-committee-on-immunization-naci/interim-guidance-immunization-programs-during-covid-19-pandemic.html

Return to footnote 2 referrer

Footnote 3

PHAC. Guidance for influenza vaccine delivery in the presence of COVID-19. [online] Available at: /content/canadasite/en/public-health/services/immunization/national-advisory-committee-on-immunization-naci/guidance-influenza-vaccine-delivery-covid-19.html

Return to footnote 3 referrer

Footnote 4

PHAC. Canadian Immunization Guide: Part 2 - Vaccine Safety. [online] Available at: /content/canadasite/en/public-health/services/publications/healthy-living/canadian-immunization-guide-part-2-vaccine-safety/page-4-early-vaccine-reactions-including-anaphylaxis.html

Return to footnote 4 referrer

Footnote 5

US CDC. 2017. General Best Practice Guidelines for Immunization: Best Practices Guidance of the Advisory Committee on Immunization Practices (ACIP). [online] Available at: https://www.cdc.gov/vaccines/hcp/acip-recs/general-recs/adverse-reactions.html

Return to footnote 5 referrer

Footnote 6

The Vaccine Administration Task Force. 2001. United Kingdom Guidance on Best Practice in Vaccine Administration [online] Available at: http://www.wales.nhs.uk/sitesplus/documents/861/UK%20best%20practice%20guidance%20vacc%20admin%202001.pdf

Return to footnote 6 referrer

Footnote 7

New Zealand Ministry of Health. 2018. Immunisation Handbook 2017 (2nd edn). Wellington: Ministry of Health. [online] Available at: https://www.health.govt.nz/system/files/documents/publications/immunisation-handbook-2017-2nd-edition-mar18-v9_1.html

Return to footnote 7 referrer

Footnote 8

Australian Government Department of Health. 2019. Australian Immunisation Handbook. [online] Available at: https://immunisationhandbook.health.gov.au/vaccination-procedures/after-vaccination

Return to footnote 8 referrer

Footnote 9

Australian Technical Advisory Group on Immunisation (ATAGI). 2020. ATAGI Clinical Statement on Vaccination Observation Time. [online] Available at: https://www.health.gov.au/resources/publications/atagi-clinical-statement-on-vaccination-observation-time

Return to footnote 9 referrer

Footnote 10

Guo B, Moga C, Harstall C, Schopflocher D. A principal component analysis is conducted for a case series quality appraisal checklist. Journal of clinical epidemiology. 2016 Jan 1;69:199-207.footnote1text

Return to footnote 10 referrer

Footnote 11

Rüggeberg JU, Gold MS, Bayas JM, Blum MD, Bonhoeffer J, Friedlander S, de Souza Brito G, Heininger U, Imoukhuede B, Khamesipour A, Erlewyn-Lajeunesse M. Anaphylaxis: case definition and guidelines for data collection, analysis, and presentation of immunization safety data. Vaccine. 2007 Aug 1;25(31):5675-84.

Return to footnote 11 referrer

Footnote 12

PHAC. Canadian Adverse Events Following Immunization Surveillance System (CAEFISS) [online] Available at: /content/canadasite/en/public-health/services/immunization/canadian-adverse-events-following-immunization-surveillance-system-caefiss.html

Return to footnote 12 referrer

Footnote 13

Baxter CM, Clothier HJ, Perrett KP. Potential immediate hypersensitivity reactions following immunization in preschool aged children in Victoria, Australia. Human Vaccines & Immunotherapeutics. 2018 Aug 3;14(8):2088-92.

Return to footnote 13 referrer

Footnote 14

Bohlke K, Davis RL, Marcy SM, Braun MM, DeStefano F, Black SB, Mullooly JP, Thompson RS. Risk of anaphylaxis after vaccination of children and adolescents. Pediatrics. 2003 Oct 1;112(4):815-20.

Return to footnote 14 referrer

Footnote 15

Brotherton JM, Gold MS, Kemp AS, McIntyre PB, Burgess MA, Campbell-Lloyd S. Anaphylaxis following quadrivalent human papillomavirus vaccination. Cmaj. 2008 Sep 9;179(6):525-33.

Return to footnote 15 referrer

Footnote 16

Cheng DR, Perrett KP, Choo S, Danchin M, Buttery JP, Crawford NW. Pediatric anaphylactic adverse events following immunization in Victoria, Australia from 2007 to 2013. Vaccine. 2015 Mar 24;33(13):1602-7.

Return to footnote 16 referrer

Footnote 17

Johann-Liang R, Josephs S, Dreskin SC. Analysis of anaphylaxis cases after vaccination: 10-year review from the National Vaccine Injury Compensation Program. Annals of Allergy, Asthma & Immunology. 2011 May 1;106(5):440-3.

Return to footnote 17 referrer

Footnote 18

McNeil MM, Weintraub ES, Duffy J, Sukumaran L, Jacobsen SJ, Klein NP, Hambidge SJ, Lee GM, Jackson LA, Irving SA, King JP. Risk of anaphylaxis after vaccination in children and adults. Journal of Allergy and Clinical Immunology. 2016 Mar 1;137(3):868-78.

Return to footnote 18 referrer

Footnote 19

Milstien JB, Gross TP, Kuritsky JN. Adverse reactions reported following receipt of Haemophilus influenzae type b vaccine: an analysis after 1 year of marketing. Pediatrics. 1987 Aug 1;80(2):270-4.

Return to footnote 19 referrer

Footnote 20

Pahud BA, Williams SE, Dekker CL, Halsey N, LaRussa P, Baxter RP, Klein NP, Marchant CD, Sparks RC, Jakob K, Aukes L. Clinical assessment of serious adverse events in children receiving 2009 H1N1 vaccination. The Pediatric infectious disease journal. 2013 Feb 1;32(2):163-8.

Return to footnote 20 referrer

Footnote 21

Patja A, Davidkin I, Kurki T, Kallio MJ, Valle M, Peltola H. Serious adverse events after measles-mumps-rubella vaccination during a fourteen-year prospective follow-up. The Pediatric infectious disease journal. 2000 Dec 1;19(12):1127-34.

Return to footnote 21 referrer

Footnote 22

Schumacher Z, Bourquin C, Heininger U. Surveillance for adverse events following immunization (AEFI) in Switzerland-1991-2001. Vaccine. 2010 May 28;28(24):4059-64.

Return to footnote 22 referrer

Footnote 23

Su JR, Moro PL, Ng CS, Lewis PW, Said MA, Cano MV. Anaphylaxis after vaccination reported to the Vaccine Adverse Event Reporting System, 1990-2016. Journal of Allergy and Clinical Immunology. 2019 Apr 1;143(4):1465-73.

Return to footnote 23 referrer

Footnote 24

Zafack JG, Toth E, Landry M, Drolet JP, Top KA, De Serres G. Rate of Recurrence of Adverse Events Following Immunization: Results of 19 Years of Surveillance in Quebec, Canada. The Pediatric infectious disease journal. 2019 Apr 1;38(4):377-83.

Return to footnote 24 referrer

Footnote 25

Stone CA, Commins SP, Choudhary S, Vethody C, Heavrin JL, Wingerter J, Hemler JA, Babe K, Phillips EJ, Norton AE. Anaphylaxis after vaccination in a pediatric patient: further implicating alpha-gal allergy. The Journal of Allergy and Clinical Immunology: In Practice. 2019 Jan 1;7(1):322-4.

Return to footnote 25 referrer

Footnote 26

Stone CA, Hemler JA, Commins SP, Schuyler AJ, Phillips EJ, Peebles RS, Fahrenholz JM. Anaphylaxis after zoster vaccine: Implicating alpha-gal allergy as a possible mechanism. Journal of Allergy and Clinical Immunology. 2017 May 1;139(5):1710-3./p>

Return to footnote 26 referrer

Footnote 27

Türktas I, Ergenekon E. Anaphylaxis following diphtheria-tetanus-pertussis vaccination-a reminder. European journal of pediatrics. 1999 Apr 1;158(5):434.

Return to footnote 27 referrer

Footnote 28

Worm M, Sterry W, Zuberbier T. Gelatin-induced urticaria and anaphylaxis after tick-borne encephalitis vaccine. Acta dermato-venereologica. 2000;80(3).

Return to footnote 28 referrer

Footnote 29

Braun MM, Patriarca PA, Ellenberg SS. Syncope after immunization. Archives of pediatrics & adolescent medicine. 1997 Mar 1;151(3):255-9.

Return to footnote 29 referrer

Footnote 30

Crawford NW, Clothier HJ, Elia S, Lazzaro T, Royle J, Buttery JP. Syncope and seizures following human papillomavirus vaccination: a retrospective case series. Medical journal of Australia. 2011 Jan;194(1):16-8.

Return to footnote 30 referrer

Footnote 31

Šubelj M, Učakar V, Kraigher A, Klavs I. Adverse events following school-based vaccination of girls with quadrivalent human papillomavirus vaccine in Slovenia, 2009 to 2013. Eurosurveillance. 2016 Apr 7;21(14):30187.

Return to footnote 31 referrer

Footnote 32

Sutherland A, Izurieta H, Ball R, Braun MM, Miller ER, KR Broder KR, Slade BA, Iskander JK, Kroger AT, Markowitz LE, Huang WT. Syncope after vaccination--United States, January 2005-July 2007. MMWR. Morbidity and mortality weekly report. 2008 May 2;57(17):457.

Return to footnote 32 referrer

Footnote 33

Woo EJ, Ball R, Braun MM. Fatal syncope-related fall after immunization. Archives of pediatrics & adolescent medicine. 2005 Nov 1;159(11):1083.

Return to footnote 33 referrer

Footnote 34

Poddighe D, Castelli L, Marseglia GL, Bruni P. A sudden onset of a pseudo-neurological syndrome after HPV-16/18 AS04-adjuvated vaccine: might it be an autoimmune/inflammatory syndrome induced by adjuvants (ASIA) presenting as a somatoform disorder?. Immunologic research. 2014 Dec 1;60(2-3):236-46.

Return to footnote 34 referrer

Footnote 35

Dreskin SC, Halsey NA, Kelso JM, Wood RA, Hummell DS, Edwards KM. International Consensus (ICON): Allergic reactions to vaccines. World Allergy Organ J. 2016; 9: 32.

Return to footnote 35 referrer

Footnote 36

McGuinness, LA, Higgins, JPT. Risk-of-bias VISualization (robvis): An R package and Shiny web app for visualizing risk-of-bias assessments. Res Syn Meth. 2020; 1- 7. https://doi.org/10.1002/jrsm.1411

Return to footnote 36 referrer

Page details

Date modified: