Risk assessment method at the Public Health Agency of Canada

On this page

• Overview
• Evidence review
• Risk pathway
• Likelihood of the adverse event
• Impacts of the adverse event
• Uncertainty
• Overall risk
• References

Overview

The Public Health Agency of Canada (PHAC) conducts risk assessments to better understand the risks posed by infectious disease threats and enable more prompt and effective action to safeguard public health. A risk assessment is a systematic process of gathering, assessing and documenting information to estimate the level of risk and associated uncertainty related to a public health event, during a specified period of time and in a specified location.^{Footnote 1}

The risk assessment method assigns qualitative estimates of likelihood and impact to public health threats, together with associated uncertainties. These estimates are derived through an integration of rapid review of the literature and expert opinion using standardized scales. The methodology used by PHAC has been adapted from the Joint Risk Assessment Operational Tool to assess the risk posed by zoonotic disease hazards developed jointly by the World Health Organization (WHO), Food and Agriculture Organization of the United Nations, and the World Organization for Animal Health (WOAH).^{Footnote 1} PHAC adapted the tool by modifying the likelihood and impact scales and associated definitions to incorporate elements from other risk assessment frameworks that are relevant to the Canadian context, including components of the WHO Rapid Risk Assessment Framework,^{Footnote 2} and the Rapid Risk Assessment Operational Tool developed by the European Centre for Disease Prevention and Control.^{Footnote 3} More detail about the PHAC risk assessment methodology can be found in a publication by Anand et al.^{Footnote 4}

Evidence review

Evidence is gathered by scientific experts using a rapid, non-systematic literature search and includes published and preprint journal articles, grey literature such as surveillance reports, and agency intelligence. Evidence and intelligence are also gathered from subject matter experts in order to help fill gaps in the published literature.

Risk pathway

It is crucial to clearly establish the specific adverse event for which the likelihood will be calculated before estimating the risk. This is achieved by having a risk framing discussion with all relevant stakeholders. A risk bowtie diagram (Figure 1) can be used to illustrate the adverse event of interest for the risk assessment (i.e., the central node).^{Footnote 5} Barriers that prevent or mitigate the drivers and impacts of the adverse event can be portrayed along the pathway (Figure 1). The risk question being assessed can then be further modeled using a risk pathway (Figure 2), a diagrammatic representation of the key components of the sequence of the hazard from its source to the adverse event of concern, subsequent spread scenario(s) and resultant impacts. Each step in the risk pathway is associated with a likelihood or impact sub-question that is addressed as part of the risk assessment. For example, a risk pathway developed for the importation of measles into Canada (Figure 3) shows the measles introduction in Canada, transmission in the population and the resultant health impacts.

Figure 1: Descriptive text

A bowtie diagram depicting drivers of the adverse event on the left, the center node is the adverse event of interest. To the right are the impacts resulting directly and/or indirectly from the adverse event. Barriers that prevent or mitigate the drivers and impacts of the adverse event are marked along the pathway.  

Figure 2: Descriptive text

Pathway connector from travellers from affected country to travellers in the infectious stage enter Canada. A potential intervention for this step is posting travel advisories.

• Risk sub-question: what is the likelihood of travellers in the infectious stage entering Canada in the next X weeks/months?

Pathway connector from Travellers in the infectious stage enter Canada and from infection present in domestic communities to Canadian exposed via close contact, non-close contact, vertical and/or indirect contact.

• Risk sub-question: what is the likelihood a Canadian is exposed via close contact, non-close contact, vertical and/or indirect contact?

Pathway connector from Canadian exposed via close contact, non-close contact, vertical and/or indirect contact to infected individual

• Risk sub-question: what is the likelihood that an individual is infected?

Pathway connector from individual infected to multiple outbreaks within specific communities or settings

• Risk sub-question: what is the most likely spread scenario?

Pathway connector from multiple outbreaks within specific communities or settings to impact on health of directly affected individuals (including social, technological, economic, environmental, political and regulatory, and population health and health systems [STEEPP])

• Risk sub-question: what is the impact on the health of directly affected individuals?

Pathway connector from multiple outbreaks within specific communities or settings and impact on health of directly affected individuals to population-level impacts (including STEEPP)

• Risk sub-question: what are the population-level impacts on the general Canadian population?

Pathway connector (not assessed in this risk assessment) from impact on health of directly affected individuals (including STEEPP) and population-level impacts (including STEEPP) to long-term health impacts (including STEEPP)

Note: Example risk pathway covering most factors considered in a risk assessment for an infectious disease, such as introduction into Canada, exposure, population susceptibility, spread, direct and indirect impacts. In this risk pathway, the adverse event (black) is the individual infected in Canada, but the adverse event could also be the spread e.g., multiple outbreaks within specific communities/settings. Examples of possible risk sub-questions are outlined on the right-hand side.

Figure 3: Descriptive text

Importation to Canada:

• Pathway connector from travellers infected with measles in other country leads to travellers in the infectious stage enter Canada
  • Risk sub-question: What is the likelihood of travellers in the infectious stage entering Canada in the next 6 weeks?

Spread within Canada:

• Pathway connector from travellers in the infectious stage enter Canada leads to adverse event of concern: Multiple outbreaks within specific communities/settings in the next 6 weeks
  • Risk sub-question: What is the likelihood of multiple outbreaks in specific communities/settings in the next 6 weeks?
• Pathway connector from multiple outbreaks within specific communities/settings in the next 6 weeks leads to adverse event of concern: Prolonged community transmission in general population in Canada in the next 3 months
  • Risk sub-question: What is the likelihood of prolonged community transmission in the next 3 months?

Health impacts:

• Pathway connector from prolonged community transmission in general population in Canada in the next 3 months to Impact on health of directly affected individuals
  • Risk sub-question: What is the impact on health of directly affected individuals?
• Pathway connector from prolonged community transmission in general population in Canada in the next 3 months to Population-level impacts
  • Risk sub-question: What are the population-level impacts in the general Canadian population?
• Pathway connector from multiple outbreaks within specific communities/settings in the next 6 weeks to Population-level impacts
  • Risk sub-question: What are the population-level impacts in specific populations/communities?

Likelihood of the adverse event

The risk pathway describes a sequence of events, and the likelihood of each event is conditional on the likelihood of preceding step(s) in the risk pathway leading up to the adverse event(s) of concern. A qualitative likelihood estimate is assigned for each sub-question (i.e., each step in the risk pathway), as in Table 1. The overall likelihood for the adverse event of concern is determined by the lowest likelihood estimated along the risk pathway. Likelihood in general represents values between zero and one, and this should be kept in mind even when dealing with qualitative estimates. Accepted methodologies in qualitative risk assessments often involve adopting the lowest value for dependent events and the highest value for independent events.^{Footnote 6}

Table 1. Scale and criteria to estimate likelihood

Likelihood estimate	Criteria
High	The situation described in the risk assessment question is highly likely to occur (i.e., is expected to occur in most circumstances).
Moderate	The situation described in the risk assessment question is likely to occur.
Low	The situation described in the risk assessment question is unlikely to occur.
Very low	The situation described in the risk assessment question is very unlikely to occur (i.e., is expected to occur only under exceptional circumstances).

Impacts of the adverse event

The impacts of the event are based on the determination of the most likely spread scenario within Canada should the event of concern occur. The most likely spread scenario is influenced by the pathogen transmissibility, speed of transmission, and effectiveness of public health measures and medical countermeasures. It is not necessarily a description of what will happen; in some situations, all spread scenarios may be unlikely to occur if they are contingent upon earlier steps in the risk pathway that are themselves unlikely. Risk assessments typically consider individual-level impacts on affected individuals and, if relevant, impacts on population sub-groups considered at higher risk (Table 2), as well as impacts on the general population (Table 3).

Table 2. Scale and criteria for estimating magnitude of effect for individuals/population sub-groups if the situation described in the risk assessment question occurs

Impact estimate	Criteria
Severe	Severe impact on disease morbidity/mortality, and/or mental health, and/or welfare (e.g., loss of income).
Major	Major impact on disease morbidity/mortality, and/or mental health, and/or welfare (e.g., loss of income).
Moderate	Moderate impact on disease morbidity/mortality, and/or mental health, and/or welfare (e.g., loss of income).
Minor	Minor impact on disease morbidity/mortality, and/or mental health, and/or welfare (e.g., loss of income).
Minimal	Minimal or no impact on disease morbidity/mortality, and/or mental health, and/or welfare (e.g., loss of income).

Table 3. Scale and criteria for estimating population-level impact in Canada

Impact estimate	Impact criteria	Impact considerations
Severe	The situation described in the risk assessment question will have severe negative consequences on the population.	Severe impact on disease morbidity/mortality, and/or mental health, and/or welfare (e.g., loss of income) Potential pandemic in the general population or large numbers of case reports, with significant impact on the well-being of the population  A major threat to day-to-day functioning of health, economic, social, and cultural systems and infrastructures  Severe disruption of normal population health, health systems, activities, and services  Drastic decline and/or collapse in sectors that support and/or depend on provision of health care resources and infrastructures  Political and regulatory interventions required to restrict movement of people, animals and goods  National/international travel and trade disrupted with mid- to long-term implications Response and recovery measures at significant cost for national and international authorities and stakeholders     High degree of disruption in most sectorsFootnote a Effect extremely serious and/or irreversible
Major	The situation described in the risk assessment question will have major negative consequences on the population.	Moderate to significant impact on disease morbidity/mortality, and/or mental health, and/or welfare (e.g., loss of income) affecting a larger proportion of the population and/or several regions Case reports with moderate to significant impact on the well-being of the population Health system functional but resource stressed   May threaten medium-term resources, capacity and stability of health system and infrastructures  Response and mitigation measures involve major cost for regional and national levels   Some disruption of regional and national economic and socio-cultural activities  Some limitations to national/international travel and trade  Moderate degree of disruption in some sectorsFootnote a Effect serious with substantive consequences, but usually reversible
Moderate	The situation described in the risk assessment question will have moderate negative consequences on the population.	Low to moderate impact on disease morbidity/mortality, and/or mental health, and/or welfare (e.g., loss of income) affecting a larger proportion of the population and/or several regions Case reports with low to moderate impact on the well-being of the population Health system functioning well with some short-term reductions in health care system capacity  Resources for health response and mitigation measures are anticipated and adequate for regional and national levels   Few to no disruptions to regional and national economic and socio-cultural activities in the short term  Mild degree of short-term disruptions in some sectorsFootnote a such as education and workforce   Effect noticeable with important consequences, but usually reversible
Minor	The situation described in the risk assessment question will have minor negative consequences on the population.	Moderate to significant impact on disease morbidity/mortality, and/or mental health, and/or welfare (e.g., loss of income) on a small proportion of the population and/or small areas (regional level or below) Rare case reports, mainly in small at-risk groups, with moderate to significant impact on the well-being of the population No threat to the health system, economic or socio-cultural activities   Few measures needed at regional level are manageable with low to moderate costs  Potential for minor short-term disruptions in other sectorsFootnote a such as travel and trade  Effect marginal, but insignificant and/or reversible
Minimal	The situation described in the risk assessment question will have minimal or no negative consequences on the population.	Negligible or no impact on disease morbidity/mortality, and/or mental health, and/or welfare (e.g., loss of income) No or very rare case reports with low to moderate impact on the well-being of the population No threat to the health system, economic or socio-cultural activities  Few manageable measures at minor cost are needed at regional and national levels  Disruptions in other sectorsFootnote a are not anticipated  Effect not usually distinguishable from normal day-to-day variation

Uncertainty

For each likelihood sub-question and impact estimation, a level of uncertainty is assigned based on various criteria (Table 4), including quality and quantity of evidence provided, expert opinion of the technical team, and inherent variability.

Table 4. Scale and criteria for estimating level of uncertainty in risk assessment estimates

Uncertainty estimate	Confidence estimate	Criteria
Low	High	Few information gaps (e.g., relevant surveillance data available, sufficient peer-reviewed literature, detailed information from local authorities regarding response capacity, including data from previous response activities).   General agreement among experts about the risk (e.g., based on current or past experience with similar situations).  Low inherent (i.e., natural) variability or the variability is well characterized. Additional data and information are unlikely to change the result of the assessment.
Moderate	Moderate	Gaps in information present; however, they are not necessarily key to assessing the risk (e.g., some surveillance data available, peer-reviewed literature to support certain estimates available, limited information from local authorities on response capacity). Some agreement among experts, especially on key aspects of the risk (e.g., experience with previous events relevant to current situation). Moderate inherent (i.e., natural) variability or variability is characterized to a limited extent. Additional data and information are likely to change the assessment.
High	Low	Critical gaps in key information needed to assess the risk (e.g., no relevant surveillance data, limited or no peer reviewed literature, knowledge about response capacity not informed by local authorities).   Low level of agreement among experts on key aspects of risk (e.g., novel type of event, lack of experience with similar situations).  High inherent (i.e., natural) variability or variability is not well characterized. Additional data and information are very likely to change the assessment.

Overall risk

It is helpful for risk communication purposes to include a statement of the overall risk that a hazard presents to the general population of Canada.  The overall risk generated is for the purpose of a given risk assessment within its particular context, therefore overall risks should not be compared and ranked across PHAC risk assessments. A risk matrix (Table 5) is used to generate an initial risk estimate in PHAC risk assessments. While risk matrices can be useful for combining results of the risk assessments into a single risk estimate for the general population, this methodology must be used carefully, given the limitations of risk matrices.^{Footnote 7}

Estimating overall risk

• Overall risk level is provided only for the general population.
• Likelihood and impact estimates from the risk assessment are plotted in the qualitative risk matrix to obtain an initial estimate of overall risk. If an overall likelihood is estimated in the risk assessment, then this is the estimate that is used in the matrix. If not, the lowest likelihood estimate from the risk sub-questions is used to approximate multiplication of probabilities (as long all likelihoods are conditional). In some cases, a particular risk sub-question may be a major consideration for the most likely spread scenario, in which case that likelihood estimate may be the most reasonable to use in the matrix. The population-level impact for the general population is used for the impact level.
• Once an overall risk level is identified, an adjustment may be made with a rationale, i.e., where it is thought that likelihood or impact should have a greater contribution to the overall risk than the risk matrix provides. It should be noted that the population-level impact is often identified as the most important contributor to overall risk, particularly as it is based on the most likely spread scenario.

Table 5. Matrix for overall risk^{Footnote a}

	Minimal impact	Minor impact	Moderate impact	Major impact	Severe impact
High likelihood	Moderate risk	Moderate risk	High risk	High risk	High risk
Moderate likelihood	Low risk	Moderate risk	Moderate risk	High risk	High risk
Low likelihood	Low risk	Low risk	Moderate risk	Moderate risk	High risk
Very low likelihood	Low risk	Low risk	Low risk	Moderate risk	Moderate risk