Evidence Based Process for developing travel and tropical medicine related guidelines and recommendations

An Advisory Committee Statement (ACS)

Committee to Advise on Tropical Medicine and Travel (CATMAT)

Table of Contents

• Preamble
• Key points/Messages
• Introduction
• Objective
• Guideline Development Methods
  • Overview
  • Methods of Guideline Development Process
  • Deciding on methodological approach for Recommendation Development
  • General approach for Development of non-GRADE Recommendations
    • Recommendation based on a literature review
    • Good practice Recommendations
    • Descriptive Statements
  • General approach for Development of a Guideline containing GRADE Recommendations
    • Developing an Analytic Framework and identifying key questions (steps 1-5)
    • Assessing the quality (confidence in the estimate of effect) of the evidence (steps 6-12)
    • Rating down the quality of the body of evidence
    • Estimating the baseline risk of disease and quality of the body of evidence (steps 9 & 10)
    • Establishing a risk threshold to guide GRADE Recommendations (step 14)
    • Strength of the Recommendation (step 15)
• Discussion
  • Benefits of GRADE in the Travel Medicine Context
  • GRADE Challenges and Limitations
  • Conclusion and future directions
• Acknowledgements
• Conflict of Interest
• References
• Appendices
  • Appendix 1: Summary of the GRADE Framework for developing recommendations
  • Appendix 2: Typhoid example of logic model and Analytic Framework
  • Appendix 3: PICO and Key Questions for Typhoid Analytic Framework
  • Appendix 4: Criteria to Judge Quality of Evidence
  • Appendix 5: Evidence profile for the efficacy of typhoid vaccine
  • Appendix 6: Summary of findings for the efficacy of typhoid vaccine
  • Appendix 7: Evidence profile for assessment of confidence in estimates of baseline risk of typhoid by region

Preamble

The Committee to Advise on Tropical Medicine and Travel (CATMAT) provides the Public Health Agency of Canada with ongoing and timely medical, scientific, and public health advice relating to tropical infectious disease and health risks associated with international travel. The Agency acknowledges that the advice and recommendations set out in this statement are based upon the best current available scientific knowledge and medical practices, and is disseminating this document for information purposes to both travellers and the medical community caring for travellers.

Persons administering or using drugs, vaccines, or other products should also be aware of the contents of the product monograph(s) or other similarly approved standards or instructions for use. Recommendations for use and other information set out herein may differ from that set out in the product monograph(s) or other similarly approved standards or instructions for use by the licensed manufacturer(s). Manufacturers have sought approval and provided evidence as to the safety and efficacy of their products only when used in accordance with the product monographs or other similarly approved standards or instructions for use.

Key Points/Messages

• CATMAT has been developing travel and tropical medicine guidelines and recommendations using an "evidence based" approach since 1994. 
• CATMAT has updated its guideline development process and will use the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) method for certain recommendations. For other recommendations, a less rigorous search and synthesis of the literature will be used. 
• The choice of recommendations to which GRADE will be applied will depend on a number of factors including: the anticipated burden of the disease, the seriousness of the outcomes to be prevented, the potential benefits and harms of the intervention, the quality of the evidence, and the resources available to the committee.
• These updated methods will provide a transparent process of developing travel and tropical medicine related recommendations.

Introduction

The Canadian Committee to Advise on Tropical Medicine and Travel (CATMAT) has been developing guidelines for travel health and tropical disease for more than 20 years. Over this period, evidence-based medicine (EBM) methodologies have evolved dramatically. To keep pace with these developments, CATMAT has recently updated its approach to developing guidelines. In 2013, CATMAT formally adopted the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) system to develop guidelines. GRADE uses a systematic  framework for assessing the quality of the body of evidence and determining the direction and strength of a recommendation^{Footnote 1 Footnote 2}.This methodology  emphasizes transparency, and clinically important outcomes. CATMAT will use GRADE to develop certain recommendations; otherwise evidence assessment and recommendation development will be developed using a less rigorous synthesis of the evidence.

The approach chosen to develop recommendations will depend on a number of factors including: the anticipated burden of the disease, the seriousness of the outcomes to be prevented, the potential benefits and harms of the intervention, the quality of the evidence, and the resources available to the committee. The first CATMAT statement to use this approach was the statement on "International Travel and Typhoid (2014)"^{Footnote 3}. It included recommendations developed with and without GRADE. A similar approach was used to develop the CATMAT statement on Travellers' Diarrhea^{Footnote 4}. A full list of CATMAT statements can be found at About CATMAT^{Footnote 5}.    

Objective

The objective of this statement is to describe CATMAT's current guideline development process. The target population for this statement are:  health care providers who use CATMAT guidelines; those who have an interest in and/or to whom the CATMAT guidelines apply; and, individuals who will be involved with developing CATMAT guidelines. The intent is that this statement will enhance the understanding of CATMAT's approach to guideline development. It should also help with the interpretation and implementation of their recommendations.

Guideline Development Methods

Overview

CATMAT is an expert panel, composed primarily of clinicians and other subject matter experts, that writes guidelines and makes recommendations related to the prevention and treatment of infectious diseases and other health hazards that may be encountered by Canadians travelling outside of the country. The work of the panel is supported by a secretariat at the Public Health Agency of Canada (PHAC). Additional information on committee functioning, reporting, membership, conduct and conflict of interest requirements are available on Government of Canada's website^{Footnote 6}. 

CATMAT identifies and prioritizes potential guideline topics at least annually. CATMAT members, PHAC, stakeholders or other organizations and individuals, may suggest topics. For each guideline topic, a working group (WG) is formed and is comprised of members from the committee, the CATMAT secretariat and, as necessary, external experts and/or interested parties. The WG is responsible to develop the scope of the proposal and to write an initial draft guideline. CATMAT reviews drafts and revisions are made accordingly. CATMAT votes on the recommendations and, once approved by the committee, the final version is presented to PHAC for approval and publication.   

Methods of Guideline Development Process

All guidelines will include a clear description of the methods including how the literature was identified and reviewed, the approach used for recommendation development (i.e. GRADE or other method) and reasons for choosing a given approach. Guidelines may include a recommendation based on a GRADE approach, a recommendation based on a synthesis of evidence (without GRADE) or Good Practice statements or a combination (see description below).  In some cases, guidelines will be purely descriptive. These reviews will synthesize the relevant literature on the topic and will highlight the epidemiology of a condition of concern for a specific population, e.g., "International Travellers who intend to Visit Friends and Relatives"^{Footnote 7}. Descriptive guidelines are valuable and a useful resource for travel medicine providers as they will serve to promote the health of particular population^{Footnote 7}.

Deciding on Methodological Approach for Recommendation Development

A CATMAT guideline document may include recommendations based on GRADE, on a synthesis of evidence (without GRADE) or a combination of both. The use of GRADE for selected recommendations will be based on the judgement of the full committee and the WG.

Criteria that enhance the likelihood that a recommendation(s) will be developed with GRADE:

• A high prevalence (common) travel-related condition for which an effective intervention is available.
• A rare, low, or moderate prevalence (less common/rare) travel-related condition associated with significant morbidity or mortality for which an effective intervention is available.
• A travel-related condition for which the potential magnitude of benefits or harms of an intervention is high.
• An intervention for which there is a narrow gradient between the balance of harms and benefits.

Criteria that reduce the likelihood that a recommendation(s) will be developed with GRADE:

• A travel-related condition for which urgent guidance is required (e.g., Zika virus in 2016)^{Footnote 8}.
• A travel-related condition for which the effectiveness of the intervention is unknown or poorly defined.
• Interventions that are not central to the key question(s) addressed in the guideline but may provide important contextual or background information (e.g., hand washing).
• Resource constraints.

General Approach for Development of Non-GRADE Recommendations

Recommendation Based on a Literature Review

Non-GRADE recommendations may be based on a literature review.  Where possible, recommendations will be informed by recent systematic reviews, relevant randomized controlled trials (RCTs) or other literature. Non-GRADE recommendations are not inherently of lower value; indeed, they might be relatively more important than a GRADE recommendation. For example, a GRADE recommendation for a vaccine that protects against a relatively low-probability threat might have less impact than a non-GRADE recommendation for an intervention that provides significant protection (e.g., washing hands, using insect repellents) against an array of threats.  

Good Practice Recommendations

Good practice recommendations are suited to interventions or other preventive approaches that entail minimal or no risk, inconvenience, or cost, and for which there is indirect or generally accepted evidence of benefit in terms of reducing the outcome or harm. This type of recommendation does not require formal literature to substantiate the recommendation. Examples include avoiding high-risk foods or suggesting that a physician think of and test for malaria in any patient with a febrile illness who has recently returned from a malaria endemic area. Good practice recommendations may address issues related to equity in health care.

Good practice statements should have the following characteristics^{Footnote 9}:

• Clear, actionable and feasible
• Have an important message
• Have an unequivocal net benefit with negligible harms and costs.

Descriptive Statements

Descriptive statements that highlight the epidemiology and baseline risk of a specific population such as the CATMAT statement on "International Travellers who intend to Visit Friends and Relatives" are also valuable as they are a useful resource for travel medicine providers and will serve to promote the health of particular populations^{Footnote 7}. These reviews will synthesize the relevant literature on the topic.

General Approach for Development of a Guideline Containing GRADE Recommendations

The following steps outline the CATMAT approach to developing guidelines that contain GRADE-based recommendations (Appendix 1). For examples of how this approach has been applied see the CATMAT statements on "International Travel and Typhoid (2014)"^{Footnote 3} and "Travellers' Diarrhea"^{Footnote 4}.

Determine the scope and specific objective(s) of the guideline.

1. Develop an analytic framework (Appendix 2).
2. Determine what, if any, specific questions will be assessed with a GRADE approach.
3. For interventions to be assessed with GRADE, develop a "PICO" (population of interest, intervention, comparison and outcome) question(s) (Appendix 3). Assign each outcome a level of importance i.e. critical, important, not-important (see below).
4. Identify other key and contextual questions that will inform the development of recommendations.
5. Conduct a literature search to identify relevant evidence.
6. If a published systematic review(s) is/are identified, assess its/their relevance and quality^{Footnote 9}. If relevant and of satisfactory quality, use the systematic review (with a literature review to identify relevant evidence since its publication) as the basis for the GRADE analyses.
7. Assess the quality of the body of evidence for each individual outcome for every intervention/comparator pair by assessing risk of bias, publication bias, imprecision, inconsistency, and indirectness.
8. Assign a degree of confidence in the estimate of effect for each outcome for each intervention assessed, i.e. very low, low, moderate, or high (Appendix 4)^{Footnote 10}.
9. Estimate the baseline likelihood for the outcome(s) of interest in the population(s) of interest.  This may highlight specific risk populations (e.g. adults/children/infants) or travel destinations (e.g. Africa, southeast Asia, South America) as appropriate.
10. Assess the quality of the body of the evidence for baseline likelihood using the same criteria as are applied to interventions i.e. risk of bias, publication bias, imprecision, inconsistency, and indirectness.
11. Summarize the evidence by outcome(s) for each intervention(s)/comparator in Evidence Profiles (EP) and Summary of Findings tables (SoF) (Appendix 5 & 6) as per the GRADE methodology.
12. Summarize the evidence by outcome(s) for the baseline risk overall and by risk groups in Evidence Profiles (EP) (Appendix 7) and Summary of Findings tables (SoF) (not included) as per the GRADE methodology.
13. Calculate the relative effect and confidence interval and, using the baseline risk, the absolute effect and confidence interval. 
14. Establish a risk threshold for use (or non-use) of the intervention(s) based on the balance of benefits and harms combined with evidence for patient values and preferences. In the absence of published evidence for patient values and preferences, the committee will use their experience in shared decision making to estimate an appropriate decision threshold.
15. Develop recommendations taking into consideration:
    1. Magnitude of benefits and harms of the intervention.
    2. Confidence in the estimates of the effect for intervention benefits and harms.
    3. The values and preferences of travellers including considerations related to cost and the resulting benefit/harm threshold.

Developing an Analytic Framework and Identifying Key Questions (steps 1-5)

An analytic framework should identify clinical preventive actions (interventions) and relevant factors that impact on outcomes including important subgroups at risk, availability or uptake of diagnostic test(s) or intervention(s), and differences in how the outcome is measured (or if surrogate endpoints will be used)^{Footnote 11}. Assign a level of importance to each outcome on a 1 to 9 scale, i.e. critical (7-9), important (4-6), and limited importance (1-3). Ranking importance in this way can help to focus effort on the most relevant questions. The target audience of CATMAT statements are clinicians and patients; therefore, in most circumstances, the primary perspective will be that of the patient and the relative importance of the outcome should reflect this. Examples of an analytic framework and PICO and key question developed for the International Travel and Typhoid statement^{Footnote 3}, are shown in Appendix 2 and 3. 

Assessing the Quality (confidence in the estimate of effect) of the Evidence (steps 6-12)

The quality of the "body of evidence" by outcome is assessed by determining the study design by which the selected evidence was produced (e.g., RCT, observational studies) and then assessing for risk of bias, inconsistency across studies, indirectness, imprecision, and publication bias^{Footnote 12}. If concerns are identified then the quality of evidence may be rated down (see below). Evidence is summarized into Evidence Profiles (EP) and Summary of Findings (SoF) tables, and each outcome/intervention combination is assigned a quality designation (very low, low, moderate, or high (see box 1  and examples of tables in Appdendix 5 & 6)^{Footnote 10}. This quality assessment reflects confidence in how close the estimate is to the true effect.  For interventions, evidence derived from RCTs starts out with high confidence whereas it starts out with low confidence for observational studies. For baseline risk estimates, the situation is different. Observational cohort studies start out as high quality evidence because they are more suited than RCTs to provide population based estimates of risk^{Footnote 13} and RCTs, if they include non-representative populations, will  be rated down for indirectness^{Footnote 13 Footnote 14} (See further discussion in section on baseline risk below). Readers are refered to a series of articles on GRADE in the Journal of Clinical Epidemiology for a thorough treatment of the quality assessment process^{Footnote 10 Footnote 12 Footnote 15 - Footnote 19}.  Other articles describe rating the quality of evidence for observational studies^{Footnote 13 Footnote 14}. What follows is a brief review of the quality assessment domains.

Box 1: Levels of Quality of the Evidence and Confidence in the Evidence

Levels	Rationale
High	We are very confident that the true effect lies close to that of the estimate of effect.
Moderate	We are moderately confident in the effect estimate. The true effect is likely close to the estimate of effect, but there is a possibility that it is substantially different.
Low	Our confidence in the effect estimate is limited. The true effect may be substantially different from the estimate of the effect.
Very low	We have very little confidence in the effect estimate. The true effect is likely to be substantially different from the estimate of effect.
Adapted from Balshem et al. 2011Footnote 10

Rating Down the Quality of the Body of Evidence

Risk of Bias

Quality of the evidence may be rated downwards by one or two levels for serious or very serious risk of bias, respectively^{Footnote 19}. Examples that could result in rating down for RCTs include: inappropriate randomization; lack of allocation concealment; failure to express outcomes by 'intention-to-treat'; inadequate blinding; substantial loss to follow-up; and stopping early for benefit.

Inconsistency

Quality of the evidence may be rated downwards by one or two levels if the estimate of effect shows substantial variation across studies^{Footnote 17}.

Indirectness

Quality of the evidence may be rated downwards by one or two levels if there are serious or very serious issues with indirectness. Examples of indirectness include: use of surrogate rather than patient-important outcomes (e.g. use of immunogenicity as opposed to episodes of infection); making indirect comparisons between two treatments; study populations that are dissimilar to the population(s) of interest; and test inaccuracies^{Footnote 16}. The extent of concerns (no concerns, serious concerns, very serious concerns) is a matter of judgment. For instance, some might view immunogenicity as sufficient evidence of benefit. If studies of the extent of protection are planned after establishing immunogenicity, this suggests the need to rate down one for indirectness.

Imprecision

Quality of the evidence may be rated downwards by one or two levels if there is serious or very serious imprecision (e.g., wide confidence intervals)^{Footnote 15}.

Publication Bias

Quality of the evidence may be rated downwards by one or two levels if publication bias is detected or strongly suspected^{Footnote 18}.

Rating Up the Quality of the Body of Evidence

In specific circumstances it may be appropriate to rate up the quaility of the body of evidence^{Footnote 20}. These are described below. 

Large Magnitude of Effect/Strength of Association

Quality of the evidence may be rated upwards by one or two levels based on magnitude of effect. As a general guideline, rating up by one level may be considered if evidence from RCTs or observational studies suggest a 2 fold or greater reduction or increase (OR/RR >=2 or =< .5) with no major residual confounders. Rating upwards by two or more levels may be considered if evidence from RCTs or observational studies suggest a 5 fold or greater reduction or increase (OR/RR >=5 or =< .2) with no major residual confounders. For instance, rigorous observational studies demonstrating reduction in incidence of the target condition by over 90% in vaccinated versus unvaccinated cohorts would warrant rating up by two levels. Generally, rating upward for a large effect should not be applied if the informing evidence suffers from serious methodologic limitations. Serious limitations include use of surrogates: we would never rate up for evidence from a surrogate such as immunogenicity. Rating up of RCTs is theoretically possible but seldom done. 

Dose-Response Gradient

Quality of the evidence may be rated upwards if there is evidence of a dose-response gradient. The dose response has long been recognized as an important criterion supporting a cause-effect relationship and its presence increases confidence in the findings of an observational study.

Mitigated Bias and Confounding

Observational studies begin as low-quality evidence because they are more likely to suffer from unmeasured or unknown sources of error and are therefore more likely to be affected by residual confounding. For such studies, if all residual confounders in a study would result in an underestimate of an apparent treatment effect then the quality of the evidence may be rated upward by one level. 

Estimating the Baseline Risk of Disease and Quality of the Body of Evidence (steps 9 & 10)

Estimating the baseline likelihood of an outcome allows for estimation of absolute intervention benefit and related measures, e.g., number needed to treat (NNT) or number needed to vaccinate (NNV) to prevent a case. Methods to assess the quality of the body of evidence from observational, non-intervention studies of baseline likelihood have recently been published^{Footnote 13 Footnote 14}. The GRADE group recommends applying the same criteria that are used for assessing the quality of intervention-based data, i.e. risk of bias, publication bias, imprecision, inconsistency, and indirectness^{Footnote 13 Footnote 14}. A notable difference when assessing quality for baseline risk studies is that observational studies start out as high-quality data because they are generally more suited than RCTs to provide population based estimates of risk^{Footnote 13}. Different tools have been developed to assess the risk of bias of observational studies including the New Castle Ottawa instrument and the Cochrane ROBINS-I tool^{Footnote 21 Footnote 22}. The criteria for assessing bias depend on the study design and should include assessment for selection, detection, and attrition bias. Key questions for this assessment include:

•  Is the study sample from a well-defined population that is representative of Canadian travellers?
•  Are the exposure and outcome(s) measures reliable and accurate?
•  Is it likely that there is complete ascertainment of the exposure(s) and outcome(s)?
•  Has follow-up been sufficiently long and complete?^{Footnote 13}

An example of the evaluation of studies of baseline risk and their quality assessment (adapted from the statement on International Travel and Typhoid statement) is included in Appendix 7^{Footnote 3}.

Establishing a Risk Threshold to Guide GRADE Recommendations (step 14)

A GRADE recommendation will be accompanied by a risk threshold determined by the balance between the importance (utility or disutility) of the relevant outcomes to the patients. It represents the point at which an intervention is recommended or not recommended. Thresholds must take into account patient values and preferences. Ideally, these will be derived from studies on the relevant population(s). In the absence of such data or as a second source of information, the experience of clinicians who regularly engage in shared decision-making can be used to estimate patient values and preferences. In all cases, the evidence and/or judgements made to establish thresholds will be explicitly stated.

An example of a risk threshold is the judgement made by CATMAT in the "International Travel and Typhoid Statement" that travellers would accept a likelihood of 1/10,000 for developing clinical disease before accepting the cost and inconvenience of typhoid vaccine^{Footnote 3}. This decision balanced factors such as the relatively high effectiveness of typhoid therapy, the low absolute risk of typhoid-associated hospitalization or death (< 1%), the costs associated with vaccination, the relatively low efficacy of the vaccine (50%), and the minor harms such as fever and gastrointestinal upset associated with the vaccine. In contrast, the risk threshold for taking Japanese encephalitis vaccine to prevent Japanese encephalitis (JE) is likely to be substantially lower, perhaps by one or two orders of magnitude than for typhoid (i.e. a risk of disease of 1/100,000 or lower). This reflects fundamental differences between these diseases and vaccines. Specifically, JE is: more severe (25% of cases die and 50% of those who survive develop severe and long-term neurologic sequelae); lacks specific treatments like antibiotics (which can be used for typhoid); and, has a higher probability of being prevented (>90% efficacy) through vaccination.    

Strength of the Recommendation (step 15)

The strength of a recommendation reflects the extent to which the committee is confident that the desirable effects of an intervention outweighs undesirable effects, taking into account the importance and impact of the intervention, and values and preferences of the patient. The following factors need to be considered (Box 2) when determining the strength associated with a given recommendation. The categories of, and implicit meanings for the various types of CATMAT GRADE-based recommendations are described in Box 3.

 Box 2: Factors Impacting on the Strength of a Recommendation

Factors	Impact on the strength of a recommendation
Certainty about the evidence of effects	The higher the quality of evidence, the more likely a strong recommendation is warranted.
Balance between desirable/undesirable effects	The larger the difference between the desirable and undesirable effects, the more likely a strong recommendation is warranted. The smaller the net benefit and the narrower the gradient, the more likely a conditional recommendation is warranted.
Importance of outcomes (more likely a weak recommendation)	The more variability or more uncertainty about the relative importance of the outcome, the more likely a conditional recommendation is warranted.
Values and Preferences	The greater the variability in value and preferences, or uncertainty in values and preference, the more likely a conditional recommendation is warranted.
Costs (resource allocation)	The higher the cost of an intervention, the more likely a conditional recommendation is warranted.

Box 3: GRADE-based Recommendation categories used by CATMAT

Recommendation	Rationale
StrongBox 3 Footnote * recommendation for	The committee believes that all or almost all well-informed people would want the recommended course of action and only a small number would not. Implication for practitioners: The balance of risks and benefits are such that most travellers would choose the intervention.
Strong recommendation against	The committee believes that all or almost all well-informed people would not want the recommended course of action and only a small number would. Implication for practitioners: The balance of risks and benefits are such that most travellers would not choose the intervention.
ConditionalBox 3 Footnote ** recommendation for	The committee believes that the majority of well-informed people would want the recommended course of action, but a minority (perhaps a large minority) would not. Implication for practitioners: With a conditional recommendation different travellers may make different choices. Practitioners should present the risks and benefits of the intervention and help each traveller make a decision consistent with his/her values and preferences.
Conditional recommendation against	The committee believes that the majority of well-informed people would not want the recommended course of action, but a minority (perhaps a large minority) would. Implication for practitioners: With a conditional recommendation different travellers may make different choices. Practitioners should present the risks and benefits of the intervention and help each traveller make a decision consistent with his/her values and preferences.
Box 3 Footnote * The GRADE working group specifies that if a recommendation is "strong", then it is expected that 90% or more of informed individuals would choose (or not choose) the recommended course of action. Return to box 3 footnote * referrer Box 3 Footnote ** The GRADE working group specified that if a recommendation is "conditional", then it is expected that less than 90% of informed individuals would choose (or not choose) the recommended course of action. The term "conditional" is used by CATMAT, and is considered as equivalent to "weak" as is articulated in much of the GRADE literature.  For conditional recommendations the wording "suggests" rather than "recommends" will be used. Return to box 3 footnote ** referrer

Discussion

Benefits of GRADE in the Travel Medicine Context

Applying the GRADE methodology to travel medicine recommendations may allow for improved harmonization of recommendations across countries. This is because GRADE provides a standardized and very widely accepted method of synthesizing and judging the quality of the evidence for interventions. Moreover, and crucial to travel medicine, GRADE-based methods for observational data such as baseline risk are well developed^{Footnote 13 Footnote 14}. Finally, the methodology stresses transparency and openness at each step in the guideline development process. Consequently, even if consensus on recommendations cannot be reached, applying GRADE will allow guideline developers and users to better understand how and why recommendations were made. 

A notable example of how GRADE could be used to enhance clarity in the travel medicine field is the divergent recommendations for chemoprophylaxis to prevent malaria. Currently, some national authorities recommend chemoprophylaxis for virtually all endemic areas, whereas others recommend it in much more limited geographic areas. If GRADE was applied by all travel medicine guideline developers, there may still not be agreement on malaria chemoprophylaxis recommendations, but the evidence on which recommendations were made including the risk threshold based on the values and preferences of patients would be transparently presented.

Another important benefit of applying GRADE to clinical aspects of travel medicine is the clear presentation of absolute risks and benefits. This should make it easier for practitioners to help patients make informed decisions related to use of preventive measures.

GRADE Challenges and Limitations

The GRADE methodology has several limitations. The systematic review and synthesis process is labour-intensive and there is a steep learning curve to understand GRADE evidence synthesis methodology and software. Specialist support (librarian, methodologist) can enhance efficiency and is required in most settings. Determining the risk threshold for a given intervention may be difficult and will be based on clinical experience (low confidence) in the absence of published or other evidence on patient values and preferences. There is often low- or very low-quality data available on the baseline risk of disease in travellers, which serves to reduce confidence in recommendations. Further, preventive interventions often have been assessed in endemic countries rather than with travellers, which also reduces confidence in the estimates (i.e. indirectness). Finally, the GRADE terminology and process may be unfamiliar to many practitioners who will need to learn the new classifications for data quality and then find ways to best translate the recommendations into clinical practice. 

Conclusion and Future Directions

GRADE can be used to develop travel medicine guidelines. However, there are limitations, not least of which are resource constraints. For this and other reasons, CATMAT will use GRADE in a strategic fashion. For non-GRADE recommendations, CATMAT will use a selected evidence synthesis process. EBM and guideline development will continue to evolve. CATMAT will, as is necessary, update its approach to EBM and guideline development.

Acknowledgements

• This statement was developed by the Evidence Based Medicine Working Group: Greenaway C (Chair), Schofield S, Geduld J, Guyatt G and approved by CATMAT.
• CATMAT members: McCarthy A (Chair), Acharya A, Boggild A, Brophy J, Bui Y, Crockett M, Greenaway C, Libman M, Teitelbaum P and Vaughan S.
• Liaison members: Angelo K (United States Centers for Disease Control and Prevention),  Audcent T (Canadian Paediatric Society) and Pernica J (Association of Medical Microbiology and Infectious Disease Canada).
• Ex officio members: Marion D (Canadian Forces Health Services Centre, Department of National Defence), McDonald P (Bureau of Medical Sciences, Health Canada), Rossi C (Medical Intelligence, Department of National Defence) and Schofield S (Pest Management Entomology, Department of National Defence).

Conflict of Interest

None declared.

References

Appendices

Appendix 1: Summary of the GRADE Framework for developing recommendations

Appendix 2: Typhoid example of logic model and Analytic Framework

Appendix 3: PICO and Key Questions for Typhoid Analytic Framework

• 1."PICO" (population of interest, intervention, comparison and outcome) question
  Among Canadian travellers, does the administration of typhoid vaccine versus no vaccine decrease the incidence of typhoid and the associated morbidity and mortality?

Key questions from the analytic framework,

• 2-5. What are the important risk factors for typhoid among travellers (e.g., destination, duration of travel, age, VFRs, comorbidities such as infection with HIV or acid suppression/achlorhydria)?
• 6. What is the efficacy of the typhoid vaccine
• 7. What are the harms associated with typhoid vaccination?
• 8. What are the values and preferences of travellers regarding the magnitude of risk reduction in typhoid that would make use of the vaccine worthwhile given the cost and inconvenience associated with the vaccine?

Additional contextual questions that were considered included:

• 9. What is the efficacy of antimicrobial treatment to reduce typhoid mortality and morbidity among travellers.
• 10. What are the side-effects and costs of antimicrobial treatment?
• 11. Does pathogen susceptibility to antimicrobial treatment vary by destination?         
• 12. Do hygiene and/or food and water precautions reduce the risk of acquiring typhoid among travellers?

Appendix 4:

Criteria to Judge Quality of Evidence ^{Footnote 10}

Quality of evidence	Study design	Rating down if…	Rating up if…
High	randomized study (RCT)	study limitations -1 serious -2 very serious inconsistency - 1 serious - 2 very serious indirectness -1 serious -2 very serious imprecision -1 serious -2 very serious publication bias -1 likely -2 very likely	magnitude of effect + 1 large + 2 very large dose-response gradient + 1 evidence of an application outcome relationship all plausible confounding + 1 would reduce a demonstrated effect + 1 would suggest a spurious effect when results show no effect
Moderate	n/a
Low	observational studyAppendix 4 Footnote *
Very low	n/a
Appendix 4 Footnote * For assessment of the quality of prognostic studies observational studies begin as high-quality of evidence. The same criteria approach as is depicted in this table should be applied to prognostic studies. Return to Appendix 4 footnote * referrer

Appendix 5:

   

Evidence profile for the efficacy of typhoid vaccine ^{Appendix 5 Footnote 3}

Quality assessment							No of patients		Effect		Quality
No of studies	Design	Risk of bias	Inconsistency	Indirectness	Imprecision	Other considerations	Typhoid vaccine	Control	Relative (95% CI)	Absolute
Three year cumulative risk of typhoid fever combined
4	randomised trials	no serious risk of bias	no serious inconsistency	serious	no serious imprecision	none	177/67767 (0.26%)	370/48168 (0.77%)	RR 0.51 (0.42 to 0.62)	4 fewer per 1000 (from 3 fewer to 4 fewer)	⊕⊕⊕⊚ MODERATE
Two year cumulative risk of typhoid fever combined
4	randomised trials	no serious risk of biasAppendix 5 Footnote 1	no serious inconsistency	seriousAppendix 5 Footnote 2	no serious imprecision	none	100/59969 (0.17%)	237/60395 (0.39%)	RR 0.43 (0.34 to 0.54)	2 fewer per 1000 (from 2 fewer to 3 fewer)	⊕⊕⊕⊚ MODERATE
Three year cumulative risk of typhoid fever Ty21a
3	randomised trials	no serious risk of biasAppendix 5 Footnote 1	no serious inconsistency	seriousAppendix 5 Footnote 2	no serious imprecision	none	147/62075 (0.24%)	304/42476 (0.72%)	RR 0.53 (0.43 to 0.65)	3 fewer per 1000 (from 3 fewer to 4 fewer)	⊕⊕⊕⊚ MODERATE
Two year cumulative risk of typhoid fever Ty21a
1	randomised trials	no serious risk of biasAppendix 5 Footnote 3	no serious inconsistency	seriousAppendix 5 Footnote 2	no serious imprecision	none	15/22170 (0.07%)	44/21906 (0.2%)	RR 0.34 (0.19 to 0.61)	1 fewer per 1000 (from 1 fewer to 2 fewer)	⊕⊕⊕⊚ MODERATE
Three year cumulative risk of typhoid fever Vi polysaccharide
1	randomised trials	no serious risk of bias	no serious inconsistency	seriousAppendix 5 Footnote 2	no serious imprecision	none	30/5692 (0.53%)	66/5692 (1.2%)	RR 0.45 (0.30 to 0.7)	6 fewer per 1000 (from 3 fewer to 8 fewer)	⊕⊕⊕⊚ MODERATE
Two year cumulative risk of typhoid fever Vi polysaccharide
3	randomised trials	no serious risk of bias	no serious inconsistency	seriousAppendix 5 Footnote 2	no serious imprecision	none	85/37799 (0.22%)	193/38489 (0.5%)	RR 0.45 (0.35 to 0.58)	3 fewer per 1000 (from 2 fewer to 3 fewer)	⊕⊕⊕⊚ MODERATE
Cumulative risk of typhoid fever by age Ty21a
2	randomised trials	no serious risk of biasAppendix 5 Footnote 1	no serious inconsistency	seriousAppendix 5 Footnote 2	no serious imprecision	none	86/56864 (0.15%)	98/32206 (0.3%)	RR 0.45 (0.33 to 0.63)	2 fewer per 1000 (from 1 fewer to 2 fewer)	⊕⊕⊕⊚ MODERATE
Cumulative risk of typhoid fever by age Ty21a - Age 5-<10
2	randomised trials	no serious risk of biasAppendix 5 Footnote 1	no serious inconsistency	seriousAppendix 5 Footnote 2	no serious imprecision	none	54/28162 (0.19%)	40/13182 (0.3%)	RR 0.61 (0.39 to 0.97)	1 fewer per 1000 (from 0 fewer to 2 fewer)	⊕⊕⊕⊚ MODERATE
Cumulative risk of typhoid fever by age Ty21a - Age ≥ 10
2	randomised trials	no serious risk of biasAppendix 5 Footnote 1	no serious inconsistency	seriousAppendix 5 Footnote 2	no serious imprecision	none	32/28702 (0.11%)	58/19024 (0.3%)	RR 0.34 (0.21 to 0.54)	2 fewer per 1000 (from 1 fewer to 2 fewer)	⊕⊕⊕⊚ MODERATE
Cumulative risk of typhoid fever by age Vi polysaccharide
2	randomised trials	no serious risk of bias	no serious inconsistency	seriousAppendix 5 Footnote 2	no serious imprecision	none	64/32107 (0.2%)	145/32797 (0.44%)	RR 0.45 (0.34 to 0.6)	2 fewer per 1000 (from 2 fewer to 3 fewer)	⊕⊕⊕⊚ MODERATE
Cumulative risk of typhoid fever by age Vi polysaccharide - Age 2-<5
2	randomised trials	no serious risk of bias	no serious inconsistency	seriousAppendix 5 Footnote 2	no serious imprecision	none	21/4251 (0.49%)	40/4419 (0.91%)	RR 0.54 (0.32 to 0.91)	4 fewer per 1000 (from 1 fewer to 6 fewer)	⊕⊕⊕⊚ MODERATE
Cumulative risk of typhoid fever by age Vi polysaccharide - Age ≥ 5
2	randomised trials	no serious risk of bias	no serious inconsistency	seriousAppendix 5 Footnote 2	seriousAppendix 5 Footnote 4	none	43/27856 (0.15%)	105/28378 (0.37%)	RR 0.42 (0.29 to 0.59)	2 fewer per 1000 (from 2 fewer to 3 fewer)	⊕⊕⊚⊚ LOW
Appendix 5 Footnote 1 Two studies did not correct for cluster design; this was judged not to present a serious risk of bias. Return to appenidx 5 footnote 1 referrer Appendix 5 Footnote 2 Studies with endemic populations, generally children; not with travellers. For Ty21a data based on three not four doses of vaccine. Return to appenidx 5 footnote 2 referrer Appendix 5 Footnote 3 Study did not correct for cluster design, this was judged not to present a serious risk of bias. Return to appenidx 5 footnote 3 referrer Appendix 5 Footnote 4 Heterogeneity in estimates of efficacy across studies. Return to appendix 5 footnote 4 referrer

Appendix 6:

Summary of findings for the efficacy of typhoid vaccine^{Appendix 6 Footnote 3}

Outcomes	Illustrative comparative risksAppendix 6 Footnote * (95% CI)		Relative effect (95% CI)	No of Participants (studies)	Quality of the evidence (GRADE)
	Assumed risk	Corresponding risk
	Control	Typhoid vaccine
Three year cumulative risk of typhoid fever combined	8 per 1000	4 per 1000 (3 to 5)	RR 0.51 (0.42 to 0.62)	115935 (4 studies)	⊕⊕⊕⊚ moderate
Two year cumulative risk of typhoid fever combined	4 per 1000	2 per 1000 (1 to 2)	RR 0.43 (0.34 to 0.54)	120364 (4 studies)	⊕⊕⊕⊚ moderateAppendix 6 Footnote 1 Appendix 6 Footnote 2
Three year cumulative risk of typhoid fever Ty21a	7 per 1000	4 per 1000 (3 to 5)	RR 0.53 (0.43 to 0.65)	104551 (3 studies)	⊕⊕⊕⊚ moderateAppendix 6 Footnote 1 Appendix 6 Footnote 2
Two year cumulative risk of typhoid fever Ty21a	2 per 1000	1 per 1000 (0 to 1)	RR 0.34 (0.19 to 0.61)	44076 (1 study)	⊕⊕⊕⊚ moderateAppendix 6 Footnote 2 Appendix 6 Footnote 3
Three year cumulative risk of typhoid fever Vi polysaccharide	12 per 1000	5 per 1000 (3 to 8)	RR 0.45 (0.30 to 0.7)	11384 (1 study)	⊕⊕⊕⊚ moderateAppendix 6 Footnote 2
Two year cumulative risk of typhoid fever Vi polysaccharide	5 per 1000	2 per 1000 (2 to 3)	RR 0.45 (0.35 to 0.58)	76288 (3 studies)	⊕⊕⊕⊚ moderateAppendix 6 Footnote 2
Cumulative risk of typhoid fever by age Ty21a	3 per 1000	1 per 1000 (1 to 2)	RR 0.45 (0.33 to 0.63)	89070 (2 studies)	⊕⊕⊕⊚ moderateAppendix 6 Footnote 1 Appendix 6 Footnote 2
Cumulative risk of typhoid fever by age Ty21a - Age 5-<10	3 per 1000	2 per 1000 (1 to 3)	RR 0.61 (0.39 to 0.97)	41344 (2 studies)	⊕⊕⊕⊚ moderateAppendix 6 Footnote 1 Appendix 6 Footnote 2
Cumulative risk of typhoid fever by age Ty21a - Age ≥ 10	3 per 1000	1 per 1000 (1 to 2)	RR 0.34 (0.21 to 0.54)	47726 (2 studies)	⊕⊕⊕⊚ moderateAppendix 6 Footnote 1 Appendix 6 Footnote 2
Cumulative risk of typhoid fever by age Vi polysaccharide	4 per 1000	2 per 1000 (1 to 3)	RR 0.45 (0.34 to 0.6)	64904 (2 studies)	⊕⊕⊕⊚ moderateAppendix 6 Footnote 2
Cumulative risk of typhoid fever by age Vi polysaccharide - Age 2-<5	9 per 1000	5 per 1000 (3 to 8)	RR 0.54 (0.32 to 0.91)	8670 (2 studies)	⊕⊕⊕⊚ moderateAppendix 6 Footnote 2
Cumulative risk of typhoid fever by age Vi polysaccharide - Age ≥ 5	4 per 1000	2 per 1000 (1 to 2)	RR 0.42 (0.29 to 0.59)	56234 (2 studies)	⊕⊕⊚⊚ lowAppendix 6 Footnote 2 Appendix 6 Footnote 4
GRADE Working Group grades of evidence High quality: Further research is very unlikely to change our confidence in the estimate of effect. Moderate quality: Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate. Low quality: Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate. Very low quality: We are very uncertain about the estimate. Appendix 6 Footnote * The basis for the assumed risk (e.g. the median control group risk across studies) is provided in footnotes. The corresponding risk (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI). CI: Confidence interval; RR: Risk ratio Return to Appendix 6 footnote * referrer Appendix 6 Footnote 1 Two studies did not correct for cluster design; this was judged not to present a serious risk of bias. Return to Appendix 6 footnote 1 referrer Appendix 6 Footnote 2 Studies with endemic populations, generally children; not with travellers. For Ty21a data based on three not four doses of vaccine. Return to Appendix 6 footnote 2 referrer Appendix 6 Footnote 3 Study did not correct for cluster design, this was judged not to present a serious risk of bias. Return to Appendix 6 footnote 3 referrer Appendix 6 Footnote 4 Heterogeneity in estimates of efficacy across studies. Return to Appendix 6 footnote 4 referrer

Appendix 7:

Evidence profile for assessment of confidence in estimates of baseline risk of typhoid by region^{Appendix 7 Footnote 3}

No of studies	DesignAppendix 7 Footnote 1	Risk of bias	Inconsistency	Indirectness	ImprecisionAppendix 7 Footnote 2	Other considerations	Estimate (cases/ 100,000)	Confidence in baseline estimate
Eastern Mediterranean
1	Observational cohort	intermediate risk of bias	no serious inconsistency	no serious indirectness	intermediate risk of imprecision	none	0.09	⊕⊕⊕⊚ MODERATE
North Africa/Middle East
3	Observational cohort	intermediate risk of bias	no serious inconsistency	no serious indirectness	intermediate risk of imprecision	none	1.48	⊕⊕⊕⊚ MODERATE
Sub-Saharan Africa
3	Observational cohort	intermediate risk of bias	no serious inconsistency	no serious indirectness	intermediate risk of imprecision	none	1.84	⊕⊕⊕⊚ MODERATE
Caribbean, Central America
3	Observational cohort	intermediate risk of bias	no serious inconsistency	no serious indirectness	intermediate risk of imprecision	none	0.33	⊕⊕⊕⊚ MODERATE
South America
2	Observational cohort	intermediate risk of bias	no serious inconsistency	no serious indirectness	intermediate risk of imprecision	none	0.95	⊕⊕⊕⊚ MODERATE
East Asia
3	Observational cohort	intermediate risk of bias	no serious inconsistency	no serious indirectness	intermediate risk of imprecision	none	0.25	⊕⊕⊕⊚ MODERATE
South Asia
4	Observational cohort	intermediate risk of bias	no serious inconsistency	no serious indirectness	intermediate risk of imprecision	none	32.73	⊕⊕⊕⊚ MODERATE
Overall Risk of Bias:  Moderate Selection Bias:  Low:  All included studies used mandatory infectious disease surveillance data of the reported cases of typhoid in the province or country as the numerator and a population based estimate of the travelling population for the same region as the denominator.  Outcome Measurement:  Low: All studies only included cases of typhoid from clinical specimens that were laboratory confirmed  Outcome Ascertainment:  Moderate:  Potential biases in surveillance data are only the most severe cases seek medical attention and have the opportunity to be diagnosed.  Furthermore different populations may have different health seeking behavior and some cases may have been missed due to diagnosis and treatment during travel.  Appendix 7 Footnote 1 Observation studies start as high quality. Return to footnote 1 referrer Appendix 7 Footnote 2 For all destinations, intermediate risk of imprecision because estimates of error (e.g., confidence limits) could not be derived across the body of evidence. Return to footnote 2 referrer Appendix 7 Footnote 3 Study did not correct for cluster design, this was judged not to present a serious risk of bias. Return to footnote 3 referrer