Evaluation of Community Associated Infections Prevention and Control Activities at the Public Health Agency of Canada

Prepared by
Evaluation Directorate
Public Health Agency of Canada/Health Canada

February 2013

Table of contents

Table of figures

Executive summary

This report presents the findings of an evaluation of prevention and control of community associated infections activities at the Public Health Agency of Canada.

Evaluation purpose

This evaluation was conducted to support program planning and decision making. Evaluative information was also needed to assess funding received to address hepatitis C in Canada. The evaluation covers activities undertaken between 2008 and 2012.

The evaluation does not take into consideration activities related to the National Microbiology Laboratory, the Canadian HIV Vaccine Initiative, the Federal Initiative to Address HIV/AIDS, the Quarantine Program and the Travel Health Program. It also excludes healthcare associated infections activities that may be covered by the Centre for Communicable Diseases and Infection Control at the Public Health Agency. These activities will be assessed in upcoming evaluations.

Description of activities

The Public Health Agency defines ‘community associated infections’ as communicable diseases transmitted person-to-person in the general community context (as opposed to health care settings). This would include hepatitis B and C, sexually transmitted infections such as syphilis, gonorrhea and chlamydia, HIV/AIDS, and tuberculosis. As mentioned above, the Public Health Agency’s activities in the Federal Initiative to Address HIV/AIDS will be covered in a separate evaluation.

There are many players addressing community associated infections in Canada. Provinces and territories, other government departments, local community groups as well as academics and researchers all have key roles in preventing and controlling these types of infections.

Within the Public Health Agency, work to prevent and control ‘community associated infections’ is led by the Centre for Communicable Diseases and Infection Control. The mandate of the Centre for Communicable Diseases and Infection Control is to create and share credible knowledge and facilitate coherent national action which contributes to the prevention and control of specific communicable disease, with a focus on key populations at risk.

This evaluation examines the community associated infections role and activities of the Centre for Communicable Diseases and Infection Control at the Public Health Agency. The Agency allocated about $12.1 million for these activities in 2012-13.

Evaluation findings, conclusions and recommendations

The synthesis and analysis of the findings from this evaluation resulted in conclusions about both relevance and performance. In turn, this led to three recommendations for Public Health Agency senior management.

Findings and conclusions

Relevance

The need to prevent and control community associated infections (namely hepatitis B and C, sexually transmitted infections, and tuberculosis) still exists. Economies can be achieve and quality of life can be improved by targeting prevention and control activities towards common populations, risk factors and social conditions underlying each disease. Disease-specific work is beneficial in many circumstances, for example, when distinct target groups exist and in areas such as public health practice, surveillance (without precluding the importance of integrated surveillance systems addressing issues of coinfection), vaccine development and biomedical research.

Infectious disease prevention and control is an integral part of the Public Health Agency’s role and mandate. With the exception of a few areas such as infection control at the Canadian border, the majority of this mandate is complementary and shared with provincial/territorial governments and other public health stakeholders in Canada. As such, the Public Health Agency’s role must provide added value within a system of multiple players. Key to this role is providing a national picture of infectious disease in Canada, aligning and coordinating national efforts along provincial and territorial directions and priorities, and filling important gaps in research, knowledge and program or policy directions that would not otherwise be addressed.

Infection control is a priority for the Public Health Agency, and is encompassed within all three of the Public Health Agency’s strategic directions for the next five years, as per corporate strategic documents: leadership on health promotion, disease prevention, and health protection; strengthened public health capacity and science leadership; and enhanced emergency preparedness and response. Within these strategic directions, tuberculosis and sexually transmitted infections (along with antimicrobial resistance) were highlighted as priorities by most internal key informants and provincial and territorial partners.

Performance

Achieving an appropriate level of collaboration with provinces, territories and stakeholders is a key outcome of the community associated infections activities in the Centre for Communicable Diseases and Infection Control. Increased awareness on the state, incidence, trends of communicable diseases and specific public health communicable disease practices in Canada as well as the increased use and application of the evidence, knowledge and best practices are also key outcomes.

The evaluation found that the Public Health Agency works collaboratively with provinces and territories as well as some stakeholders in their activities to prevent and control community associated infections.  Mechanisms are in place to share information, to collaborate through working groups, reviewing guides and providing input. Experts in the area are engaged to develop guides.

The Agency has also encouraged partnerships through funded community projects seeking to address hepatitis C locally. Partnerships were fruitful in helping funding recipients achieve their project objectives more efficiently (i.e. using fewer resources) and more effectively (i.e. having greater impact).

There are also indications that Public Health Agency products increase knowledge in the area of community associated infections and are being used by partners and stakeholders. Assessments of some tuberculosis, sexually transmitted infections, sexual health and hepatitis C products demonstrate that there is increased knowledge after usage. However, there are also signs that increased knowledge could be at a threshold for some community-based projects.

There is documented evidence of the use of Agency products. For example, when the Tuberculosis online modules were evaluated, doctors responded to questions on how they will change their practice. Several groups reported that they would be more vigilant in screening and are more aware of risk groups. Medical schools have used the Sexually Transmitted Infections guidelines/online modules for instructional purposes. Key informants also noted that Public Health Agency products have been used for instruction, for development of guidelines in other countries and to improve public health practice.

However, there are opportunities for improvement. The Public Health Agency could have extended its collaboration network with specific stakeholders and provided national information in a timely fashion, which may have impacted the usefulness of some Agency products. In addition, there is limited performance measurement in place to help the Centre for Communicable Diseases and Infection Control monitor progress and assist in programmatic decision making.

Recommendations

Collaboration

The Public Health Agency shares jurisdiction over public health and health promotion with numerous stakeholders. Each player has a different role and makes a unique contribution to the overall Canadian effort. Through collaboration, the Public Health Agency can reduce duplication of effort and maximize complementarity of programming.

To prevent and control community associated infections, the Centre for Communicable Diseases and Infection Control’s primary partners are: other federal government departments, provincial and territorial governments, non-governmental organizations, researchers and academics, and international agencies. Collaboration tends to be working well with provinces and territories as well as some key internal partners, which is important during an outbreak. However, collaboration with other partners could be better sustained. While there were indications of some positive activities with other government departments, some survey respondents, including provinces and territories, stated that the Agency could be more effective and foster greater engagement with other government departments. Similar sentiments were expressed by non-governmental organization stakeholders, international partners and academics.

Recognizing that there are many potential ways to achieve results in addressing community associated infections, the Centre for Communicable Diseases and Infection Control will need to focus on collaborative activities that further advance current Public Health Agency priorities.

RECOMMENDATION 1

Review the Public Health Agency’s collaborative activities in community associated infections.  Where gaps in achieving Public Health Agency outcomes are identified, develop an appropriate response to enhance coherence in prevention and control activities.

Knowledge translation

Community associated infections activities and outputs aim to achieve increased awareness on the state, trends, prevention and control public health practices for communicable disease in Canada. Direct outputs of Public Health Agency activities include but are not limited to: guidelines for health professionals and decision makers, professional desk references, surveillance reports, brochures and other public awareness products, training materials and tools, publications and research outputs.

In the past five years, the prevention work in the field of sexually transmitted infection conducted by Centre for Communicable Diseases and Infection Control has been mainly about knowledge development, translation and exchange. It appears as though the knowledge products produced by the Centre are appropriate. They appear to address the current and emerging needs, such as antimicrobial resistance. Survey respondents and key informants did not provide any indication that the Public Health Agency was not developing products to meet their needs.

However, partners and stakeholders noted that timeliness was a key obstacle in using the Public Health Agency’s products. The evaluation found that timeliness was impacting the Public Health Agency’s ability to be relevant in the Agency’s community associated infections role. 

While the Centre for Communicable Diseases and Infection Control has taken steps to ensure that some products are available to stakeholders in a more timely fashion, standards, protocols or procedures should be developed to ensure products are generally available when needed.

RECOMMENDATION 2

Review internal processes to improve the timeliness of community associated infection product distribution. Where needed, develop and implement standards to address identified gaps.

Performance measurement

The first step in a performance measurement study is to determine whether or not the program was implemented according to its proposed theory of change (typically represented by the program or activity’s logic model). The measurement and assessment of the Centre for Communicable Diseases and Infection Control’s progress towards the achievement of its intended outcomes appears to be inconsistent and unsystematic. As such, the Centre can speak to several outcome studies demonstrating success and areas for improvement, but is unable to frame its work into a comprehensive story of what it has achieved.

There has been systematic tracking of both outputs and outcomes of project work funded through the hepatitis C grants and contribution program. However, despite the national roll-out of this tool over three full fiscal years, the data was never rolled up into a national report, and there is no evidence that the information was used to inform program or policy decisions at the national level. There were a number of factors that contributed to this situation. Regardless, while the information was collected and available, it was never used to measure, assess and adjust performance for the overall hepatitis C program.

There are promising activities currently in the works which hint at the development of a more comprehensive performance measurement culture within the Centre for Communicable Diseases and Infection Control. The Centre is beginning to develop tools and methodologies to measure and assess performance. While these early developments are positive indications of a more effective performance measurement function with the Centre, future work must be comprehensive in its approach (documenting outputs and outcomes of both direct and grants and contributions activities), grounded in a clear theory of change, systematically integrated into operations, and then actively used in management practices to assess and adjust performance.

RECOMMENDATION 3

Develop a comprehensive and feasible performance measurement system to help support programmatic decision making.

Figure 1: Findings, conclusions and recommendations
  Findings Conclusions Recommendations
Relevance
Community associated infections affect many Canadians, especially vulnerable populations

The Agency’s mandate in preventing and controlling community associated infections is generally clear. While some duplication of activities occurs, the Agency has a leadership role in this area, which is expected and appreciated by the Public Health Agency’s partners.

  1. Review the Public Health Agency’s collaborative activities in community associated infections. Where gaps in achieving Public Health Agency outcomes are identified, develop an appropriate response  to enhance coherence in prevention and control activities.
  2. Review internal processes to improve the timeliness of community associated infection product distribution. Where needed, develop and implement standards to address identified gaps.
  3. Develop a comprehensive and feasible performance measurement system to help support programmatic decision making.
The prevention and control of infectious disease is a priority for the Government of Canada as well as the Public Health Agency 
There is a federal public health role in the prevention and control of community associated infections
Work plans reflect community associated infections needs, federal public health roles and Public Health Agency priorities
Performance
There is evidence, while limited, that Agency products led to an increase in community associated infections knowledge

The evaluation found that the Public Health Agency works collaboratively with provinces and territories as well as some stakeholders in their activities to prevent and control community associated infections. Mechanisms are in place to share information, to collaborate through working groups, reviewing guidelines and providing input. Guides are developed through the engagement of experts in the area.

There are indications that products lead to increased knowledge and are being used to prevent and control community associated infections in Canada.

However, there are always opportunities for improvement. The Public Health Agency could have extended its collaborative networks with specific stakeholders and provided national information in a timely fashion, which may have impacted the usefulness of the Agency’s products. In addition, there is limited performance measurement in place to help the Centre for Communicable Diseases and Infection Control monitor progress and assist in programmatic decision making.

Public Health Agency funded hepatitis C projects had some success in achieving their knowledge and awareness objectives, although it may have plateaued in recent years
There is evidence that some Public Health Agency products are used by public health professionals to enhance training and practice
Timeliness is a key challenge that limits stakeholders’ use of Agency products
Projects funded through the hepatitis C grants and contributions demonstrated use of evidence through improved access to health services, changes in practice and policy, and changes in individual behaviour
Collaboration activities are important to infectious disease work in Canada, and the Centre for Communicable Diseases and Infection Control’s collaboration activities are well perceived by some stakeholders
Collaboration activities have recently declined with other partners and stakeholders although the impact of this decline is not readily evident
The Public Health Agency encourages partnerships through hepatitis C grant and contribution funding with reported beneficial effects
Although some performance data are collected, there is limited  performance measurement information available for programmatic decision making
Leveraging of volunteer hours led to efficiencies for hepatitis C funding
Report a problem or mistake on this page
Please select all that apply:

Thank you for your help!

You will not receive a reply. For enquiries, contact us.

Date modified: