Audit of Surveillance Activities

October 2013

Table of Contents

Executive summary

1. Introduction
   1. Background
   2. Audit objective
   3. Audit scope
   4. Audit approach
   5. Statement of conformance
2. Findings, recommendations and management responses
   1. Governance
      1.1 Strategic direction
      1.2 Roles and responsibilities
      1.3 Performance measurement
   2. Risk management
      2.1 Risk management
   3. Internal controls
      3.1 Planning
      3.2 Data management
      3.3 Privacy practices
      3.4 Performance measurement
3. Conclusion

• Appendix A – Lines of enquiry and criteria
• Appendix B – Scorecard
• Appendix C – Surveillance systems

Executive summary

The focus of the audit was on surveillance activities. Surveillance is a core public health function for the Public Health Agency of Canada (Agency). Effective and timely surveillance is critical to the ability of the government and provinces/territories to accurately track health information, and plan for, and respond to public health events.

The objective of the audit was to determine the effectiveness of the management control framework for surveillance to allow relevant, timely and accurate public health data to support decision making. The audit examined governance, risk management and performance measurement for surveillance activities across the Agency and tested the internal controls on a sample of selected surveillance activities for fiscal year 2012–13.

Over the last year there has been a well–engaged, Agency–wide focus on surveillance activities. The Agency has developed, and is in the process of implementing, a 3–year surveillance strategic plan (2013–2016). The Agency has appointed a Chief Health Surveillance Officer to advise and guide the corporate approach. The goal will be streamlined, effective and fiscally responsive Agency management and practice of public health surveillance. To date the governance model is established and will be enhanced over the year to incorporate a decision–making model which will aim to ensure that the Agency's surveillance commitments are aligned to public health priorities as well as Agency and Government of Canada priorities.

The Agency has identified and assessed the risks associated with surveillance activities overall. The mitigating strategy to address the risks is contained in the PHAC Surveillance Strategic Plan (2013–2016). These risks will be reduced (and in some instances mitigated) with the full implementation of the plan.

The plan calls for a surveillance performance measurement framework. The Surveillance performance measurement framework has been developed and approved and provides for a consistent process for surveillance performance measurement. It is intended to identify areas for improvement in surveillance practice and management, and should contribute to the Agency's Performance Measurement Framework. The audit notes that it would be important to have key inputs related to the investment in surveillance included in the performance measurement framework.

The audit sampled 12 surveillance activities, some of which included both an epidemiology and laboratory component, in the Infectious Diseases Prevention and Control Branch. The audit requested information related to: surveillance rationale; objectives and expected results; planned activities, milestones and timing; planned resources (financial and human resources); and planned outputs (and associated timing) of products. The results of the analysis noted that only a few of the surveillance activities are well planned and managed (Nosocomial Infection Surveillance Program) while there are others where basic information was unavailable (Transfusion Transmitted Injuries Surveillance System). As well, most surveillance data holdings sampled have not yet been reviewed for privacy requirements. Lastly, data quality assessments (used to determine data management capabilities) were completed for 5 of 12 surveillance activities examined.

As the strategic surveillance plan is implemented it will be important for surveillance managers to support the initiative through: the timely and objective completion of the performance measurement self–assessment; the timely completion of the data quality assessments; and assessing privacy requirements.

Management has agreed with the 3 recommendations and provided a detailed action plan that, once implemented, will strengthen the effectiveness of the management control framework supporting the delivery of surveillance activities within the Agency.

A – Introduction

1. Background

Surveillance is a core public health function for the Public Health Agency of Canada (Agency). The Agency defines surveillance as the tracking and forecasting of any health event or health determinant through the on–going collection of data, the integration, analysis and interpretation of data into surveillance products, and the dissemination of those surveillance products to those who need to know the situation to undertake necessary actions or responses. The overarching purpose of surveillance is to generate information and knowledge for public health action in the short and long term.

Surveillance Uses

1. Detecting potential outbreaks and threats to public health
2. Detecting cases for intervention
3. Monitoring trends in health events
4. Directing public health interventions
5. Guiding decision making and action to reduce morbidity and mortality and to improve health through an emphasis on primary and secondary prevention
6. Guiding planning, implementation and evaluation of public health programs
7. Providing a basis for epidemiological research

Effective and timely surveillance is critical to the ability of the federal, provincial and territorial governments to accurately track health information and plan for, and respond to public health events. The Agency's surveillance and population health assessment initiatives enable many partners and stakeholders to better collect, analyze and interpret data; track and forecast public health events; and enhance support to health practitioners and decision makers. According to the Agency's 2013–2016 Strategic Surveillance Plan, a successful system for surveillance requires a combination of strong and flexible systems with knowledgeable people who can use them; effective data collection and sharing across jurisdictions, expert analysis and communication of findings (see Appendix B).

The expected outcomes of the public health surveillance function are information and knowledge, which in turn inform public health policy and guide action to improve public health outcomes.

The Government of Canada and the Public Health Agency of Canada have a mandate and role in public health surveillance. Various acts, including the Department of Health Act and the Public Health Agency of Canada Act confer authority on the federal government to carry out public health surveillance. Under the World Health Organization (WHO)'s International Health Regulations, Canada also has an obligation to carry out surveillance.

Surveillance–related systems are spread across all major program areas and are applied to a range of health issues including communicable diseases, chronic diseases and injuries, healthy child development and environmental health. The Agency has approximately 53 surveillance systems/programs (see Appendix C). For fiscal year 2011–2012, the expenditures related to these surveillance systems were estimated as $62M^{Footnote 1} which is largely divided between two branches: the Infectious Disease Prevention and Control Branch and the Health Promotion and Chronic Disease Prevention Branch.

Infectious Disease Prevention and Control Branch

($152.7M including $36.2M for surveillance)

The Infectious Disease Prevention and Control Branch (IDPCB) is the national and international focal point within the Agency for infectious diseases as it relates to public health. Its mandate is to lead efforts that promote health and enable the prevention and control of infectious diseases, contribute to the improvement in the health of those infected and support infectious disease health policy. This is carried out in collaboration with provinces and territories, other federal departments and other national and international stakeholders.

The Branch has 5 broad goals: develop targeted prevention and control initiatives; strengthen the national public health system and build capacity; enhance national surveillance of infectious diseases; establish a comprehensive knowledge transfer and communication plan and strengthen management processes to position IDPC for the future. These 5 goals are delivered organizationally by three Centres and two Laboratories. (See Appendix C)

Health Promotion and Chronic Disease Prevention Branch

($257.4M including $25.4M for surveillance)

The Health Promotion and Chronic Disease Prevention Branch works in conjunction with stakeholders at all levels to provide guidance and leadership at the national and international levels on health promotion and chronic disease prevention, surveillance and control. Chronic diseases are those conditions that are generally incurable, are often caused by a complex interaction of factors, and have a prolonged clinical course.

Conducting surveillance on major risk factors and determinants of chronic diseases that can be modified gives the information needed to inform sound and effective health policy that in turn leads to effective control and prevention of major chronic diseases in the population. This branch operates the Centre for Chronic Disease Prevention (see Appendix C).

2. Audit objective

The objective of the audit was to determine the effectiveness of the management control framework for surveillance within the Agency, and to determine that the surveillance activity is governed and managed to provide relevant, timely and accurate public health data to support decision making.

3. Audit scope

The audit took an Agency–wide perspective in relation to surveillance governance, risk management and performance measurement. To examine internal controls related to the surveillance activity, the audit focused on selected surveillance systems in the Infectious Disease Prevention and Control Branch (IDPC) for fiscal year 2012–13. IDPC surveillance activities are ranked by public health experts as having the highest priority for national and international reporting. IDPC also expends more in surveillance funds ($36.2M 2011–12) than any other Branch and has the majority of the surveillance activities (38) spread across three Centres and two laboratories (see Appendix C).

4. Audit approach

The audit approach included a review of documentation, policies, standards, guidelines and frameworks; interviews and observation; inquiry, testing and analysis. The audit included an assessment of management controls related to a sample of surveillance systems (see Appendix C), selected on the basis of their resource investment, complexity of stakeholder interaction (for example, provincial/territorial involvement), and complexity of internal operations (for example, surveillance activities delivered via multiple Agency stakeholders).

More specifically, the audit included an assessment of Agency mechanisms and processes to effectively manage the surveillance activities and the supporting systems. Specifically, the audit assessed the effectiveness of the management control framework for selected surveillance systems including the adequacy of procedures, systems, tools and controls in place. In relation to the management control framework, the audit included as assessment of overall Agency approaches in relation to governance, risk management and performance measurement considerations. For internal management control considerations (for example, operational planning), based on the audit risk assessment, the audit focused on a sample of surveillance activities, provided at Appendix C.

The audit noted that an evaluation was conducted recently and tabled in January 2013. This evaluation examined the surveillance function at the Agency, however, it did not involve examination of the relevance or performance of individual surveillance activities designed to support programs (such as HIV/AIDS). Accordingly, this audit included an assessment of management control considerations related to the aforementioned sample of specific surveillance activities. The audit did not examine emergency preparedness or the management of outbreaks.

The audit criteria, outlined in Appendix A, were derived from the Office of the Comptroller General Internal Audit Sector's Audit Criteria Related to the Management Accountability Framework: A Tool for Internal Auditors (March 2011), the Public Health Agency of Canada 2013–2016 Surveillance Strategic Plan, the World Health Organization, Canadian Institute for Health Information (CIHI)'s Data Quality Framework (2009), the Communications Policy of the Government of Canada (2006), the Treasury Board (TB)'s Policy on Privacy Protection (2008) and the WHO's International Health Regulations (2005).

5. Statement of conformance

In the professional judgment of the Chief Audit Executive, sufficient and appropriate procedures were performed and evidence gathered to support the accuracy of the audit conclusion. The audit findings and conclusion are based on a comparison of the conditions that existed as of the date of the audit, against established criteria that were agreed upon with management. Further, the evidence was gathered in accordance with the Internal Auditing Standards for the Government of Canada and the International Standards for the Professional Practice of Internal Auditing. The audit conforms to the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program.

B – Findings, recommendations and management responses

1. Governance

1.1 Strategic direction

Audit criterion: The Agency sets the strategic direction for the surveillance activity.

The Agency has an approved 2013–2016 PHAC Surveillance Strategic Plan (more commonly referred to as the PSSP2). The Plan was approved by Executive Committee (the highest decision–making body) in December 2012. The strategic plan is intended to guide the Agency as it renews the planning, alignment, and delivery of its public health surveillance activities. More specifically the Plan details four core objectives: develop a coherent system for public health surveillance; establish a corporate model for surveillance decision making and strategic alignment of surveillance activities; build a corporate model for data management and technology to support surveillance; establish the foundations for an integrated public health surveillance framework in Canada.

The strategic plan articulates a renewed surveillance governance structure which has also been active in supporting the implementation of the strategic plan. To date, significant progress has been made in establishing an "advisory structure" to support Agency decision making related to surveillance and to support better communication of surveillance activities. This structure includes: the Executive Committee; the Science, Policy and Management Committee; and the Surveillance Integration Team.

The Executive Committee is the Agency's most senior decision making body (Tier 1). The committee is chaired by the Deputy Head and is attended by senior management. The committee meets weekly. In that regard, key items related to surveillance receive final consideration and approval by the committee. A review of minutes and agendas note that key surveillance items such as the 2013–2016 PHAC Surveillance Strategic Plan are discussed and when appropriate approved.

Science Policy and Management Committee (Tier 2) receives items related to science policy and science management (including scientific research, surveillance and monitoring related matters). This Committee is composed of seniors with specific science policy and management expertise. For example, the Committee reviews items related to surveillance and provides advice where required and recommends items for Executive Committee approval. The Chief Health Surveillance Officer is a member of the Committee.

Surveillance Integration Team (Tier 3) is chaired by the Surveillance and Risk Assessment Division and is the focal point for technical and strategic discussions about surveillance issues within the Agency. The team also provides functional advice and support for the implementation of the PHAC Surveillance Strategic Plan (PSSP2).

The Agency has made progress through the establishment of a formalized surveillance governance structure, to ensure that mechanisms and processes are in place for the discussion and communication of surveillance information and issues, and to support public health policy, decision–making, and action.

While this has been a good first step towards bringing a horizontal approach to the Agency's core business of surveillance, work on several cross–Agency initiatives has reinforced the need to bring further coherence to the Agency's surveillance activities. In addition to the renewed governance structure, the Surveillance Strategic Plan includes an initiative currently under development focused on development and implementation of a strategic surveillance decision making process (see recommendation 1).

The aim of these efforts is to better equip the Agency for strategic and operational planning and decision making. Management also anticipates that these changes will support better integration of surveillance activities within the Agency by ensuring that public health surveillance activities are flexible and responsive to public health program, policy and practice information needs.

The Strategic Plan (Part I) is supported by a detailed Implementation Plan (Part II) that identifies the key initiatives, activities, milestones, and timelines including a responsible lead and key partners for each activity. Full implementation is anticipated to be completed in 2015. It was also noted that progress towards its deliverables are routinely monitored and communicated.

1.2 Roles and responsibilities

Audit criterion: The roles and responsibilities between the Chief Surveillance Officer and the branches are clearly articulated as they relate to surveillance.

Surveillance requires multiple collaborations involving diverse networks and partnerships; varying methodologies and approaches for chronic, infectious disease and laboratory surveillance; and clear linkages with the Agency's other enabling functions, such as research and health equity. In support of this complex environment, it is paramount to have internal roles and responsibilities well documented, communicated and understood.

The 2013–2016 PHAC Surveillance Strategic Plan focuses on defining clear corporate roles and responsibilities for the: Chief Health Surveillance Officer; the Surveillance Coordination Unit; Data Coordination and Access Program; Subject Matter Expert Reference Group and Surveillance Programs/System Managers.

The Chief Health Surveillance Officer (CHSO), reporting to the Branch Head of Health Security Infrastructure Branch, is responsible for overseeing, monitoring and evaluating the PSSP2's timely implementation. The CHSO is to monitor and report quarterly on progress and will evaluate the Implementation Plan process and progress at the Plan's mid–point in 2014 and on its conclusion in 2016. As appropriate, the CHSO is to brief executive management on PSSP2 and Agency–wide surveillance issues and to seek guidance and / or corporate approvals.

The Surveillance Coordination Unit is responsible for supporting the functions of the Chief Health Surveillance Officer, including the design, development and delivery of PSSP2 implementation activities. This is accomplished through collaboration with internal and external surveillance partners such as the Surveillance Integration Team (SIT), the Surveillance Transformation Subject Matter Expert Reference Group (SME–RG), and the National Surveillance Infrastructure Task Group (NSI–TG) under the Public Health Infrastructure Steering Committee of the Federal/Provincial/Territorial Public Health Network.

The Data Coordination and Access Program is responsible for the development of a data management framework and roadmap as well as the assessment of the Agency's data architecture. The framework will address how the myriad of data issues fit together and identify and demonstrate relationships between Agency surveillance processes and data management. The roadmap will guide the implementation of the data management framework that will reflect organizational roles and responsibilities, decision making, planning and implementation for surveillance activities.

The Subject Matter Expert Reference Group is a 14 member working group, primarily at the Director level, supporting the Chief Health Surveillance Officer and the Surveillance Coordination Unit in the development and implementation of the PSSP2 on an 'as needed' basis. This group of key surveillance stakeholders in the Agency are consulted as needed on projects under development.

Surveillance Programs/System Managers are responsible for conducting surveillance activities. The Agency has 53 surveillance systems that are managed under the direction of two Assistant Deputy Ministers. These surveillance managers are responsible for the day–today surveillance such as: detection and notification of health events, collection and consolidation of pertinent data, investigation and confirmation of cases or outbreaks; routine analysis and reports and Government of Canada publications.

The 2013–2016 PHAC Surveillance Strategic Plan has a detailed implementation plan with clear roles and responsibilities for each of the four strategic objectives. The Plan details the activity, the leader, key partners, key milestones and timeline to complete. Currently, the planned activities that will influence the surveillance operations are to be led at the corporate level with the key partners being the surveillance managers. Unquestionably, successful implementation will only be able to be realized with a strong partnership to guide the technical work and leadership that has the authority to make recommendations for approval.

1.3 Performance measurement

Audit criterion: The Agency has identified surveillance outcomes that are reported and regularly monitored to support priority setting and other decision making.

Public health surveillance represents one of the Agency's core areas of focus, as such, it is important that there are appropriate performance measures linked to planned results and that this information is used to support management decision making.

To date, the Agency has developed a surveillance program logic model which summarizes the component activities, outputs, and expected outcomes related to Agency surveillance efforts.

Enlarge Figure 1

As well, a performance measurement framework has been developed, piloted, and subsequently approved by Executive Committee. The Surveillance Performance Measurement Framework (SPMF) is intended to provide a consistent process for surveillance performance measurement. It is also intended to identify areas for improvement in surveillance practice and management, and should contribute to the Agency's Performance Measurement Framework. Moreover, it is expected to bring a common view of surveillance system performance attributes (for example, resources, public value, outcomes) based on assessments performed by surveillance system managers. The SPMF is currently in its first phase of implementation, after which the assessment indicators will be revised as appropriate and reviewed by Executive Committee prior to the next implementation phase.

While the performance measurement framework is expected to support a common view of performance considerations across surveillance systems, it is unclear through the proposed "self–assessment" process as to how the Agency will be able to establish an objective view of planned versus actual surveillance system inputs (for example, financial and FTE resources), outputs (for example, timeliness of surveillance reports) and results (for example, performance against established objectives/criteria). As well, some of the questions in the self assessment were "client satisfaction" type questions – such as, "do you, as the surveillance manager, have sufficient Information Technology and Human Resources support?" While these are important questions, framed differently, they may offer a better indicator. It is expected these considerations will be reviewed through the first phase of implementation.

The Evaluation of the Surveillance Function at the Public Health Agency of Canada (tabled January 2013) recommended that the Agency–wide performance measurement framework for surveillance systems be finalized and implemented along with developing the necessary tools to monitor economy and efficiency. As well, the surveillance activity was included in the Agency's strategic operating review exercise which resulted in the Surveillance Transformation project. The transformation project is being tracked by the Business Transformation Office. Through these governing activities the performance measurement activity will be reviewed and adjusted as required. The Branch Head, Health Security and Infrastructure Branch is accountable to report on the Surveillance Transformation line, which is lead by the Chief Health Surveillance Officer. Project management tracking of Surveillance Transformation indicates the project is on schedule for all items as of October 2013.

2. Risk management

2.1 Risk management

Audit criterion: The Agency profiles the external and internal risks related to the surveillance activity.

The Agency's Policy on Integrated Risk Management states that risk management involves systematically considering the potential effects of risk on achieving objectives, and addressing key risks to objectives through appropriate decision making and risk treatment actions. The Policy also states that "risk management shall be a consideration in strategic and operational planning and reporting activities." In support of this policy, risk management processes should exist to support the identification, assessment, monitoring, mitigation, and reporting of strategic and operational risks related to surveillance activities.

Strategic Risk Management

Surveillance in and of itself is a risk mitigating activity. It serves as an early warning system and identifies public health emergencies. Within the Corporate Risk Profile (2012), various surveillance activities are prominently referenced both in terms of control mechanisms currently in place to manage identified risks and as risk treatment strategies. For example, surveillance is identified as a "risk driver" in four instances and as "risk treatment strategy" in thirteen instances. Surveillance is often seen as a means to mitigating risks related to infectious diseases. The risk treatment strategies to manage risks related to infectious diseases included: enhanced national surveillance; research to develop tools for surveillance; national guidelines for surveillance; and an options analysis for a comprehensive vaccine effectiveness surveillance system in Canada. Because these defined strategies are not actively tracked or monitored on an ongoing basis it is difficult to know if they occurred.

Surveillance Risks

1. Vision/priorities may not be well understood by partners.
2. Roles and responsibilities may not be well understood.
3. Senior management engagement may be insufficient.
4. Principles/priorities may be needed to guide investments.
5. Mechanisms are required to set priorities.
6. Surveillance management and practices may vary.
7. Limited surveillance system interoperability may exist.
8. Duplication of data / information requests to partners.
9. Multiple uncoordinated efforts to manage/ administer data.
10. Fragmented and outdated technical infrastructure.
11. Complex, one–off undertakings to share information.
12. Federal accountabilities for reporting on matters of public health.
13. Provinces/Territories not clear on surveillance strategic direction.
14. Dissemination rather than knowledge translation focus.
15. Timelines to release surveillance products.

Operational Risk Management

The 2013–2016 PHAC Surveillance Strategic Plan identifies 15 surveillance risks (see text box) with corresponding risk treatment strategies. More specifically the Agency recognizes that in the past the overall surveillance vision was not well understood and that surveillance priorities were not identified using a systematic analysis of risks to human health to justify the selection of diseases or public health matters to be monitored. The new plan states that there is an opportunity to have a systematic engagement to establish surveillance priorities, as part of a federated approach. Once completed, this will address a 2008 recommendation made by the Office of the Auditor General to use public health threat assessments to set objectives and priorities for its national surveillance activities.

The audit examined a sample of surveillance systems and requested specific risk surveillance information. For some surveillance systems analyzed, various tools (for example, activity work plans, surveillance system data quality assessments) were found to contain descriptions of risks and mitigation strategies although these risk management tools were found to vary significantly in their consistency and completeness across the various surveillance systems. The audit noted other surveillance system examples where operational risks had not been explicitly identified, nor had risk management strategies been documented. Discussions with surveillance system managers confirmed that risk management in these cases was conducted in an ad hoc manner.

The 2013–2016 PHAC Surveillance Strategic Plan captures both strategic and operational risks and notes that additional management focus is required to ensure surveillance systems continue to support the achievement of Agency objectives. As planned, it will be important for surveillance system managers to continue to partner and support the Chief Health Surveillance Officer in identifying and assessing the specific surveillance system risks in order to manage (and in certain instances mitigate) the corporate surveillance risks.

3. Internal controls

3.1 Planning

Audit criterion: The Branches prioritize surveillance activities in order to allocate financial and human resources to meet established objectives.

Setting out the strategic direction for surveillance should assist the Branches in reaching operational goals and strengthen inter–branch partnerships between those who conduct the surveillance and those who implement its findings. It was expected that at the Branch level there would be operational plans with appropriate allocation of funding and staff to conduct the necessary surveillance.

The audit noted that a variety of planning tools have been developed to support the management and delivery of branch programs, including surveillance activities. These tools include the Infectious Diseases Prevention and Control Branch draft Strategic Plan, Branch Operational Plans and the National Microbiology Laboratory Operational Business Plan which details the priorities, key initiatives and high level resources of the Branch.

In relation to surveillance systems and activities, the audit requested from surveillance managers planning documentation related to: surveillance rationale; objectives and expected results; planned activities, milestones and timing; planned resources (financial and human resources); planned outputs (and associated timing) of products. In relation to these expectations, the audit noted that the operational planning and monitoring information available to guide surveillance system delivery and support management decision making varies widely in its level of detail and consistency.

For example, surveillance conducted in the National Microbiology Laboratory is guided by a 2012–13 Operational Business Plan. This plan integrates the surveillance activity within each of the science programs. Science programs are further detailed by goals and activities/outputs. The plan provides information related to resources, however it is reported at the science program level not at the specific surveillance system level. At the other end of the spectrum was the Transfusion Transmitted Injuries Surveillance System which had less documented evidence of its purpose, objectives and expected results; planned resources and planned outputs.

Overall, the audit did not find any consistent and complete cost and full–time–equivalent information for surveillance systems sampled. Without complete information it is difficult to determine value–for–money. Since surveillance activities should be supporting the Agency's top management needs related to public health it would be important to establish surveillance priorities through a corporate decision making process. This will also position the Agency for enhanced strategic and operational planning related to surveillance activities.

Recommendation 1

It is recommended that the Assistant Deputy Minister, Infectious Disease Prevention and Control and the Assistant Deputy Minister, Health Promotion and Chronic Disease Prevention work with the Chief Health Surveillance Officer to ensure that a process for public health surveillance decision making is implemented.

Management response

3.2 Data Management

Audit criteria: The Branches have established data management capabilities to ensure surveillance data is received, analyzed and reported on in a timely, complete and accurate manner.

The surveillance activity is effective when it is collecting, analyzing and reporting timely and relevant information. It is a starting point that leads to the production of accurate and timely public health intelligence. The Surveillance Strategic Plan highlights challenges related to data management (that is, collection, analysis and reporting); data quality; and management of knowledge transfer. In response to these identified gaps, the Plan establishes priority actions to: (1) design, develop and implement a Data Management Roadmap and Framework; (2) develop and implement a Data Acquisition Strategy; and (3) develop recommendations for a corporate data architecture to support Agency surveillance.

The audit sampled 12 surveillance activities to determine if management controls were in place to ensure that surveillance systems are receiving, analyzing and reporting in a timely and effective manner. Of the 12 activities analyzed 5 surveillance activities had taken the first steps towards having a better understanding of their data management capabilities as the system managers have completed a comprehensive Data Quality Assessment Report.

Data Quality Assessments involve an assessment of the common data management practices employed at an individual surveillance system level. The criterion assesses data dimensions related to: accuracy; timeliness; serviceability; usability and relevance. Of the assessments that have been completed, the audit notes that surveillance system risks/issues/gaps have been identified as well as action plans to addressed identified deficiencies. However, it was noted that these action plans vary in their completeness and level of detail (for example, identification of specific actions, responsibilities and timeframes). Branches will benefit from following up on deficiencies to further improve the data management capabilities.

One of the strategic objectives to be addressed by the PSSP2 is the development of a corporate model for surveillance data management and technology. According to the PSSP2, the Agency will benefit from implementing a systematic approach to acquiring data and effectively sharing data within the Agency, other government departments, and with external partners. The Agency can also benefit from assuring that privacy obligations to data providers and Canadians are addressed and data holdings are managed more efficiently, effectively and economically.

The Agency is in the early stages of assessing its data management capabilities. Once all the data quality assessments are completed for each of the surveillance activities the Branches will be in a better position to streamline activities and to create a common surveillance platform.

Recommendation 2

It is recommended that the Assistant Deputy Minister, Infectious Diseases Prevention Control and the Assistant Deputy Minister, Health Promotion and Chronic Disease Prevention in conjunction with the Chief Health Surveillance Officer establish a schedule to complete the remaining data quality assessments.

Management response

3.3 Privacy Practices

Audit criterion: The Branches have processes and procedures to protect personal information through the proper collection, use and disclosure.

Anticipating and analyzing the privacy risks attached to public health surveillance is an important aspect in the design of any system. Privacy protection helps to ensure that personal health information of Canadians is protected in accordance with legislation. It would be expected that surveillance systems at the Agency would have been examined to identify any privacy requirements. If a privacy management strategy is required it would be expected that privacy impact assessment would have been completed to evaluate the effects of the data capture on an individual's privacy.

The Agency is now working within an approved enterprise Privacy Management Framework that includes tools and processes to assess the privacy risks associated with information sharing. This framework includes a Directive on the Collection, Use and Disclosure of Information Relating to Public Health. Under this new structure, the Privacy Management Framework as well as a Privacy Impact Assessment process are being defined for the Agency, but are still at the early stages of implementation.

An analysis of a sample of surveillance systems showed an inconsistency in the application of privacy guidelines. While the audit noted that some surveillance system owners have completed privacy impact assessments, interviews with them confirmed an inconsistent level of understanding of privacy impact assessment requirements or status of privacy impact assessment activities for the surveillance system. Recognizing these inconsistencies, a draft Privacy Impact Assessment Toolkit as well as a draft privacy impact assessment process has been created to support implementation of a consistent privacy impact assessment process. Both the toolkit and the assessment process are being reviewed by the Health Partnership Privacy Committee (a joint Public Health Agency of Canada/Health Canada advisory committee) and scheduled to be approved in the fall of 2013.

Recommendation 3

It is recommended that the Assistant Deputy Minister, Infectious Diseases Prevention Control and the Assistant Deputy Minister, Health Promotion and Chronic Disease Prevention work with the Chief Health Surveillance Officer to determine the privacy implications and requirements for each surveillance activity.

Management response

3.4 Performance Measurement

Audit criterion: The Branches have identified appropriate surveillance performance measures linked to planned results.

Based on the assertion that public health surveillance represents one of the Agency's core areas of focus, it is important that the Branches have identified appropriate performance measures linked to planned results and that this information is used to support management decision making.

At a broad level, the Branches have developed a logic model as part of their strategic plan which includes a summary view of the surveillance activity, outputs and outcomes.

At an operational level, it was noted through analysis of a sample of surveillance systems examples of performance measures being defined, however, there is general lack of completeness and consistency in the definition of surveillance system performance measures to enable robust and consistent articulation and monitoring of:

• Surveillance system inputs – for example, the resources required for surveillance system delivery
• Surveillance system outputs – for example, expected products (and associated timing) resulting from the surveillance system
• Surveillance system results – for example, definition of the expected value add of the surveillance system

In response to this gap (which extends beyond this Branch) is a surveillance performance measurement framework has been developed as part of the Surveillance Strategic Plan. This performance measurement framework, planned for early–2015 implementation, is intended to bring a common view of surveillance system performance attributes (for example, resources, public value, outcomes).

In addition, as referenced, the Surveillance Strategic Plan includes an initiative focused on development and implementation of a strategic surveillance decision making process. It is expected that this process and supporting decision making criteria will enable more consistent measurement of performance at an Agency, Branch and surveillance system level (see recommendation 1).

C – Conclusion

Surveillance is a core public health function for the Public Health Agency of Canada (Agency). Effective and timely surveillance is critical to the ability of the government and provinces/territories to accurately track, plan for, and respond to diseases. The Agency's surveillance and population health assessment initiatives enable many partners and stakeholders to better collect, analyze and interpret data; track and forecast public health events; and enhance support to health practitioners and decision makers.

In relation to Agency surveillance governance, significant progress has been made in establishing an "advisory structure" to inform Agency decision making and to support better communication of surveillance activities across the Agency. As recognized in the strategic plan, the Agency is still in the early stages of establishing a "decision–making" structure to which is intended to establish the prioritization and investment in new and existing surveillance activities.

From an internal control perspective there are a variety of management control practices employed in relation to surveillance system operational planning, data management, privacy management and performance measurement and these practices vary significantly in consistency and completeness across individual surveillance activities. These areas remain a focal point for various Surveillance Strategic Plan initiatives that are either at the early stages of implementation (for example, Data quality assessments, performance measurement framework) or planned for 2014–15 delivery (for example, corporate decision making structure).

In conclusion, the Agency has established elements of a management control framework for surveillance; however, focus on the following elements will serve to strengthen the surveillance activity:

• development of Branch surveillance priorities, planned activities and outputs, as well as planned financial and human resources as a part of the corporate decision–making model;
• progress against completing Data Quality Assessments; and
• review of privacy requirements for surveillance systems.

Appendix A – Lines of enquiry and criteria

Appendix B – Scorecard

Appendix C – Surveillance systems

Following is a summary of surveillance systems presented by Branch. A number of surveillance systems include both an epidemiology and laboratory component and therefore may appear multiple times in this summary. As referenced in the audit approach, those accompanied by an asterisk (*) were sampled for additional analysis in relation to the internal control audit criteria.

Infectious Disease Prevention and Control Branch – $36.2 M

• Centre for Immunization and Respiratory Infection – $3.8 M
  • 1. Canadian Adverse Events Following Immunization Surveillance System^Footnote*
  • 2. Canadian Measles and Rubella Surveillance System
  • 3. Enhanced Acute Flaccid Paralysis Surveillance
  • 4. Fluwatch
  • 5. Immunization coverage Surveillance
  • 6. Immunization Monitoring Program ACTive (IMPACT)^Footnote*
  • 7. Routine Surveillance of Vaccine Preventable Diseases
  • 8. Outbreak Summaries Surveillance (under development)
  • 9. National Enhanced IMD Surveillance System
  • 10. Invasive Group A Streptococcus Surveillance
  • 11. International Circumpolar Surveillance System

• Laboratory for Foodborne Zoonoses (LFZ) – $4.8M
  • 12. Canadian Integrated Program for Antimicrobial Resistance Surveillance^Footnote*
  • 13. National integrated Enteric Pathogen Surveillance Program

• Centre for Communicable Diseases and Infection Control (CCDIC ) – $14.4M
  • 14. Antimicrobial Resistant Neisseria Gonorrhoeae Surveillance System (under development)
  • 15. Canadian Nosocomial Infections Surveillance Program^Footnote*
  • 16. Canadian Notifiable Diseases Surveillance System^Footnote*
  • 17. Canadian Tuberculosis Laboratory Surveillance System
  • 18. Canadian Tuberculosis Reporting System
  • 19. Cells, Tissues and Organ Surveillance System
  • 20. Community Acquired MRSA Surveillance (under development)
  • 21. Enhanced Surveillance for Lymphogranuloma Venereum in Canada (under review)
  • 22. Transfusion Transmitted Injuries Surveillance System^Footnote*
  • 23. Transfusion Error Surveillance System
  • 24. Track Enhanced Surveillance Surveys
  • 25. Routine Surveillance of Reportable Sexually Transmitted Infections
  • 26. Routine Surveillance of Hepatitis B and C
  • 27. International Circumpolar Surveillance System^Footnote*
  • 28. HIV/AIDS Surveillance System
  • 29. HIV Strain and Drug Resistance Surveillance System
  • 30. Enhanced Surveillance of Canadian Street Youth

• Centre for Food–Borne, Environmental and Zoonotic Infectious Diseases – $4.5M
  • 31. Canadian Integrated Program for Antimicrobial Resistance Surveillance^{Footnote *}
  • 32. Lyme Disease Surveillance
  • 33. National Enteric Surveillance Program^{Footnote *}
  • 34. National Integrated Enteric Pathogen Surveillance Program^{Footnote *}
  • 35. Enhanced Listeriosis Surveillance
  • 36. Sentinel Surveillance for Avian Influenza and Other Wildlife Zoonoses
  • 37. Pharmacy Syndromic Surveillance System^{Footnote *}
  • 38. Outbreak Summaries Surveillance (under development)
  • 39. National Surveillance for West Nile Virus and other Mosquito–borne Diseases
  • 40. Canadian Creutzfeldt–Jakob Disease Surveillance System^{Footnote *}

• National Microbiology Laboratory (NML) – $8.6 M
  • 41. Canadian Measles and Rubella Surveillance System (Lab component)
  • 42. Canadian Nosocomial Infections Surveillance Program (lab Component)^{Footnote *}
  • 43. Canadian Tuberculosis Laboratory Surveillance System (lab component)
  • 44. Enhanced Invasive Pneumococcal Disease Surveillance (Lab Component) (Under Development)
  • 45. HIV Strain and Drug Resistance Surveillance System (Lab Component)
  • 46. National Integrated Enteric Pathogen Surveillance Program (Human Lab component)
  • 47. Streptococcal Disease Surveillance (Lab Components of iGAS, Invasive Streptococcus pneumonlae)
  • 48. PulseNet Canada^{Footnote *}
  • 49. National Enteric Surveillance Program (Lab Component)^{Footnote *}
  • 50. National Enhanced IMD Surveillance System (Lab Component)
  • 51. International Circumpolar Surveillance System (ICS) (Lab Components of TB, Invasive Meningococcal Disease, Haemophilus influenzae surveillance)^{Footnote *}
  • 52. Canadian Integrated Program for Antimicrobial Resistance Surveillance (CIPARS) (Human Lab Component)^{Footnote *}

Health Promotion and Chronic Disease Prevention Branch (HPCDP)

• Centre for Chronic Disease Prevention (CCDP) – $25.4 M
  • 53. Canadian Chronic Disease Surveillance System (CCDSS)
  • 54. Adult Cancer Surveillance Program
  • 55. Arthritis and Osteoporosis Surveillance Program
  • 56. Cardiovascular Disease Surveillance Program
  • 57. Chronic Respiratory Diseases Surveillance Program
  • 58. Diabetes Surveillance Program
  • 59. Mental Illness Surveillance Program
  • 60. Neurological Conditions Surveillance Program (under development)
  • 61. Childhood Obesity Surveillance Program (under development)
  • 62. Cancer in Young People in Canada Surveillance Program (CYP–C)
  • 63. Canadian Perinatal Surveillance Program (CPSS)
  • 64. Canadian Paediatric Surveillance Program (CPSP)
  • 65. Child Maltreatment Surveillance
  • 66. Developmental Disorders Surveillance Program (under development)
  • 67. Injury Surveillance