Audit of the Risk Management Framework

December 2014

For readers interested in the PDF version, the document is available for downloading or viewing:

Audit of the Risk Management Framework (PDF document - 826 KB- 23 pages)

Table of Contents

Executive summary

The focus of the audit is the risk management framework at the Public Health Agency of Canada (the Agency). Effective risk management equips organizations so that they can to respond actively to change and uncertainty by using risk-based information to enable more effective decision-making.

The objective of the audit was to assess the effectiveness of the Agency's risk management framework to support the delivery of its mandate. The audit examined governance, risk management and internal control considerations related to the risk management framework. The audit was conducted in accordance with the Treasury Board Policy on Internal Audit and the International Standards for the Professional Practices of Internal Auditing. Sufficient and appropriate procedures were performed and evidence gathered to support the audit conclusion.

The Agency has established a continuous, systematic process for profiling its risks on an Agency-wide basis. The Agency's risk management framework reaffirms the core principles and approaches to risk management that have been in place in the Government of Canada since 2001 and reflect, where appropriate, international and national standards related to risk management. In 2013-14, the Treasury Board of Canada Secretariat encouraged the inclusion of mandate-specific risks in the Corporate Risk Profile. Currently, the Agency's profile of risks focuses on five mandate-specific public health risks.

The process used to develop the corporate risk profile was found to be effective in identifying and assessing risks, developing mitigating strategies and coordinating risk management responses and activities. The process is supported by the Integrated Risk Management Policy, the Risk Management Standard and a best practice guide. The policy has recently been updated to reflect current roles and responsibilities.

To support integrated risk management, an executive-level Risk Management Oversight Committee has been established to provide additional horizontal oversight and leadership. The committee meets regularly and has been effective at raising the awareness of Agency-wide risk management expectations and driving more integration. The audit found some evidence of integrated risk management practices and notes that where branches and functions have taken the lead to allocate dedicated resources towards risk management, they have been able to take a risk-informed approach.

The audit found that the Agency could benefit from clearly measureable mitigation strategies that include milestones, deliverable dates and resource requirements. The audit also recommends that the Agency continue with its intention to develop performance measures for each of the mitigating treatments, as well as develop a process to monitor the risks

Management agrees with the two recommendations made in the audit and has prepared a management action plan that will serve to further strengthen the risk management process.

A - Introduction

1. Background

Risk management is defined by the Government of Canada as a systematic approach to setting the best course of action under uncertainty by identifying, assessing, understanding, making decisions on and communicating risk issues. Effective risk management equips federal government organizations to respond actively to change and uncertainty by using risk-based information to enable more effective decision-making. In turn, increased capacity and demonstrated ability to assess, communicate and manage risk builds trust and confidence, both within the government and with the public.

Embedding risk management into an organization's structures and programs using a consistent risk management process creates a cohesive integrated risk management environment. Integrated risk management supports planning, priority setting, program management, financial reporting, corporate business planning, business continuity, operations and performance assessment, and other key functions throughout an organization at the departmental, branch and program levels.

The Treasury Board of Canada Secretariat's (TBS) 2010 Framework for the Management of Risk provides broad risk management principles and clarifies the roles and responsibilities. The framework sets out expectations for deputy heads and their departments and agencies in leading the implementation of effective risk management practices at all levels of the organization. It supports strategic priority setting and resource allocation, informed decisions with respect to risk tolerance and improved results. New in 2013-14, TBS encouraged the practice of including mandate-specific risks in the corporate risk profiles, rather than focusing solely on internal services and management risks.

The framework reaffirms the core principles and approaches to risk management that have been in place in the Government of Canada since 2001 and reflects, where appropriate, international and national standards related to risk management. Using a risk management framework helps identify and classify risks, formulate an integrated picture of risks and

Risk Management Process
  1. Risk identification
  2. Risk assessment
  3. Risk response
  4. Risk communication
  5. Risk monitoring

coordinate risk management functions and activities. The Management Accountability Framework (MAF), which is a key performance management tool for the federal government, also expects departments to incorporate risk management practices and principles into the organization's strategic, operational and functional activities.

Public Health Agency of Canada

Through the development of a Corporate Risk Profile (CRP) and its annual update, the Public Health Agency of Canada (the Agency) notes that it is systematically identifying, understanding and seeking to address key public health and organizational risks, which drives its priority setting, planning and programming. The CRP is intended to be an important management tool, promoting and informing balanced decision-making and mitigating key risks that may impact the achievement of Agency objectives. More specifically, the profile serves to identify the Agency's most important corporate risks, risk treatments and related performance indicators, and should be integrated and operationalized through the Agency's operational planning and performance management processes.

The Strategic Policy, Planning and International Affairs Branch (SPPIAB) is responsible for the risk management function. Within that branch, the Operational Planning, Reporting and Risk Directorate is responsible for the CRP, the Integrated Risk Management Policy, standards and other tools and guidance to support the Agency's risk management activities. As well, governance committees are in place to exercise the collective accountability in relation to the management of the Agency's risks. In particular, the Risk Management Oversight Committee (RMOC) monitors the CRP and provides direction on risk management as a part of corporate governance and advice to the Executive Committee (EC), which approves all Agency matters related to risk management.

Text equivalent for Agency's project management lifecycle

The Operational Planning, Reporting and Risk Directorate reports to Governance Planning and Reporting. It has a total budget of 1.5 million dollars and 11 FTEs, of which 1.2 FTEs are devoted to risk management.

The Office of International Affairs for the Health Portfolio, Governance Planning and Reporting Directorate and Strategic Policy and Ministerial Services report to the Strategic Policy, Planning and International Affairs Branch.

The Strategic Policy, Planning and International Affairs Branch reports to the President and to the Chief Public Health Officer.

Enlarge Figure

2. Audit objective

The objective of the audit was to assess the effectiveness of the Agency's risk management framework to support the delivery of its mandate.

3. Audit scope

The audit examined the Agency's fundamental controls for risk management, and the scope included the shared responsibilities of both corporate and branch activities, the CRP and risk integration and treatment. Specifically, the audit examined internal controls related to governance, policy, procedures and practices, continuous monitoring and learning related to risk management during the 2013-15 fiscal years. The scope of the audit did not include processes to manage events or emergencies or the activities of the Health Portfolio Operations Centre (HPOC), which are not part of the risk management framework and were the subject of two separate Health Canada audits in 2010.

4. Audit approach

The audit team used methodologies, including but not limited to a literature review of risk management practices, interviews with corporate and branch leads and a review of the Agency's planning and reporting documents (for example, strategic plan, operational plans and corporate reports). A review of previous audit reports with recommendations made or outstanding during the period under review were examined, as well as MAF recommendations related to the risk management area of management.

The audit criteria were derived from the Office of the Comptroller General's Audit Criteria related to the Management Accountability Framework: A Tool for Internal Auditors (TBS, March 2011) and criteria approved by management.

5. Statement of conformance

In the professional judgment of the Chief Audit Executive, sufficient and appropriate procedures were performed and evidence gathered to support the accuracy of the audit conclusion. The audit findings and conclusion are based on a comparison of the conditions that existed as of the date of the audit, against established criteria that were agreed upon with management. Further, the evidence was gathered in accordance with the Internal Auditing Standards for the Government of Canada and the International Standards for the Professional Practice of Internal Auditing. The audit conforms to the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program.

B - Findings, recommendations and management responses

1. Governance

1.1 Governance

Audit criterion: Corporate-level senior organizational committees provide leadership and active support on risk management.

Effective risk management practices equip federal government organizations to respond proactively to change and uncertainty by using risk-based approaches and information that enable more effective decision-making throughout an organization.

The Public Health Agency of Canada (the Agency) has established a governance structure to oversee risk management within the organization that includes three committees: the Executive Committee, the Risk Management Oversight Committee and the Departmental Audit Committee.

The Executive Committee (EC) is responsible for risk management. It approves the Corporate Risk Profile (CRP) and risk management guidance and tools, and promotes a culture of integrated risk management as an inherent component to good decision-making throughout the Agency.

The Risk Management Oversight Committee (RMOC), created in 2012, provides horizontal oversight and leadership on risk issues across the Agency. The RMOC is a Tier II committee that reports directly to the Agency's EC. Membership in the committee is determined by the Chief Public Health Officer (CPHO) and the Associate Deputy Minister. Members are chosen based on their unique skills and contributions, rather than by position within the Agency. The RMOC meets monthly (more frequently as required) and currently consists of 17 members, including the Chair.

As mandated by the Policy on Internal Audit, the Departmental Audit Committee (DAC) reviews risk management documents (that is, the CRP) and processes and provides objective advice and recommendations to the CPHO and the Associate Deputy Minister regarding the sufficiency, quality and results of assurance on the adequacy and function of the Agency's risk management framework. In 2013-14, DAC reviewed and advised on the risk management arrangements established by the Agency, including its review and recommendations for modifications to the draft and final CRP for 2013-15.

In 2012, the Agency developed a draft Concept of Operations for integrated risk management (see Appendix D), which outlines the relationship between the three committees.

The auditors collected and analyzed the various committees' documentation such as the terms of reference, membership, agendas, records of decisions, attendance at RMOC and decisions and recommendations. In conclusion, corporate-level senior committees provide leadership and active support on risk management.

1.2 Risk policy

Audit criterion: The Agency has a risk management policy and standard that are approved and maintained to support the risk management process.

The Agency's Integrated Risk Management Policy was developed in 2010 and updated in 2013. The objective and content of the Agency's Integrated Risk Management Policy is aligned with the Treasury Board of Canada Secretariat's (TBS) Framework for the Management of Risk (2010) and the associated guidance.

In 2014, the Strategic Policy, Planning and International Affairs Branch (SPPIAB) modified the policy to expand the roles and responsibilities and to reflect enhancements to the risk management framework. The revised policy better aligns with the 2013-15 CRP. It was presented to RMOC and is to be tabled at EC for approval.

Supporting the policy is the Integrated Risk Management Standard (2009). It expands on the three-page policy and details the risk management process. In particular, it details the process related to risk communication, identification of stakeholders, establishment of context (risk tolerance), risk identification, risk analysis, risk evaluation, monitoring and review and risk profiling.

The Agency has a current Integrated Risk Management Policy and the Standard that support risk management processes.

2. Risk management

2.1 Risk-informed approach

Audit criterion: The Agency takes a risk-informed approach for the management of its operations.

Risk management at the Agency should include the activities around the implementation and management of the CRP and should demonstrate a risk-informed approach to managing ongoing activities.

To assess the risk-informed approach, the audit examined committee operations, branch operational plans and past risk-related audit findings. The results of this analysis noted some inconsistency in branch reporting on risk management, despite the revised branch operational planning templates that expect program and functional risks to be documented, with mitigating strategies, performance indicators and mid-year monitoring. However, it was found that if the program or function was not included in the CRP, specific branch or functional risks were not always documented and monitored. Nevertheless, evidence shows that where branches and functions have taken the lead to allocate dedicated resources towards risk management, they have been able to take a risk-informed approach. The following three examples illustrate some of the Agency's risk-based approaches.

Daily intelligence and risk assessment meetings

Every weekday morning at 8:30 a.m., the Agency reviews and assesses events that might have a public health impact. This meeting provides an opportunity for senior managers to review key threats (events/issues/risks), prioritize appropriate risk management options and approaches and initiate briefings. The meeting is co-chaired by the Assistant Deputy Minister, Infectious Disease Prevention and Control, and the Branch Head of the Health Security and Infrastructure Branch. Members are accountable for performing reliable and valid risk assessments for each potential and/or actual threat and opportunity identified. The co-chairs brief executives at 9:00 a.m. through the daily update meeting and assess the merits of convening the Executive Planning Group. The daily update membership includes the Chief Public Health Officer, the Associate Deputy Minister, branch heads and additional senior management and staff, as appropriate. Risks requiring Agency mitigation and escalation are identified in the daily intelligence report, which includes a risk tolerance chart corresponding to the risk decision chart used by the World Health Organization and International Health Regulations.

Grants and contributions

A structured risk management approach for the Agency's Grants and Contribution programs has been transformed into a comprehensive Risk-based Monitoring Framework that introduces strengthened risk-based monitoring strategies. These new strategies include project and financial monitoring by programs, in consultation with the recipients' self-assessment questionnaires, a new IT system to manage the process (GCIMS – Grants and Contributions Information Management System) and structured mitigation measures to address areas of non-compliance. The risk mitigation measures outline the degree of monitoring that will be taken based on the results of the risk assessment of the contribution agreement (that is, low, medium and high). The risk assessment results trigger the mitigation measures that will be followed and include enhanced financial and work plan monitoring requirements, as well as the guidelines around what percentage of recipients within a risk assessment range will be subject to an annual site visit.

Surveillance activities

The PHAC Surveillance Strategic Plan 2013-2016 identifies 15 surveillance risks, with corresponding risk treatment strategies. The Agency has defined its overall surveillance vision. Surveillance priorities are identified using various corporate assessment processes that include the systematic analysis of risks to human health, to justify the selection of diseases or public health matters to be monitored. A new decision-making framework is being implemented that will facilitate strategic alignment of surveillance activities with the Agency, the Government of Canada, emerging public health issues and collaborative priorities. This process will include the assessment of new and ongoing surveillance and surveillance-related activities against a specific set of corporate, strategic and public health criteria.

It is anticipated that the Agency will continue to mature in its risk management practices, given the current senior management attention and a comprehensive risk management framework. In addition, the analysis of past audit recommendations related to risk noted that all recommendations have been actioned by management in a timely manner.

In conclusion, the various approaches to managing risk outlined above, combined with the ongoing efforts around the CRP, support the risk-informed approach that the Agency has adopted in the management of its operations.

3. Internal controls

3.1 Guidance and tools

Audit criterion: Guidance and supporting tools are developed and up-to-date to support the risk management process.

Guidance documents

The Agency has developed a best practice guideline entitled Integrating Risk Management at the Agency (2014), which is based primarily on practices identified in TBS's Framework for the Management of Risk and the Guide to Integrated Risk Management.

In addition, the Agency has recently produced a draft companion document to the Agency's 2013-15 CRP. It was presented to RMOC for endorsement prior to seeking final approval at EC. The companion document was developed in response to an EC request to provide Agency employees with a concise overview of the CRP. The three-page document provides a quick reference on the risk profile, outlining specifically why it is an important risk management tool, the intended purpose of the CRP and the anticipated use of the document. This is the first instance in which such a document has been developed. The document was presented to RMOC for endorsement prior to seeking final approval at EC.

On the Agency's intranet site, there is an easily accessible Risk Management section under the "Strategic Focus" tab, which provides risk management information for employees, including the Treasury Board's and Agency's Integrated Risk Management Frameworks, Frequently Asked Questions and Key Concepts. The content of the site would benefit from an update, to reflect current operational direction, as well as more recent contact information (it currently cites the support for risk management as residing in Emergency Management and Corporate Affairs Branch), thus better assisting employees in fulfilling their risk management responsibilities, in accordance with the current Agency direction.

Tools

Risk registers, also known as risk logs, have the ability to improve the management of identified risks. The risk register serves as a central repository for the organization's risk information and allows for the results from the risk management process to be suitably sorted, standardized and merged for relevance to the appropriate level of management. Its key function is to provide significant information on the risks faced by the organization. The risk register also gives the organization's risk management stakeholders a clear view of the current status of each risk.

The Agency has researched global and Government of Canada usage, opportunities and risks, the benefits and the applicability of utilizing risk registers. This effort included an RMOC presentation (May 2013) on Explaining a PHAC Risk Registry Database, as well as the development of a Risk Registry Study Report (May 2014), which included a comprehensive description and analysis and five recommendations in favour of the adoption of risk registers. These recommendations included a consistent approach to managing risks across the Agency and a live, real-time register of all risks and process improvements to be recognized from the use of the register, which include improvements to risk identification, treatment, monitoring and review.

The Agency's Concept of Operations model illustrates how risk registers are intended to make the connection between the Agency's program/branch risk management layer and the Agency's enterprise risk management. In that regard, SPPIAB (through the operational planning exercise and risk management exercise) has developed templates that contain a sample of the requirements for a risk register. At the time of writing this report, one directorate, the Centre for Biosecurity within the Health Security Infrastructure Branch, has liaised and used the templates to develop a risk register.

3.2 Risk identification

Audit criterion: There is an established, continuous, systematic process for identifying risk on an Agency-wide basis.

The Agency's process offers a systematic way to structure the identification, assessment, response, communication and monitoring of significant risks through an established governance structure. Three workshops were used to identify the corporate risks. They were attended by RMOC members, senior executives, regional representatives, evaluation, finance and others such as planning and policy representatives.

The first workshop reviewed the former risks, identified new risk areas and addressed other risks that had been tracked throughout the previous year in a change registry. One of the purposes of the first workshop was to distinguish actual risks from other events that the Agency considered to be issues, preoccupations or conditions. In order to make this distinction, the Agency used a decision-making model, as outlined in Table 1.

Table 1: Decision-making model for the 2013-15 Corporate Risk Profile

Degree of influence for Agency role in managing the risk (based on existing controls/proposed treatment strategies)

Agency role and authority to manage risk:

  1. Contribute to the prevention and control of disease and injury and to the promotion of health;
  2. Enhance quality of surveillance data and expand knowledge of disease;
  3. Provide federal leadership in managing public health emergencies;
  4. Serve as a focal point for sharing Canada's expertise with the rest of the world, applying international research and development to Canada's public health programs;
  5. Strengthen partnerships and intergovernmental collaboration and facilitate national approaches to public health policy, planning and governance.

Federal role – Cross-jurisdictional responsibility with all levels of government, other public health stakeholders, etc.

Disposition (i.e., venues where risks are addressed)

Voting criteria:

  1. Risk - CRP
  2. Risk - AOP
  3. Risk - Branch Risk Registry (where applicable)
  4. Issue/ preoccupation - AOP
  5. Issue/ preoccupation - Issue management
  6. Issue/ Preoccupation - Branch plans
  7. Other

The most significant risks identified during the first workshop were those that involved the Agency's role in the management of public health risks that may have adverse effects on the Agency's objectives, as opposed to public health risks to Canadians. The Agency's senior management also identified and discussed a range of existing risks and proposed some new risks, both internal and external, that were vetted against the decision-making model. The result of this exercise was a short list of risks identified by risk area.

The next step in the process was for the risk leads to develop risk statements. The Agency utilized a risk summary sheet based on the decision-making model and a review of best practices. Risk leads completed the risk summary sheets by providing (1) the Risk Statement from the 2012-13 CRP, if applicable; (2) the Risk Statement for the Risk to Canadians; and (3) a Risk Statement indicating the residual risk to the Agency.

The second workshop focused on a review of the risk statements and the selection of the risk statement options that best reflected the key corporate risks to the Agency. At this point in the process, the Agency put its efforts into ensuring that the "right" risks had been captured. With the risks identified at this point, further information was added to the risk summary sheets that could inform decision-making and that included risk drivers, risk controls and risk treatments.

At the final workshop, the risk leads were asked to provide an overview of each risk. The workshop participants then assessed each risk by using a set of challenge questions (see text box) to ensure that the proposed risks met the requirements of a risk that could be considered for inclusion in the Agency's CRP.

Table 2: Workshop challenge questions

CHALLENGE QUESTIONS

  • Is the proposed "risk" a risk? (Does it involve uncertainty of occurrence; does it have the potential to affect the fulfillment of the Agency's mandate; does the Agency have significant responsibilities for the risk/significant ability to effect outcomes, etc.?)
  • Is the Risk Statement clear and concise? Does the Risk Statement include a clear identification of the effect on outcomes? [that is, there is a risk that x (event) will create/increase y (incident/occurrence), resulting in z (outcome/impact)].
  • Does the residual risk to the Agency flow from the risk to Canadians, taking into consideration the roles and responsibilities of the Agency?
  • Have the mitigation activities been considered in developing the Residual Risks?
  • Are the identification of drivers, controls and treatments sufficiently complete to provide a sound basis to enable the development of performance indicators?
  • Does the Risk Summary Sheet provide a sound basis to enable risk ranking?
  • Do we have a complete list of all the risks that are appropriate for inclusion in the 2013-15 Corporate Risk Profile?

Participants also discussed other risks, such as surveillance, human resources and information technology, and identified these as horizontal enablers that are critical to the Agency's mandate. These risks were not considered to be corporate risks because they did not meet one or more of the threshold requirements identified through the challenge questions. The comprehensive process followed to identify the corporate risks conforms to the expectations of both the Agency and the Treasury Board of Canada Secretariat.

3.3 Risk assessment

Audit criterion: Risks are analyzed and prioritized.

The next step in the Agency's risk management process was an assessment of the risks that have been identified. During the assessment process, risks were analyzed and prioritized.

Branch heads and senior executives participated in a workshop where they assessed the five corporate risks identified for inclusion in the Agency's risk profile by voting on criteria related to the impact and probability that the risk would have on the organization reaching its objectives. The voting took place using a computer application designed to enable the anonymous assessment of risks in a workshop environment. The information collected was used to generate the Agency's heat map, which shows the ranking of each risk in terms of probability and impact.

The workshop participants initially voted on the inherent (pre-mitigation) risk to the Agency and then residual risk (after consideration of all mitigation/treatment activities such as risk controls and risk treatments).

Figure 2: The Agency’s heat map

Figure 2: The Agency’s heat map
Legend

1A.  Emerging and Re-Emerging Infectious Respiratory Diseases- Pandemic (including, but not limited to influenza)
1B.  Infectious Disease - Antimicrobial Resistance
1C. Food Safety - Emerging and Re-
Emerging Food-Borne Diseases 1D. Infectious Disease - Emerging and Re- Emerging Vector-Borne Zoonotic Infectious Diseases
2. Chronic Disease - Risk factors and conditions for Chronic Disease, Mental Illness and Injury

Text equivalent for The Agency's Heat Map

The heat map indicates the level of the Agency's risk tolerance for inherent risks and residual risks.

Inherent risks:

1A – Emerging and re-emerging infectious respiratory diseases – pandemic (including but not limited to influenza) is placed as a high risk impact and a probable risk probability.

1B – Infectious disease – antimicrobial resistance is placed as a high risk probability and a very probable risk probability.

1C – Food safety – Emerging and re-emerging food-borne diseases is placed as a medium-high risk impact and a probable-very probable risk probability.

1D – Infectious Disease – Emerging and re-emerging vector-borne zoonotic infectious diseases is placed as a high risk impact and probable-very probable risk probability.

2 – Chronic disease – risk factors and conditions for chronic disease, mental illness and injury is placed as a high-very high risk impact and a very probable risk probability.

Residual risks:

1A – Emerging and re-emerging infectious respiratory diseases – pandemic (including but not limited to influenza) is placed as a medium risk impact and as a probable risk probability.

1B – Infectious disease – antimicrobial resistance is placed as a high risk impact and as a probable risk probability.

1C – Food safety – Emerging and re-emerging food-borne diseases is placed as a medium risk impact and as a probable risk probability.

1D – Infectious Disease – Emerging and re-emerging vector-borne zoonotic infectious diseases is placed as a high risk impact and as a probable risk probability.

2 – Chronic disease – risk factors and conditions for chronic disease, mental illness and injury is placed as a high risk impact and as a probable risk probability.

Enlarge Figure

The placement of each residual risk on the heat map is an indication of the Agency's risk tolerance. Risk tolerance is the willingness of an organization to accept or reject a given level of residual risk, which may differ across the organization but must be clearly understood by the individuals making risk-related decisions on a given issue. Clarity on risk tolerance at all levels of the organization is necessary to support risk-informed decision-making and foster risk-informed approaches. Given the significance of these decisions, it is important for the Agency to develop performance measures and implement a systematic monitoring system to assess the effectiveness of the risk treatments and subsequent movement of the residual risks on the heat map (see Recommendation 2).

The comprehensive process followed to assess the corporate risks conforms to the expectations of the Agency and the Treasury Board of Canada Secretariat.

3.4 Risk response

Audit criterion: A continuum of measures of risk mitigation are developed and implemented to address an identified risk.

Mitigation strategies for the five corporate risks identified in the 2013-15 CRP have been developed. Branch risks have also been developed and form part of the branch operational plans. The Agency has progressively included risk information in the branch and Agency operational plans. Risk reporting in these plans has been enhanced with the introduction of risk mitigation strategies, branch linkages to corporate risks identified in the CRP, branch risks, risk controls and risk treatments.

Although risk treatment strategies, actions and targeted completion dates are reported in both documents, they are vaguely worded and lack milestones and concise deliverable dates. For example, the risk response is often to "enhance" an activity, yet enhancement is not defined or measured. As well, resource requirements are summarized by branch priority, which includes both operational activities and risk treatments, making it difficult to discern which resources are attributable to ongoing priorities as opposed to risk mitigating activities. Assessing performance of mitigation strategies and decisions to increase or decrease resources allocated to risk treatments is difficult without more precise information.

Risk mitigation strategies for both the CRP and branch risks could be strengthened with the increased precision of timelines, delivery milestones, actions and linkages to resource requirements. The Agency notes in its 2014-15 Report on Plans and Priorities that specific indicators are under development for each of the risk treatments.

In conclusion, the Agency has a continuum of risk mitigation responses but does not have milestones to track implementation or performance indicators to track performance.

Recommendation 1

It is recommended that the Assistant Deputy Minister, Strategic Policy, Planning and International Affairs Branch, in consultation with branch assistant deputy ministers/heads, develop performance indicators to track the progress of risk mitigation strategies.

Management response

Management agrees with the recommendation

In its 2014-15 Report on Plans and Priorities, it was noted that specific performance indicators in the Risk Analysis section are under development for the risk treatments.

The Strategic Policy, Planning and International Affairs Branch (SPPIAB) will work with branches to develop performance indicators for risk mitigation strategies for the Corporate Risk Profile and branch plans (that is, risk treatment and risk controls) to track and monitor progress towards risk mitigation. This work will be aligned with the Agency Operational Plan.

3.5 Risk reporting

Audit criterion: Risks are consistently reported in corporate documents.

Key corporate reports such as the Department's Report on Plans and Priorities (RPP) and Departmental Performance Report (DPR, should discuss the key strategic risks identified in the CRP that could prevent the branches–and therefore the Agency as a whole–from achieving their objectives.

The audit examined the four separate risk reporting mechanisms that are producing risk results (the CRP, the RPP, the DPR and branch operational plans). The audit found that the risks reported in the CRP, align with the summary descriptions in the DPR and the RPP. As well, the audit found that corporate risks detailed in the CRP generally align with the branch operational plans (see Section 3.6).

3.6 Risk monitoring

Audit criterion: Risks are re-assessed, monitored and reported.

A process of regular review ensures that new risks are identified and considered as they arise; that existing risks are monitored to identify any changes that may impact the Agency; that risk treatments are being implemented according to the planned schedule; that risk controls are still in place and working effectively; and that information on risk is reported to the RMOC regularly and to EC as required.

In May 2013, risk leads for the 2012-13 CRP presented an update to RMOC on the status of their assigned risk. A review of the 2013-15 CRP has not yet taken place but is planned to occur prior to the end of the calendar year.

Both the CRP and the branch operational plans have risk mitigation implementation strategies and fiscal year deliverable timeframes. However in some cases, deliverable dates span extended periods yet no milestones are indicated or resource requirements identified. Without milestones and resource requirements, it will be difficult to monitor whether treatment strategies in both the CRP and branch operational plans are being implemented according to the planned schedule and whether existing controls are effective. The current state of the mitigation strategies will not support the definition of concise indicators for effective performance measurement.

As well, performance indicators are required for monitoring purposes and will be required to support the performance measurement process. The 2013-15 CRP notes that performance measures are "under development".

While the majority of branches have identified branch risks outside of the five corporate risks detailed in the CRP, not all branches have included branch-specific risks. Risks identified in the branch operational plans are also not currently monitored at an Agency-wide level.

The audit expected to find a greater level of detail in the branch operational plans with respect to mitigation strategies; however, most either replicated or had less information (lack of dates) than the CRP. Additionally, one branch did not include the CRP mitigation strategies that they have a responsibility to implement, as per their branch operational plan.

Without performance review and oversight, the Agency may be less aware of the progress (or lack thereof) on risk mitigation strategies, of whether resources are well-utilized for treatments, and of whether the impact or likelihood of a risk occurring is trending in a positive direction.

Recommendation 2

It is recommended that the Assistant Deputy Minister, Strategic Policy, Planning and International Affairs Branch, establish and implement a monitoring process of branch and corporate risks.

Management response

Management agrees with the recommendation

The Strategic Policy, Planning and International Affairs Branch (SPPIAB) will develop and implement a monitoring process for branch and corporate risks that is integrated with the operational planning and reporting processes.

C - Conclusion

The Public Health Agency of Canada (the Agency) has implemented an effective risk management framework to support the delivery of its mandate. The Agency has established a continuous, systematic process for profiling its risks on an Agency-wide basis. The Agency's risk management framework reaffirms the core principles and approaches for risk management that have been in place in the Government of Canada since 2001 and reflect, where appropriate, international and national standards related to risk management.

The Agency has a number of good practices, exhibiting risk integration in branch operations, including activities in Grants and Contributions, Surveillance, the Daily Intelligence and Risk Assessment Meetings (DIRAM) of senior staff members and the Daily Update meetings that include the Chief Public Health Officer, the Associate Deputy Head, branch heads and senior staff.

Moderate enhancements can be made to the risk management framework to support the Agency in its ongoing commitment to manage the identified risks. The Agency could further strengthen the risk management function by developing performance indicators for each of the risk mitigation strategies and by developing a process to monitor the risks.

The audit report makes two recommendations to address these findings and the Agency has developed a suitable management action plan that will serve to further strengthen the existing good practices.

Appendix A - Lines of enquiry and criteria

Audit of Project Management Framework
Criteria Title Audit Criteria
Line of Enquiry 1: Governance
1.1 GovernanceFootnote 1 Corporate-level senior organizational committees provide leadership and active support on risk management.
1.2 Risk policyFootnote 1 The Agency has a risk management policy and standard that are approved and maintained to support the risk management process.
Line of Enquiry 2: Risk Management
2.1 Risk-informed approachFootnote 2 The Agency takes a risk-informed approach for the management of its operations.
Line of Enquiry 3: Internal Controls
3.1 Guidance and toolsFootnote 1 Guidance and supporting tools are developed and up-to-date to support the risk management process.
3.2 Risk identificationFootnote 2 There is an established, continuous, systematic process for identifying risk on an Agency-wide basis.
3.3 Risk assessmentFootnote 2 Risks are analyzed and prioritized.
3.4 Risk responseFootnote 2 A continuum of measures of risk mitigation are developed and implemented to address an identified risk.
3.5 Risk reportingFootnote 2 Risks are consistently reported in corporate documents.
3.6 Risk monitoringFootnote 2 Risks are re-assessed, monitored and reported.

Appendix B - Scorecard

Scorecard – Audit of the Risk Management Framework
Criterion Rating Conclusion Rec #
Governance
1.1 Governance Satisfactory  A corporate-level senior committee (Risk Management Oversight Committee) provides leadership and active support on risk management.  
1.2 Risk policy Satisfactory The Agency's Integrated Risk Management Policy was updated in May 2014 and needs to be approved.  
Risk Management
2.1 Risk-informed approach Satisfactory  The Agency not only relies on the Corporate Risk Profile to manage risk, but has also adopted a risk-based approach to a number of its areas of operations.  
Internal Controls
3.1 Guidance and tools Needs minor improvement  The Agency should develop a formal risk register in order to better track Agency risks.

As well, the website should be updated.

 
3.2 Risk identification Satisfactory The Agency has established a risk identification process that includes consultative workshops attended by the Agency's senior management team.  
3.3 Risk assessment Satisfactory The Agency has an established risk assessment process.  
3.4 Risk response Needs minor improvement  Mitigation strategies have been developed for all branch and corporate risks, but the Agency needs to develop performance indicators to track progress. 1
3.5 Risk reporting Satisfactory Risks are reported in corporate documents.  
3.6 Risk monitoring Needs moderate improvement The Agency should develop a process to monitor and report on the implementation of planned mitigation treatments and performance measures. 2

Appendix C - Corporate risks, 2013-15

Corporate Risk Risk Risk Statement: Residual Risk to Agency
1A-Infectious Disease – Pandemic Pandemic (including but not limited to influenza) There is a risk that the Agency will not be able to effectively monitor, detect and coordinate a response to infectious respiratory disease outbreaks, and effective medical countermeasures will not be available.
1B-Infectious Disease – Antimicrobial Resistance Antimicrobial Resistance There is a risk that the absence of a comprehensive national action plan may exacerbate the growing impact of antimicrobial resistance on the health and well-being of Canadians.
1C–Infectious Disease – Food Safety Emerging and Re-Emerging Food-Borne Diseases There is a risk that the Agency will not receive all relevant, integrated information to inform early interventions and that partners and stakeholders will not be aware of the information generated by the Agency in a timely manner required to prevent illness.
1D–Infectious Disease – Lyme Emerging and Re-Emerging Vector-Borne Zoonotic Infectious Diseases There is a risk that the total burden of vector-borne disease will increase without a national approach to monitor and assess these diseases and to enable the implementation of prevention and control measures.
2- Chronic Disease – Risk Factors and Conditions for Chronic Disease, Mental Illness and Injury Effective upstream interventions (to address risk factors and conditions and protective factors). There is a risk that the Agency's leadership in health promotion and disease prevention could be impacted without further refocusing the Agency's activities in science/research, surveillance, policies/programs and partnerships towards the upstream–social determinants, protective factors and risk factors.

Source: Public Health Agency of Canada's Corporate Risk Profile 2013-15. Approved March 2014

Appendix D - Concept of operations and risk information

Concept of operations and risk information
Text equivalent for Concept of operations and risk information

 

Text Equivalent - Figure 3 – Concept of operations and risk information

Appendix D is an illustration of the Public Health Agency of Canada's (the Agency) "Concept of Operations" and the integration of risk information into the organization.

The diagram is composed of three main areas with two of them depicting the Agency (Enterprise Risk Management and Program/Branch Risk Management). The two Agency areas are shown as reporting to the Minister of Health. The Enterprise Risk Management area depicts the committee structure of the organization in terms of risk management. It not only identifies the Public Health Agency of Canada Executive Management Committee Audit Committee, and Risk Management Oversight Committee which are involved with the Corporate Risk Profile; it also highlights the Daily Assistant Deputy Minister meeting and daily reports which represent how the Agency manages risk on a regular basis.

The Program/Branch Risk Management area depicts how the organization manages risk at the branch level and how this information is related to the Corporate Risk Profile and the operational planning process.

Overlaying, these two main Agency areas are three circles that demonstrate the relationship between the Agency's "Daily" risk management process, the Corporate Risk Profile and the operational planning and reporting process.

The last part of the diagram is a depiction of how the Global Public Health Environment impacts the overall activities of the Agency, and how there is a two way flow of information between the Agency and its global public health partners. The impact of this information is considered in both risk management and operational decisions.

Enlarge Figure 3

Appendix E - CRP voting criteria

In a facilitated workshop, a voting technology (Resolver Ballot) was used to rank Agency risks against the following criteria.

Risk Impact Criteria

Risk Impact Rating 5= Very High

Impact Descriptors

  • Significant impact on public health (PH) systems or health of populations (on scale of: Regions = 500,000, Province, Territory or Canada)
  • Significant impact on Agency capacity/programs/results (on scale of: Agency, Branch, Program Activity (PA) or SA (Sub-Activity)
  • Significant impact on Agency reputation/credibility (on scale of: external blue ribbon commissions/reports; sustained national attention/publicity)
  • Widespread change in stakeholder trust/willingness to work with the Agency

Risk Impact Rating 4= High

Impact Descriptors

  • Significant impact on PH systems or health of populations (on scale of: Regions 100,000 = population < 500,000)
  • Significant impact on Agency capacity/program/results (on scale of: Directorate or SSA)
  • Significant impact on Agency reputation/credibility (on scale of: external authoritative reports; sustained regional attention/publicity)
  • Considerable change in stakeholder trust/willingness to work with the Agency

Risk Impact Rating 3= Medium

Impact Descriptors

  • Significant impact on PH systems or health of populations (on scale of: Regions < 100,000)
  • Significant impact on Agency capacity/programs/results (on scale of: Division or multiple Planned Business Activities)
  • Significant impact on Agency reputation/credibility (on scale of: internal reports/audits; sustained local attention/publicity)
  • Some change in stakeholder trust/willingness to work with the Agency

Risk Impact Rating 2= Minor

Impact Descriptors

  • Significant impact on Agency capacity/programs/results (on scale of: Unit or single Planned Business Activity)

Risk Impact Rating 1= Negligible

Impact Descriptors

  • No appreciable impact

 

Risk Probability Criteria
Risk Probability Rating Probability Descriptors Approximate probability equivalents in terms of frequency
5= Very Probable 90-100 % / yr. (90 < p ≤ 100) Once per year
4= Probable 60-90 % / yr. (60 < p ≤ 90) Once every 1 – 2 years
3= Somewhat Probable 30-60 % /yr. (30 < p ≤ 60) Once every 2 – 3 years
2= Improbable 10-30 % / yr. (10 < p ≤30) Once every 3 – 10 years
1= Very Improbable 0-10 % / yr. (0 ≤ p ≤10) Once every 10 or more years

Source: Public Health Agency of Canada's Corporate Risk Profile 2013-15. Approved March 2014


Page details

Date modified: