Audit of the Management of Public Health Workforce Development

January 2013

For readers interested in the PDF version, the document is available for downloading or viewing:

Final Audit Report - Audit of the Management of Public Health Workforce Development (PDF document - 195 KB- 25 pages)


Executive summary

The Public Health Agency of Canada (the Agency) undertakes a variety of Public Health Workforce Development (PHWD) activities in collaboration with its stakeholders (for example, provinces/territories), across a variety of areas of focus including public health competency based training, public health professional recruitment and mobilization and the funding of public health capacity tools and public health scholarships. The Office of Public Health Practice (OPHP) has lead responsibility for the coordination and delivery of a number of these program activities. In addition, there are various PHWD activities delivered by other branches within the Agency.

The objective of the audit was to provide assurance that there is a management control framework in place to effectively manage PHWD across the Agency. In relation to the management control framework, the audit focused on governance considerations including mandate, strategic directions and governance structures. The audit also focused on risk management and operational management practices including operational planning, stakeholder communications and collaboration and performance measurement. The audit was conducted in accordance with the Internal Auditing Standards for the Government of Canada and the International Standards for the Professional Practices of Internal Auditing. Sufficient and appropriate procedures were performed and evidence gathered to support the audit conclusion.

The Agency has generally articulated its strategic focus on building public health capacity, which includes PHWD; however, the Agency's mandate in relation to PHWD and its various facets (for example, training, recruitment, capacity funding, public health competency development and implementation etc.) has not been clearly articulated.

Similarly, strategic directions related to PHWD have been generally articulated in various forms (for example, Agency Strategic Plan, Agency Operations Plan), however, these various directional documents present an incomplete picture of PHWD within the Agency as they primarily focus on the activities of the OPHP versus the broader PHWD activities undertaken across the Agency.

External governance structures currently in place, such as the Public Health Infrastructure Committee, co-chaired by the Director General, OPHP, provide an opportunity for the Agency to interact, maintain communications and collaborate with key stakeholders (for example, provinces/territories) involved in PHWD. Within the Agency, governance structures have been clearly established within OPHP, however, beyond the Executive Committee, there is no formal governance structure in place to oversee and coordinate Agency-wide PHWD efforts.

PHWD program risks have been articulated as part of the development of the Agency's Corporate Risk Profile and risk mitigation approaches have been established. At an operational level, various risk management methods and tools are used in support of PHWD program activity areas; however, there is a lack of consistency in the risk management approaches employed.

Similarly, there are a variety of operational planning and monitoring methods and tools maintained in support of the management of individual PHWD program activities. Additional focus is required to ensure the implementation of a consistent and systematic operational planning approach to ensure that PHWD program activities and accomplishments are meeting Agency management expectations.

The Agency has access to various fora to communicate with key external stakeholders, including provincial/territorial representatives. In addition, from an operational perspective, there is a management focus on establishing communication channels and communicating with the stakeholders of PHWD program activities. For PHWD program activities (for example, Skills Online, Public Health Scholarships and the Capacity Building Initiative), OPHP management has reviewed and implemented various collaborative service delivery models to support the achievement of program activity objectives.

In conclusion, the Agency has established elements of a management control framework, however management attention is required to address identified gaps, including:

  • Development of a comprehensive mandate, related strategic objectives and supporting strategies for the Agency in relation to its PHWD program delivery;
  • establishment of a clear internal governance structure to support integrated delivery of PHWD program activities across the Agency;
  • expansion and formalisation of a risk management process in relation to the achievement of OPHP and program objectives;
  • definition and communication of OPHP requirements for operational planning and monitoring of PHWD programs and activities; and
  • enhancement of OPHP's performance measurement approach to include a consistent focus on PHWD activities, outputs and results in order to better inform management decision-making.

Management has agreed with the six recommendations and provided a detailed action plan that, once implemented, will strengthen the effectiveness of the management control framework supporting the delivery of PHWD within the Agency.

A. Introduction

1. Background

Public Health Workforce Development (PHWD) is a component of the Public Health Preparedness and Capacity program activity and represents a strategic focus for the Agency.

This program is intended to support public health decision-makers in putting the right people in the right place at the right time to ensure that the public health system has the right competencies, capabilities and connections to carry out public health functions. The Program identifies and facilitates the closing of gaps in Canada's public health human resource capacity, the ability to prepare for and respond to public health issues and events and the functioning of public health networks within and outside Canada.

The program has evolved from the 2003 First Minister's Accord on Health Care Renewal, through which the provinces, territories and federal governments made a commitment to work together to improve health human resources planning and management. In the 2004 Ten-Year Plan to Strengthen Health Care, first ministers reaffirmed their commitment to strengthening public health.

Stemming from this direction, in 2005, the Joint Task Group on Public Health Human Resources issued a Pan-Canadian Framework for Public Health Human Resources Planning entitled “Building the Public Health Workforce for the 21st Century”. This framework outlines the following vision:

“Through collaborative planning, all jurisdictions in Canada will have a flexible, knowledgeable public health workforce working in safe supportive environments to meet the population's public health needs, and reduce health and social disparities”

The framework also articulates a series of goals, strategies and actions required to enhance public health delivery in Canada and to articulate the Public Health Agency of Canada's (the Agency's) role as follows:

“The Chief Public Health Officer and the Public Health Agency of Canada (the Agency) will provide leadership in many activities designed to establish standards and ensure consistency within the public health sector. The Agency provides a structure that is focused on public health, and will promote collaboration with provinces and territories to renew public health in Canada and support a sustainable health care system”

Currently, the Office of Public Health Practice (OPHP) has lead responsibility for the coordination, and in some cases, the delivery of core PHWD program elements. Within OPHP, the Field Service Training and Response Division and the Policy, Partnerships and Professional Development Division (representing 121 of 205 OPHP full-time equivalents as at June, 2012) are primarily responsible for PHWD program activities.

OPHP Organization - Full Time Employees
OPHP Organization FTEs
Director General Office 8
Field Service Training and Response Division 66
Policy, Partnerships and Professional Development Division 55
Knowledge Information Data Systems Division 25
Planning, Business Integration and Reporting Division 40
Surveillance Coordination Unit 5
Population Health Assessment and Scenarios Team 6
Total 205

Actual expenditures for 2011-12 for the Office totalled $34.9M, consisting of $16.5M in salaries, $7.3M in operations and maintenance expenditures and $11.1M in grants and contributions.

Key program activities managed by the OPHP divisions, highlighted in the table above, include:

  • Public Health Human Resource Planning – including the development of core competencies for public health and the provision of secretariat support for the Public Health Human Resources Task Group;
  • Public Health Agency of Canada Public Health Professional Development - including the administration of the Executive Committee (EC) Development Program delivered in partnership with other Agency functions (for example, human resources) and the Masters of Public Health/Masters of Public Administration Internship Program;
  • Skills Online Delivery and Development – representing an internet-based continuing education program for public health professionals in Canada;
  • Field Service Training and Response – including the delivery of the Canadian Field Epidemiology Program and the Canadian Public Health Service Program;
  • Public Health and Preventative Medicine Residents – a program involving the placement of public health and preventative medicine residents within the Agency;
  • Public Health Scholarships and Capacity Building – including the Workforce Development Products and Tools Contribution Program and the Public Health Scholarship and Fellowship Grant Program.

In addition to these OPHP-led activities, there are a number of other PHWD activities undertaken within the Agency. Examples include:

  • Laboratory Liaison Technical Officers Program – a program involving the secondment of the Agency resources to supplement provincial/territorial public health capacity. This program is administered by the Infectious Disease Prevention and Control Branch;
  • HIV Field Surveillance Officers - who seek to improve HIV/AIDS surveillance in Canada by providing assistance through a Field Surveillance Officer network to provinces and territories for system development, implementation and support of surveillance activities;
  • Emergency Response Training Programs – the Emergency Management and Regulatory Affairs Branch delivers several training programs including emergency preparedness and response training for federal employees, Chemical, Biological, Radiological, Nuclear, Explosive (CBRNE) “1st receivers” training, and CBRNE “1st responders” training.
  • Health Professional Surge Mechanism – a program focused on evolving the current health emergency response teams roster to a pan-Canadian Health Professional Surge Mechanism. This Program is administered by the Emergency Management and Regulatory Affairs Branch.

The Agency works with a variety of stakeholders in facilitating PHWD efforts. This includes collaboration with:

  • Provincial/territorial agencies and organizations responsible for public health (for example, ministries of health and education, regional and local health authorities, training institutions and regulatory bodies);
  • Federal agencies (for example, Health Canada and research agencies such as the Canadian Institutes of Health Research, and the Canadian Institute for Health Information);
  • Municipal governments that fund or deliver public health services; and
  • Non-governmental organizations (for example, the British Columbia Institute of Technology in support of the Skills Online Program, the Canadian Public Health Association).

In addition to ongoing program delivery, the Agency, as part of its business transformation agenda, has commenced an initiative that will impact PHWD. Through this initiative, the Agency will “review its role in building and supporting public health capacity. This will ensure that efforts are aligned with our mandate, and recognize the responsibilities and capacity of other jurisdictions and partners.”Footnote 1 This initiative will be completed over the next two years.

2. Audit objective

The objective of this audit was to provide assurance that there is a management control framework in place to effectively manage Public Health Workforce Development across the Agency.

3. Audit scope

The scope of the audit included an assessment of Agency management processes related to the planning and delivery of PHWD activities. Specifically, the audit addressed management processes in relation to the planning, delivery and coordination of PHWD activities by the OPHP, as well as identified PHWD activities undertaken by other Agency branches (for example, Infectious Disease Prevention and Control, Emergency Management and Regulatory Affairs).

The scope of the audit covered program activities undertaken during the fiscal years 2010-11, 2011-12, and 2012-13 to date. The audit was conducted from June to September 2012.

4. Audit approach

The audit methodology included the review of documentation, policies, processes and procedures related to the administration of Agency PHWD activities; interviews with Agency key personnel as well as representatives of program partners; inquiry and analysis.

The audit criteria, outlined in Appendix A, were derived from the Office of the Comptroller General Internal Audit Sector's Audit Criteria Related to the Management Accountability Framework: A Tool for Internal Auditors (March 2011).

5. Statement of assurance

In the professional judgment of the Chief Audit Executive, sufficient and appropriate procedures were performed and evidence gathered to support the accuracy of the audit conclusion. The audit findings and conclusion are based on a comparison of the conditions that existed as of the date of the audit, against established criteria that were agreed upon with management. Further, the evidence was gathered in accordance with the Internal Auditing Standards for the Government of Canada and the International Standards for the Professional Practice of Internal Auditing.


B. Findings recommendations and management responses

1. Governance

1.1 Mandate

Audit criterion: A clearly communicated mandate with defined roles and responsibilities has been established for Public Health Workforce Development within the Agency.

A clearly defined program mandate supports an organization in articulating the focus and the boundaries for program activities. It also supports the formulation of clear program strategic directions.

The Agency's strategic focus related to public health capacity which includes Public Health Workforce Development (PHWD), has been generally articulated in the Agency Strategic Plan 2007–12. Within this Plan, there is recognition that, across Canada, there remains a lack of qualified public health professionals, gaps in systems for communications and information sharing and uneven resources and capacity across jurisdictions. In response to these challenges, the Plan indicates that “The Agency is working in partnership with stakeholders and other governments to develop strategies that will address these gaps in capacity which limit our ability to take action on major health issues”.Footnote 2

In support of PHWD, the Agency currently has a number of areas of focus and performs a variety of activities as illustrated on the following page.

Focus

Development and delivery of
public health competency
based training
to internal-PHAC
and external audiences.

Example Program Activities

•  Skills On-line trianing modules,
•  Public Health 101
•  First responders training, and First
    receivers training.

Public Health professional
recruitment and mobilization
internally within PHAC, and in
support of external organizations.

 
•  EC Development Program,
•  Canadian Field Epidemiology Program,
•  Canadian Public Health Service.

PHWD capacity funding, via
grants and contributions.

•  Public health capacity tools funding
•  Public health scholarship funding

While the Agency delivers a range of program activities in support of PHWD, the audit found that questions remain as to the nature and extent of the Agency's mandate in relation to the various facets of PHWD (for example, competency based training, recruitment of public health professionals).

As referenced in foundational documents supporting the Agency's ongoing Business Transformation Initiative, public health capacity and public health workforce development are not well defined. In addition, OPHP leadership acknowledges that “capacity building” could include many things that the Agency already does but there is no coherent vision, strategy, prioritization or assessment of the impact of investments.

Based on an analysis of available documentation and interviews with Agency representatives, the Agency's mandate focus has not been clearly articulated in relation to the various areas of PHWD, however, OPHP management indicated that this is one of the objectives of the Business Transformation Initiative.

Recommendation 1

It is recommended that the Director General, Office of Public Health Practice develop a comprehensive Agency mandate statement in relation to Public Health Workforce Development and obtain Executive Committee approval.

Management response

Management agrees with the recommendation noting that the Agency's Public Health Workforce Development mandate must be developed in collaboration with internal Agency stakeholders, including the Director General of human resources in the area of workforce capacity building and in collaboration with federal/provincial/territorial stakeholders. Much of the work required to develop a comprehensive Agency Public Health Workforce Development mandate statement will occur as part of the workforce capacity business transformation initiative. The Director General, Office of Public Health Practice will engage relevant stakeholders within the Agency to develop a mandate. The Agency will engage external partners via the public health network to validate a shared understanding of the mandate across Canadian jurisdictions.


1.2 Strategic directions

Audit criterion: The Agency has clearly defined and communicated strategic directions and objectives for Public Health Workforce Development, aligned with its mandate.

The Agency has generally articulated its strategic directions related to PHWD in various forms.

For example, the Agency Strategic Plan (2007–12) articulates that the Agency will place a priority on collaborating with other levels of government to fill gaps in public health human resources, both within the Agency and in other jurisdictions. The Strategic Plan also indicates that the Agency will work with partners to provide leadership in various areas including training and skills development.

As another example of a strategic direction document, the 2012-13 Agency Operational Plan for OPHP outlines a number of priority activities that will be undertaken toward strengthening public health capacity, including PHWD. This document however focuses only on OPHP initiatives and not on the broader Agency view.

Similarly, OPHP has developed a draft PHWD logic model that articulates the relationship between OPHP activities (for example, Canadian Public Health Service, Public Health Scholarships and Capacity Building Program), their outputs and their outcomes. This logic model only focuses on the activities of OPHP and does not encompass a broader Agency view of PHWD.

While these various forms of direction setting documents and guidance exist, the audit confirmed that a comprehensive strategic plan has not yet been articulated for PHWD within the Agency. As part of the Business Transformation Initiative, one of the first steps towards defining the broader strategy and strategic plan is being undertaken whereby the Agency is in the process of creating an “inventory” of recognized PHWD programs and activities across the Agency.

This “strategic directions” gap has been confirmed with senior program leadership and represents a focal point for the ongoing Business Transformation Initiative which, according to the 2012-13 Agency Operational Plan, will result in the development of an overarching vision or strategy articulating the Agency's role and contribution for pan-Canadian Public Health Capacity Building, including PHWD.

Recommendation 2

It is recommended that the Director General, Office of Public Health Practice articulate a strategic plan, including clear strategic objectives and supporting strategies, to describe the Agency's intent in supporting Public Health Workforce Development.

Management response

Management agrees with the recommendation noting that the Strategic Plan must be developed in collaboration with other Agency branches, including the Director General of human resources. The Director General, Office of Public Health Practice suggests the further step of obtaining the Agency Executive Committee approval of the Strategic Plan prior to implementation. The Workforce Capacity Business Transformation Initiative will develop a strategic plan for the Agency's Public Health Workforce Development activities based on a clearly defined mandate and with significant consultation with the Agency branches through individual Branch Executive Committees and discussions at the Assistant Deputy Minister Steering Committee. This consultation will identify and validate the elements of a strategic plan to ensure that it aligns with business and program objectives, such as training, skills management and identification of workforce needs. Further consultation with external stakeholders through the public health network will be undertaken as required.

1.3 Governance structure

Audit criterion: The Agency has clearly defined governance structures in place to support delivery against program objectives and incorporate all key program delivery partners.

The Agency interacts with a number of external stakeholders (for example, provinces and territories, academic institutions) in collaborating and delivering various PHWD program activities. In addition, the Agency works with various committees and task groups in establishing direction, soliciting/providing program advice and delivering PHWD program activities. These fora include:

Public Health Infrastructure Committee – Co-chaired by the DG, OPHP, this Committee supports the Pan-Canadian Public Health Network (PHN) by providing public health policy advice and recommendations to PHN Council regarding pan-Canadian collaborative approaches to public health policy in Canada. The Committee, comprised of senior government officials representing federal, provincial and territorial jurisdictions, also oversees the development of deliverables to support and advance the strategic priorities of the Pan-Canadian PHN.

Public Health Human Resources Task Group – Accountable to the Public Health Network Council, the mandate of this task group, made up of pan-Canadian public health experts, was to advance the implementation of the Public Health Human Resources Planning Framework (Building the Public Health Workforce for the 21st Century: A Pan-Canadian Framework for Public Health Human Resources Planning). This task group, which was co-chaired by an Agency representative and was supported via secretariat services through OPHP, was disbanded in May, 2012.

Within the Agency, there are a number of PHWD program activities delivered. These activities are managed by various functions across the Agency.

OPHP has delivery responsibility for a number of PHWD programs including the Canadian Field Epidemiology Program (CFEP), the Canadian Public Health Service (CPHS), and Skills Online. For these programs, governance considerations (for example, assignment of roles/responsibilities, decision-making etc.) are addressed within the scope of the OPHP organization which includes defined organization charts and assigned responsibilities for the leadership of the various program areas.

In addition to OPHP-led program activities, there are a number of PHWD activities led by other directorates within the Agency (for example, the Laboratory Liaison Technical Officers Program which is administered by the Infectious Disease Prevention and Control Branch). These programs are governed under their own branches/directorates.

In the current environment, the Agency has not established an overarching governance structure in support of PHWD program activities (including those led by OPHP and other branches). Broader governance considerations are addressed by the Agency's Executive Committee (EC).

The audit identified one example where the Agency had attempted to establish an overarching governance structure for one component of PHWD, that being an Agency-wide approach to public health competency development on the part of internal Agency staff. As part of this approach, a governance committee, the Public Health Agency of Canada Learning Council, was established in 2009. This committee co-chaired by OPHP and the Human Resources Directorate representatives, included representation from organizational units throughout the Agency. At the time of audit, however, the Council was not active.

This governance gap, that is to say the Agency-wide PHWD governance, is recognized in the 2012-13 Agency Operational Plan as observed by the risk that “there is a lack of consensus throughout the Agency as to how to play this role” in reference to Agency efforts toward strengthening public health capacity.

In response to this risk, one of the areas of focus for the Business Transformation Initiative will involve OPHP engaging Agency organizations in the clearer definition and articulation of an overarching governance structure to guide an integrated approach to PHWD program activities across the Agency.

Recommendation 3

It is recommended that the Director General, Office of Public Health Practice establish a formal internal governance structure to provide direction and oversight to the delivery of Agency Public Health Workforce Development activities.

Management response

Management agrees with the recommendation and recognizes that there is value to be gained from using governing systems that are already in place. The Director General, the Office of Public Health Practice will consult with branch executive committees and/or the Assistant Deputy Minister Steering Committee to explore the feasibility of utilizing existing governing structures, specifically the Workforce Management Committee, to provide direction and oversight to the Agency's Public Health Workforce Development activities. The Public Health Workforce Development Strategic Plan will address the changes required to the governance structures and processes to enable decisions to be made in alignment with Agency priorities and objectives.

2. Risk management

2.1 Risk management

Audit criterion: Management identifies program and operational risks that may preclude the achievement of its objectives, and it develops and maintains appropriate risk management strategies.

The Agency's Policy on Integrated Risk Management states that risk management involves systematically considering the potential effects of risk on achieving objectives, and addressing key risks to objectives through appropriate decision-making and risk treatment actions. The Policy also states that “risk management shall be a consideration in strategic and operational planning and reporting activities”.

From a PHWD program perspective, it was noted that the following risk has been identified within the 2011 Corporate Risk Profile:

“There is a risk that Canada lacks the appropriately trained workforce, tools, organizational capability and inter jurisdictional systems to respond to and plan for public health threats”

It was further noted that the Corporate Risk Profile includes the proposal of two initial strategies to address this risk. The OPHP, within its 2012-13 draft Agency Operational Plan, further elaborates on these risk mitigation approaches in response to the risk identified in the Corporate Risk Profile.

From an operational risk perspective, in relation to PHWD, it was noted that OPHP has introduced limited elements of risk management into its operational work planning and monitoring processes. For example, some individual PHWD program/activity work plans and/or status reports were found to contain descriptions of risks and mitigation strategies although these risk tools varied in their consistency and completeness.

Illustrative examples of the operational types of risks identified include the risk of inadequate resourcing to support a specific program area or initiative, or the risk that an internal reorganization may delay delivery against a project milestone.

The audit noted other PHWD program/activity examples where operational risks had not been explicitly identified, nor had risk management strategies been developed. Discussions with program/activity managers confirmed that risk management in these cases was conducted in an ad hoc manner.

Based on the analysis, additional management focus is required to ensure that a consistent and systematic risk management approach is implemented across the PHWD program areas in order to support the appropriate identification and management of risks that may preclude the achievement of PHWD program/activity objectives.

Recommendation 4

It is recommended that the Director General, Office of Public Health Practice expand and formalize the risk management process to include the implementation of consistent processes for identifying and managing risks as part of its Public Health Workforce Development management processes.

Management response

Management agrees with the recommendation and highlights the importance of the Director General of human Resources and Risk Management Oversight Committee involvement in this process. The Director General, Office of Public Health Practice will ensure that a comprehensive risk management process is clearly identified in the Agency's Public Health Workforce Development Strategic Plan. The Director General, Office of Public Health Practice will ensure that improved risk management practices related to Public Health Workforce Development are highlighted in the next iteration of the Agency's Corporate Risk Profile. Internally, risks will be managed through the existing monthly reporting as part of the performance measurement framework.

3. Operational management

3.1 Operational planning

Audit criterion: The organization has in place operational plans and objectives aimed at achieving its strategic objectives and commitments.

In support of the planning and delivery of PHWD program activities, OPHP has implemented an organizational structure in which each of the PHWD program/activity areas are responsible for their own operational plans.

Through analysis, it was identified that OPHP has developed annual operational plans (for example, Integrated Operations Plan and Agency Operational Plan) to summarize the activities and resources of the Office, as required under the Agency operational planning requirements.

In regard to individual PHWD program/activities within OPHP, it was noted that the Office maintains a variety of operational planning and monitoring tools in support of the management of individual program activities.

Illustrative operational planning tools developed for specific program activities ranged from summary presentations of a GANTT chart highlighting quarterly activities and outputs for some program activities (for example, Canadian Public Health Service and Canadian Field Epidemiology Program), to other examples where operational plans identified program activities, deliverables, success indicators, budgets and timelines (for example, Skills Online).

Based on the review of operational planning tools used for various PHWD programs/activities, the audit identified that the levels of information available were inconsistent. Examples of inconsistencies included the level of detail and/or availability of information in operational plans relating to:

  • expected results for the program/activity;
  • performance indicators;
  • planned resources (operations & maintenance expenditures and human resource levels); and
  • key tasks, deliverables and timing.

In reviewing the available PHWD program/activity monitoring tools that are used to manage PHWD programs/activities, the audit similarly identified a lack of consistency in the status information available to support the systematic monitoring of program/activity operational plans.

Based on the analysis, additional management focus is required to ensure that a consistent and systematic operational planning and monitoring approach is implemented across the PHWD program areas under the responsibility of OPHP.

Recommendation 5

It is recommended that the Director General, Office of Public Health Practice establish a common process for Public Health Workforce Development program/project planning to ensure a systematic approach to operational planning and monitoring.

Management response

Management agrees with the recommendation noting that the operational planning and monitoring processes must be developed in collaboration with other Agency branches and with the Director General of human resources.

The Strategic Plan and governance guidelines will include frameworks to assist the Agency branches to coordinate the Public Health Workforce Development planning and monitoring activities.

The Director General, Office of Public Health Practice will convene a group of Agency stakeholders to draft Public Health Workforce Development planning and monitoring systems that feed in to the Agency Operational Plan. It is strongly recommended that this activity be undertaken in partnership with the Director General of human resources.

3.2 Stakeholder communications

Audit criterion: Effective communication exists between the organization, its clients and other internal and external stakeholders to support timely, relevant and reliable information sharing.

In delivering PHWD program activities, the Agency collaborates and interacts with a broad range of partners and stakeholders including provinces, territories, academic institutions and delivery partners, such as the B.C. Institute of Technology that provides technology infrastructure support of the Skills Online program activity.

From a broad PHWD perspective, it was noted that the Agency and OPHP have access through the Public Health Infrastructure Steering Committee to communicate with external stakeholders including provincial and territorial representatives. This steering committee is co-chaired by the DG, OPHP. The Public Health Human Resources Task Group was also utilized to communicate with external stakeholders; however, this group was disbanded in May, 2012.

From an operational perspective, a number of communication mechanisms have been established to facilitate stakeholder communication in the various PHWD program areas. Illustrative examples include:

  • Canadian Field Epidemiology Program (CFEP) management maintains communication with its stakeholders on a monthly basis through an email newsletter informing them of issues and topics of concern for the Program. In addition, CFEP management conducts an annual meeting with its stakeholders (conference call) to identify and seek to resolve program issues.
  • Canadian Public Health Service management conducts placement site reviews which serve to generate feedback from placement sites (external stakeholders) regarding the work of the assigned public health officers. Site reviews are performed three times during the typical two year rotation of a public health officer.

It was observed that OPHP has established communication channels to support effective communication with its external stakeholders, both in terms of its overall PHWD focus (for example, communication with provincial/territorial representatives), and in terms of its individual program activities (for example, communication with a Canadian Public Health Service placement site supervisor).

3.3 Collaboration for service delivery

Audit criterion: The organization leverages, where appropriate, collaborative opportunities to enhance Public Health Workforce Development service delivery.

As noted in this report, the Agency has not yet clearly defined its mandate in relation to Public Health Workforce Development and this area for improvement is a focus of the Agency Business Transformation Initiative. Through this initiative, it is expected that the Agency will clearly establish:

  • the scope of the PHWD services that the Agency should be supporting;
  • the nature and extent of Agency roles in supporting PHWD service delivery (for example, providing public health capacity through placement programs versus building public health capacity through facilitation, funding, coordination etc.); and
  • the optimal service delivery model for those services (for example, the nature of interaction between the Agency and its service delivery partners such as, provinces/territories, non-governmental organizations, academia etc.).

Accordingly, OPHP management represents that the Business Transformation Initiative will include a focus on evaluating collaboration opportunities with partners in order to enhance PHWD service delivery.

While acknowledging this longer term initiative, a number of current state examples were observed where the Agency has actively reviewed collaboration opportunities and has implemented collaborative service delivery models. These include:

Skills Online – The Agency has engaged the BC Institute of Technology (BCIT) as a delivery partner for the program. In order to maximize efficiencies within the program, the Agency has outsourced its administrative infrastructure, wherein BCIT provides a comprehensive e-learning capability online course facilitation services and technical support. This arrangement is supported by a Memorandum of Agreement between the two parties.

The Public Health Scholarships and Capacity Building Program – This program, in part, supports the development of products and tools for training and professional development in public health that meet the needs of public health organizations and their staff. This program, facilitated via multi-year contribution agreements supports regional and national partnerships between and among universities and public health organizations.

Professional Development Division strategic planning – In 2011, the Professional Development Division of OPHP, which has the responsibility to deliver for PHWD initiatives such as Skills Online, developed a strategic plan. This process involved the Division engaging a variety of internal and external stakeholders in the assessment of current state and identification of improvement opportunities.

While questions remain regarding the Agency's longer term PHWD focus and approach, these examples recognize a focus on the part of OPHP to evaluate collaboration opportunities in relation to its various operational program areas as part of its continuous improvement efforts.

3.4 Performance measurement

Audit criterion: Management has identified appropriate performance measures linked to planned results, monitors actual performance against planned results and adjusts course as needed.

OPHP has drafted a PHWD logic model which articulates the relationship between OPHP activities (for example, Canadian Field Epidemiology Program, Canadian Public Health Service (CPHS), Public Health Scholarships and Capacity-building Program), their outputs and their outcomes. While this effort represents an initial step point in performance measurement identification, OPHP management confirms that the logic model has not been expanded into a full performance measurement framework, which would include the definition of appropriate performance measures related to program activities, outputs and outcomes.

It was noted that as part of the Integrated Operations Plan developed by OPHP in 2010-11, performance measures were identified for various component program areas of PHWD, but these program areas and related performance measures did not align with the program areas defined in the PHWD logic model. It was also noted that the defined performance measures have not been actively monitored and reported in a meaningful way to date (for example, reporting against the expected results defined for Public Health Workforce is expected to be initiated in March, 2014).

Within OPHP at a PHWD program activity level, it was noted that a variety of performance measurement tools and techniques have been established across individual PHWD activity areas (for example, Skills Online) to support the measurement and monitoring of program inputs, activities and outputs. Examples of these performance measurement tools and information include:

  • Program input measurement focus: Budget to actual salary and operations & maintenance expenditure comparisons for OPHP divisions and OPHP PHWD program areas (for example, Skills Online);
  • Program activity measurement focus: client satisfaction evaluations conducted with CPHS placement sites; measurement of the status of recommendations from a 2008 audit on the Canadian Public Health Service; and
  • Program output measurement focus: number of learners that have completed Skills Online training modules; number of CPHS public health officers hired.

Although it was noted that various performance measurement tools and reporting have been established throughout OPHP to support the measurement of program inputs, activities and outputs, beyond the measurement of financial resources, there is a lack of consistency in the definition of program activity and program output performance measures across the various OPHP PHWD program activity areas. Furthermore, expected results from the various program activity areas have not been consistently articulated raising questions as to how OPHP leadership is able to monitor progress and ensure that individual program activity areas are addressing management expectations.

Recommendation 6

It is recommended that the Director General, Office of Public Health Practice enhance the performance measurement approach for Public Health Workforce Development to include a comprehensive and consistent focus on the measurement of program activities, outputs and results in order to better inform management's decision-making.

Management response

Management agrees with the recommendation. The Agency is in the process of finalizing the performance measurement framework for Public Health Workforce Development as part of the development of the Program Alignment Architecture. The Agency's Public Health Workforce Development Strategic Plan will be a key element of Workforce Capacity Business Transformation Initiative. The Public Health Workforce Development strategic and operational plans will incorporate clear performance measurement indicators that will ensure that activities are aligned with overarching Program Alignment Architecture measures.

C. Conclusion

The Agency undertakes a variety of Public Health Workforce Development (PHWD) activities, primarily in the areas of public health competency based training, public health professional recruitment and mobilization, and the funding of public health capacity tools and public health scholarships. In delivering these activities, the Agency collaborates with a number of stakeholders including the provinces and territories.

The Agency has generally articulated its focus and strategic directions in supporting PHWD; however, its mandate and supporting strategies in relation to the specific facets of PHWD (for example, training, recruitment, capacity funding etc.) require additional clarification.

From a governance perspective, the external governance structures currently in place, such as the Public Health Infrastructure Committee, afford the Agency the opportunity to interact with key stakeholders. Internal governance structures have been clearly established within the Office of Public Health Practice (OPHP), however, beyond the Executive Committee, there is no formal governance structure in place to oversee and coordinate Agency-wide PHWD efforts.

From an operational management perspective, OPHP has established various management processes and tools to support risk management, operational planning and performance measurement; however additional focus is required to ensure that consistent and systematic operational management processes are implemented across PHWD program initiatives.

In conclusion, the Agency has established elements of a management control framework, however management attention is required to address a number of opportunities for improving the management control framework, including:

  • Development of a comprehensive mandate, related strategic objectives and supporting strategies for the Agency in relation to its PHWD program delivery;
  • establishment of a clear internal governance structure to support integrated delivery of PHWD program activities across the Agency;
  • expansion and formalisation of a risk management process in relation to achievement of OPHP and program objectives;
  • definition and communication of OPHP requirements for operational planning and monitoring of PHWD programs and activities; and
  • enhancement of OPHP's performance measurement approach to include a consistent focus on PHWD activities, outputs and results in order to better inform management's decision-making.

Management has agreed with the six recommendations and has provided a detailed action plan that, once implemented, will strengthen the effectiveness of the management control framework supporting the delivery of PHWD within the Agency.

Appendix A - Lines of enquiry and audit criteria

Audit of the Management of Public Health Workforce Development
Criteria Title Audit Criteria
Line of Enquiry 1: Governance
1.1 Mandate A clearly communicated mandate with defined roles and responsibilities has been established for Public Health Workforce Development (PHWD) within the Agency.
1.2 Strategic directions The Agency has clearly defined and communicated strategic directions and objectives for PHWD, aligned with its mandate.
1.3 Governance structure The Agency has clearly defined governance structures in place to support delivery against program objectives and incorporate all key program delivery partners.
Line of Enquiry 2: Risk management
2.1 Risk management Management identifies program and operational risks that may preclude the achievement of its objectives, and it develops and maintains appropriate risk management strategies.
Line of Enquiry 3: Operational management
3.1 Operational planning The organization has in place operational plans and objectives aimed at achieving its strategic objectives and commitments.
3.2 Stakeholder communications Effective communication exists between the organization, its clients and other internal and external stakeholders to support timely, relevant and reliable information sharing.
3.3 Collaboration for service delivery The organization leverages, where appropriate, collaborative opportunities to enhance Public Health Workforce Development service delivery.
3.4 Performance measurement Management has identified appropriate performance measures linked to planned results, monitors actual performance against planned results and adjusts course as needed.

Appendix B - Scorecard

The rating and supporting explanations summarize the current status for each audit criterion.

Criterion Rating Conclusion Rec
Satisfactory
Needs Minor Improvement
Needs Moderate Improvement
Needs Improvement
Unsatisfactory
Unknown; Cannot Be Measured
Governance
1.1 Mandate NI The Agency’s public health capacity mandate has been broadly articulated in various forms (for example, Strategic Plan, Report on Plans and Priorities etc.).  However, the Agency’s mandate in relation to the various facets of Public Health Workforce Development (PHWD) (for example, training, recruitment, capacity funding etc.) has not been clearly articulated. 1
1.2 Strategic directions NI Strategic directions related to PHWD have been articulated in various forms (for example, Strategic Plan, Agency Operational Plan), however, the focus of these various directional documents is on the activities of OPHP versus the broader activities of the Agency. Management attention is required to clearly articulate an Agency strategy for PHWD. 2
1.3 Governance structure NI External governance structures currently in place afford the Agency the opportunity to interact with key stakeholders.  Internal governance structures within the Office of Public Health Practice (OPHP) are clear, however, beyond the Executive Committee, there is no formal governance structure in place to oversee Agency-wide PHWD efforts beyond OPHP (for example, to oversee priority setting, investment decisions etc.). 3
Risk Management
2.1 Risk management NMO Broad program risks have been articulated as part of Agency corporate risk profiling efforts and mitigation approaches have been established. At an operations level, various risk management methods and tools are used in support of PHWD program areas; however, there is a lack of consistency in the risk management approaches taken.   4
Operational Management
3.1 Operational planning NMO OPHP maintains a variety of operational planning methods and tools in support of the management of individual PHWD program initiatives. Additional focus is required to ensure that a consistent and systematic work planning and monitoring approach is implemented across OPHP’s program areas of responsibility. 5
3.2 Stakeholder communications S From a broad program perspective, the Agency has access to various fora to communicate with key external stakeholders, including provincial/territorial representatives. From an operational perspective, evidence was noted of a management focus on establishing communication channels and communicating with program stakeholders. -
3.3 Collaboration for service delivery S For individual OPHP PHWD programs (for example, Skills Online, Public Health Scholarships and Capacity Building Initiative), evidence was noted of an OPHP management focus on reviewing and implementing collaborative service delivery models to support program objectives. -
3.4 Performance measurement NMO OPHP has developed a PHWD logic model but it has not yet been translated into a comprehensive performance measurement framework. At an operational level, various performance methods and tools are used; however, there is a lack of consistency in the performance measurement approaches taken across OPHP program areas. 6

Page details

Date modified: