ARCHIVED - Formative Evaluation of the National Collaborating Centres for Public Health Program (NCCPH)

1

PHAC ensures the NCCPH Program governance structures remain sound and effective

AGREE
The Agency will continue to ensure NCCPH Program is relevant and aligns with federal Government and PHAC requirements and to use the governance, accountability and performance measurement structures and processes in identifying and addressing gaps.

Performance measurement, Terms and Conditions of Contributions Agreements and other PHAC requirements for the NCCPH Program will continue to be implemented.

Annual $9.5 million program funding through the Contribution Agreements will be maintained, subject to available resources.

Monitoring, budget planning and funding allocation reports including:

Ongoing

Program secretariat

DG

NCCs progress reports,
work plan reviews,
annual risk assessments,
other reporting requirements (i.e., on-site visits to host organizations and NCCs with auditor (as requested by PHAC processes and subject to available resources)

Ongoing

Program secretariat

OPHP will continue to work with PHAC’s G&C Alignment activities.

Participation at meetings for G&C Renewal (i.e., PIT)

Q4 2008-09

Program secretariat

1.1 The Advisory Council and NCC Advisory Board structure be maintained

AGREE WITH CONDITIONS with 1.1
PHAC will revisit the relationship structure of the Advisory Council.

PHAC will review the relationship structure of the Advisory Council.

Continued support of the current AC relationship structure, and as revised

Description for supporting the approved process and procedures for amending the AC’s structure

Inclusion of revised AC relationship in the program guide (see in 2.1 below)

Q1 2009-10

Program secretariat

DG

1.2 The NCCPH Program Secretariat and Advisory Council review the role of the Advisory Council in view of the support needed by NCCs as they move beyond the implementation phase into stable organizations

AGREE WITH CLARIFICATION with 1.2
PHAC, in consultation with the NCCPH Program Secretariat and Advisory Council will revisit the technical advice and guidance role to the NCCs as
they develop into stable organizations. It should be noted that from the outset, the federal Government anticipated that it would take 15 years to see the full impact of this program.

NCCPH Program Secretariat and Advisory Council, will undertake an annual review of the Terms of Reference (TOR) of the Advisory Council.

Recommendation for TOR changes for G&C

Q4 2008-09
Q1 2009-10

Program secretariat

DG

1.3 PHAC Senior Management investigates and addresses underlying NCCPH Program Secretariat recruitment and retention issues

AGREE with 1.3
OPHP will address human resources issues to maintain the NCCPH Program Secretariat’s capacity to support NCCPH Program monitoring and development.

Current HR processes underway will be expedited.

Completion of HR processes now underway

Q4 2008-09

Director

DG

Human resources recruitment and retention strategy will be implemented to attract and retain qualified personnel.

Human resources strategy

Q2 2009-10

Program secretariat

DG

The OPHP will support staff training opportunities to build capacity for NCCPH Program.

List of training opportunities and trained staff

Q1 2009-10

Program secretariat

DG

2

PHAC, in collaboration with the Advisory Council and NCCs, review the existing NCCPH mandate, specifically to:

2.1 Review and clarify the parameters for NCC involvement in regards to knowledge generation, capacity building, advocacy and professional supports

AGREE WITH CLARIFICATION with 2.1
This recommendation does not reflect the context in which PHAC operates as an arm’s length funder to the NCCs.

The program secretariat already participates in strategic review with the Advisory Council and the NCCs for the purpose of providing guidance on the program’s sustainability.

As well, the recommendation to review the existing program mandate contradicts the overall evaluation findings that “... the fundamental design of the NCCPH model is appropriate and no major changes to the defining elements are suggested” (page i, from Executive Summary).

Program secretariat has begun work on a strategic sustainability plan with PHAC’s Advisory Council, the NCCs and the NCC Advisory Board Chairs to clarify the priorities and determine the direction and scope of activities in the next cycle of the program.

Strategic plan includes: communication plan,
completion of program guide and transition plans developed in collaboration with NCCs

Q3 2009-10

Program secretariat

DG

2.2 Review the organizing framework for the NCC areas of focus with a view to enhancing the logic, reconciling redundancies, and clarifying target audiences, and increasing the flexibility to address new and emerging priority areas

AGREE WITH CLARIFICATION with 2.2
See the response to 2.1 above and based on the “arm’s length” requirement of the Contribution Agreements. PHAC cannot direct how NCCs choose to organize their activities.

PHAC will continue to review and monitor the program structure and to clarify KSTE priorities and communicate these to the NCCs, primarily through consultations with the Advisory Council and public health stakeholders.

Review report of program logic model and reporting structures (through RFP process)

Q1 2009-10

Program secretariat

DG

2.3 Revisit NCC funding allocations in view of the decisions arising from the reviews proposed in 2.1 and 2.2

AGREE WITH CLARIFICATION with 2.3
PHAC will continue to support the NCCs through Contribution Agreements recognizing the arm’s length relationship as described in 2.1 and 2.2.

NCCPH Program Secretariat will continue to review and monitor funding requirements of NCCs to support proposed activities, subject to available funds.

G&C funding allocation and annual budget planning

Ongoing

Director

DG

3

PHAC, in consultation with the Advisory Council and NCCs, establish an independent mechanism to strengthen leadership, support and coordination.

AGREE WITH CLARIFICATION with 3.0
PHAC agrees with the concept of “establishing an independent mechanism”, however, this recommendation does not reflect the context in which PHAC operates as an arm’s length funder to the NCCs. It is not PHAC’s role“to establish an independent mechanism”.

This role is specific to collaboration among NCCs and is being addressed by NCCs with the creation of an enhanced common NCC Leads secretariat to increase numbers of collaborative projects and activities among the Centres.

As well, the Advisory Council also provides guidance and advice to NCCs through a peer review process which occurs 2 or 3 times per year. The feedback provided on the NCCs’ progress reports and workplans focuses on relevance, strengthening leadership, support and coordination.

Thus, PHAC has a limited role as indicated in 3.1 and 3.2 below.

3.1 The mechanism be designed to strengthen the following essential
functions:

- Strategic direction and leadership for the NCCPH Program in KSTE, including enhanced presence and linkages with national and international, partners, collaborators, and involved parties;

The NCCPH Program Secretariat and Advisory Council will continue to provide technical advice and support to the NCC leadership as required.
This includes assistance in broadening national KSTE public health networks.

PHAC has a major role to play in building KSTE capacity for public health across Canada. PHAC will advocate for the expansion of the KSTE mandate
within its divisions/programs and within other federal departments.

Interdepartmental forum to present NCCs activities and directions and facilitate broader networking. Will develop a list of potential mechanisms for interdepartmental KSTE dialogue.

Q1 2009-10

Program Secretariat

DG

-Technical KSTE support to NCCs, including further development of a common KSTE framework, enhancing collaborative effort towards KSTE strategies, processes and tools, reducing duplication of effort, and continued KSTE knowledge and capacity building for NCC staff; - NCC coordination activities, including but not limited to the existing NCC Leads secretariat functions, hosting and maintaining the common web portal, organizing the Summer Institute and other common forums, and further developing centralized systems that support all NCCs.

NCCs are working together to address these issues.

The Program secretariat will continue to link the NCCs with potential partners among the PHAC areas and other federal departments. PHAC’s Advisory Council are also providing guidance with these concerns.

NCCPH Program Secretariat and Advisory Council will continue to encourage leadership functions of the NCC Leads secretariat by providing guidance and support.

Continue to provide advice and guidance to NCCs through feedback mechanisms, such as the Advisory Council’s review of progress reports and workplans.

Communications through formal feedback letters and ongoing discussions (teleconferences etc.)

Ongoing

Program secretariat

DG

3.2 PHAC provide sufficient additional resources for the implementation of Recommendation 3.1.

Existing NCC and NCCPH Program Secretariat budgets would not cover the range of functions and activities envisioned

CLARIFICATION REQUIRED with 3.2

This recommendation contradicts the overall evaluation findings : “...findings do not suggest a need for an increase to the original total annual $9.15M NCC base funding envelope at this time”(Executive summary, p. ii)

NCCs will continue to receive an annual $9.5 million program funding through the Contribution Agreements as described in recommendation 1.0. PHAC has also provided additional G&C funding ($ 25k) for a Communications Officer for the common NCC Leads secretariat in Q4 2008-09. Request for additional funds will be considered if possible.

4

The NCCPH Program Secretariat meets with host organization representative(s) to determine and resolve any concerns identified

AGREE WITH CLARIFICATION
The NCCPH Program Secretariat has welcomed discussion of concerns with host organizations. However, the role of host organizations is not to direct the activities of the NCCs which in some instances, has been a concern.

Discussions with host organizations will continue as part of the planned on-site visits for performance and financial reviews with external auditors (subject to available resources) as well as through other meetings.

On-site visits to host organizations subject to available resources

Meeting reports with host organizations

ongoing

Director

Program Secretariat

Director

5

The NCCPH Program Secretariat, in consultation with the Advisory Council and NCCs, determines a long term strategy for sustainability of the NCCs. This strategy should include considering the appropriate approach to and parameters associated with accessing funding from other sources

AGREE
The NCCPH Program is a sustainable “A- based” program.

The NCCPH Program Secretariat, Advisory Council, NCCs, NCC Advisory Boards Chairs are already in the process of developing a plan to address program sustainability. (See response to 2.1 above)

NCCs need to explore other potential funding sources for
collaborations and partnerships for successful independence and long term program sustainability. Already some of the NCCs have leveraged considerable funds for their work.

See 2.1 above

See 2.1 above

6

PHAC review the NCC reporting requirements and processes with a view to reducing the paper work burden.

AGREE
The NCCPH Program Secretariat has monitored the progress of NCCs, has seen positive performance improvements and acknowledges the reporting burden. The achievements of NCCs have been validated through this formative evaluation.

The reporting concerns will be discussed at the next PHAC’s Advisory Council meeting for their recommendations.

Report of PHAC’s Advisory Council meeting

Q4 2008-09

Director
Program Secretariat

DG

The Program secretariat is developing an online reporting tool that will reduce the reporting burden.

Online reporting tool

Q3 2009-10

IMIT

IMIT

6.1 The amount of reporting required of NCCs should be in proportion to the assessed risk.

See 6.0 above

6.2 For well established NCCs with low assessed risk, reporting should be reduced to once annually and be results rather than activity based

See 6.0 above

7

PHAC clarify and communicate to PHAC programs and NCCs the expectations of appropriate mechanisms for the NCCs to collaborate and link with, and provide KSTE products and services to PHAC programs and services.

AGREE WITH CLARIFICATION
The Program Secretariat has encouraged and will continue to encourage PHAC programs to contact the NCCs to do collaborative projects. These can be done via contracts, MOUs, etc. However, as stated above, PHAC cannot direct the activities of NCCs.

8

Revise the NCCPH Program logic model prior to initiating a summative evaluation of the NCCPH Program

AGREE
The NCCPH Program Secretariat, in consultation with PHAC and stakeholders will develop an action plan to set priorities and prepare for the summative evaluation process for 2010-2011.

For the summative evaluation, an action plan will be developed with contracts through an RFP process, with the support of PHAC’s Centre of Excellence in Evaluation and Program Design (CEEPD). This will include a review of the logic model as stated above.

Summative Evaluation action plan report

Q4 2009-10

Director
CEEPD Program Secretariat

DG
CEEPD