Evaluation of the Canadian Association of Physician Assistants Contribution Program (CAPA)

May 2016

1258-230 (ADM(RS))

Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

Acronyms and Abbreviations

Executive Summary

This report presents the findings and recommendations of the Evaluation of the Canadian Association of Physician Assistants (CAPA) Contribution Program. The evaluation was conducted by Assistant Deputy Minister (Review Services) (ADM(RS)) between January and July 2015, in compliance with the Treasury Board Secretariat (TBS) Policy on Evaluation (2009). As per the TBS policy, the evaluation examined the relevance and the performance of the CAPA Contribution Program over a five-year period (fiscal year (FY) 2009/10 to FY 2013/14).

Background

The CAPA is a not-for-profit association founded in 1999 by the Canadian Forces Medical Services School to create, promote and advocate Physician Assistant (PA) as a recognized health occupation.¹ In 2003, the Canadian Medical Association (CMA) officially recognized the PA as a credentialed health science profession in Canada. The CAPA represents and certifies all PAs in Canada, both military and civilian.

The CAPA Contribution Program agreement was for $5 million for five years (FY 2009/10 to FY 2013/14), which was then extended for two additional years in 2014 with no additional funding. The current terms and conditions of the CAPA Contribution Program are effective until March 31, 2016.

Program Description

The CAPA Contribution Program was initiated in 2009 to provide financial support to the CAPA. The CAPA, under the auspices of the CMA, supports the PA profession and sets professional standards through independent national governance and representation. For the Canadian Armed Forces (CAF), the CAPA provides certification for military PAs through a subsidiary certification council that authorizes military PAs to be employed in any clinical environment. The CAPA also provides the necessary National Competency Profile standards for the accreditation of the PA curriculum used at the Canadian Forces Health Services Training Centre (CFHSTC) and at Canadian universities.

Relevance

PAs provide an important role within the DND/CAF in support of the delivery of operational and domestic health care responsibilities. The CAPA provides the necessary independent national governance and representation of the PA profession, and without the contribution agreement, alternative funding would have to be found. However, some activities of the CAPA, such as the promotion of the PA profession across Canada, do not necessarily align with DND/CAF strategic outcomes.

Performance

The principal objective of the contribution agreement was to support the CAPA as it established its key objectives, including financial self-sufficiency. The CAPA directly contributed to the accreditation of the PA education program at the CFHSTC and the certification of PAs within the CAF. The CAPA has also been successful in implementing a National Competency Program for the PA profession, obtaining incorporation, increasing membership, establishing fees for certification, and developing professional programs. However, the CAPA had only limited success in promoting the PA profession nationally among provincial health care bodies, and it was not able to achieve the Contribution Program objective of reaching financial self-sufficiency.

Table 1 presents the CAPA Contribution Program objectives and an assessment of the degree of achievement. Although many objectives were achieved, the most critical CAPA objectives necessary to becoming a financially viable and fully self-sufficient organization were not achieved.

Table 1. Achievement of CAPA Contribution Program Objectives. This table provides an assessment of the level of achievement related to the CAPA Contribution Program objectives.

Conclusion

The role of the PA is a critical component of the CAF Health Services program. Significant efforts have been expended by the CAPA to establish the PA profession and to set the national standards and certifications. The program has had limited success in establishing the PA profession outside of the CAF and has not achieved the goal of financial self-sufficiency.

One of the objectives of the CAPA is “to contribute to the development of a nationwide health care profession.” This objective may help ensure the long-term viability of the PA profession; however, the funding of this objective is outside the mandate of the DND/CAF.

Continued support to the CAPA, through a DND-sponsored Contribution Program, is not recommended. If the Association does not become self-sufficient, CFHSG must take action to ensure the PA profession remains integral to the CAF Health Services Program.

Key Findings and Recommendation

Relevance

Key Finding 1: The Contribution Program permitted the CAPA to address a specific, demonstrable and continued need, which contributed to the health care of CAF members.

Key Finding 2: By contributing to the improvement of the CAF health care system and the Canadian health care system, the CAPA Contribution Program aligns with federal roles and responsibilities.

Key Finding 3: The CAPA Contribution Program was well aligned with government priorities.

Performance (Effectiveness)

Key Finding 4: Key stakeholders confirmed the emergent confidence of CAF Health Services customers in military PAs. However, the survey administered by the CFHSG did not provide a specific measurement for the PA profession.

Key Finding 5: With the sound management of scarce CAF Canadian Certified Physician Assistant (CCPA) resources, CFHSG has met all operational health care requirements without negatively impacting in-garrison health care needs. However, the CAF PA career path, as it is now, remains a concern and could affect operations in the future if not addressed.

Performance (Efficiency and Economy)

Key Finding 6: The CAPA made progress in developing some revenue-generating activities, but permanent sources of funding are presently not able to ensure the financial self-sufficiency of the organization.

Key Finding 7: ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

Key Finding 8: The CAPA could have generated other revenues without jeopardizing funding from the CAPA Contribution Program.

Key Finding 9: Progress was made towards the full achievement of Contribution Program outcomes.

Key Finding 10: The cost generated by the PA certification process is greater than certification process revenues.

Key Finding 11: The efficiency of delivering expected CAPA Contribution Program outputs was inconsistent.

Key Finding 12: The Contribution Program met many of the objectives required in the initiating documents, but did not succeed in achieving financial viability and self-sufficiency.

Key Finding 13: Limitations in policy restrict the ability of DND to reimburse CAF members for professional memberships in the CAPA

Note: Please refer to Annex A—Management Action Plan for the management responses to this recommendation.

1.0 Introduction

This evaluation report presents the findings and recommendations applicable to the Evaluation of the CAPA Contribution Program. The $5-million CAPA Contribution Program was initiated in 2009 for a five-year period. In 2014, TBS approved a two-year extension with no additional funding. This extension period will end on March 31, 2016.

The evaluation was conducted between January and July 2015 and was supported by an Evaluation Advisory Group comprised of key stakeholders. The evaluation was designed to meet the coverage required by the Financial Administration Act and the TBS policy applicable to contribution programs. In accordance with the Policy on Transfer Payments,³ this evaluation assesses the extent to which established CAPA Contribution Program objectives were achieved during the observation period (FY 2009/10 to FY 2013/14). Being an initial program, this is the first evaluation conducted of the CAPA Contribution Program. As such, there are no previous baselines to assess continuous improvement.

1.1 Context for the Evaluation

The 1999 Chief Review Services⁴ review of the Canadian Forces’ health care system identified significant deficiencies in the management and provision of health care in domestic and international operations. In addition, the closure of military hospitals in the 1990s necessitated that military health care providers obtain training in civilian health care facilities.

The Director General Health Services sponsored the establishment of the CAPA in 1999 to directly support and certify PAs, permitting them to work or train in civilian hospitals. In response to the 2007 OAG Report on Military Health Care, the Department and the Surgeon General established the CAPA as the professional certifying body⁵ for the PA profession. The CAPA also provides standards for the accreditation of the PA curriculum used at the CFHSTC and at Canadian universities.

From 1999 to 2001, the CAPA received some DND funding support and was staffed by member volunteers. In FY 2001/02, the CAPA started to receive funding from DND through the Rx 2000 Project to establish the Association as a national body through organizational and certification exam development. From FY 2004/05 to FY 2008/09, the Association received approximately $1.3 million from DND funding to cover salaries, development of the exam model, public relations, and marketing with the provinces and medical schools. The five-year $5-million CAPA Contribution Program commenced in FY 2009/10 with the primary objective of transitioning the CAPA from a DND-administered association to an independent national organization.

1.2 Program Profile

1.2.1 Program Description

The CAPA is a not-for-profit association founded in 1999 by the Canadian Forces Medical Services School to create, legitimize, and advocate PA as a recognized health occupation. The CAPA Contribution Program, initiated in 2009, aimed to transition the CAPA to become an independent, lead national association for the promotion of the PA profession in Canada.

1.2.2 Contribution Program Objectives

At the inception of the CAPA Contribution Program, the program objectives⁶ were, within a five-year period, the following:

• Make the CAPA a financially viable and fully self-sufficient organization;
• Obtain CAPA incorporation;
• Enter into revenue-generating and cost-efficient agreements with other provincial, national and international health care organizations;
• Obtain sponsorship support for events, research and awards, and lastly, revenues from donations and individuals, research agencies, global companies, and foundations;
• Increase the membership;
• Ensure the establishment of fees from certification and licensing requirements; and

• Develop medical education requirements and programs, professional development programs, annual certification examinations for new graduates and re-certification programs for experienced professionals, and initiate a professional and educational fees system.

1.2.3 Stakeholders

Table 2 identifies the stakeholder organizations and groups having an interest in, or being affected by the CAPA Contribution Program. This list served the evaluation to determine: candidates for interviews; potential sources for data; relevant authorities for advice; and relevant membership for forming the Evaluation Advisory Group.

Table 2. List of CAPA Contribution Program Key Stakeholders. This list shows the key stakeholders of the CAPA Contribution Program.

1.3 Evaluation Scope

The scope of this evaluation is limited to assessing the relevance and performance of the CAPA Contribution Program and to addressing the extent to which the Contribution Program’s objectives (listed in Section 1.2.2) have been achieved during the period of observation (FY 2009/10 to FY 2013/14). While the evaluation also provides observations and suggestions regarding the PA profession in the CAF, the ADM(RS) recommendation is specific to the CAPA Contribution Program.

1.3.1 Coverage and Responsibilities

The CAPA Contribution Program is situated within Activity 4.0 (Defence Capability Element Production) of the Program Alignment Architecture, Sub-activity 4.1 (Military Personnel and Organization Lifecycle), and Sub-sub-activity 4.1.8 (Military Personnel – Healthcare). The sponsor of the CAPA Contribution Program is the CAF Surgeon General. The delegated Program Manager is the CFHSG Chief of Staff who, through the Surgeon General, is accountable to the CMP and the Chief of the Defence Staff.

1.3.2 Resources

During the evaluation period, the CAPA employed between 3.5 and 5 full-time equivalents, and was governed by a Board of Directors comprised of 14 volunteers.

Table 3 displays the Vote 10 funding provided to the CAPA through the Contribution Program from FY 2009/10 to FY 2013/14.

Table 3. DND/CAF Expenditures.

Table 3. DND/CAF Expenditures. This table shows the amounts released from the CAPA Contribution Program to the CAPA from FY 2009/10 to FY 2013/14.

The two-year extension to the Contribution Program was approved in 2014 to make available the remaining $1,511,778 of the initial Contribution Program funding that had not been spent.

1.3.3 Evaluation Issues and Questions

In accordance with the TBS Directive on the Evaluation Function (2009),⁷ the evaluation addressed the five core issues related to relevance and performance. The methodology used to gather evidence in support of the evaluation questions can be found at Annex B. A logic model providing a theory of change for this contribution program is included at Annex C. An evaluation matrix listing each of the evaluation questions, with associated indicators and data sources, is provided at Annex D.

2.0 Findings

Sections 2.1 to 2.3 discuss the relevance of the CAPA Contribution Program; sections 2.4 and 2.5 will cover the performance aspects. Other issues not related to the relevance and performance aspects appear at section 2.6.

Relevance

In assessing relevance, the evaluation examined the extent to which the Contribution Program: addressed a demonstrable and continued need; was aligned with federal roles and responsibilities; and was aligned with Government of Canada (GC) priorities and DND/CAF strategic outcomes.

2.1 Relevance – Continued Need

To determine whether the CAPA Contribution Program addressed and continues to address a demonstrable need, the following key indicators were used:

• evidence of a current and future need for the CAPA Contribution Program;
• evidence that the CAPA Contribution Program is reactive to the needs of the DND/CAF and Canadians; and
• extent to which the CAPA Contribution Program complements the services provided by other departments, agencies, and/or organizations (internal/external).

The observations and findings are based on evidence originating from literature reviews, document reviews, and on-line resources, in addition to information extracted from interviews with key informant stakeholders.

Key Finding 1 : The Contribution Program permitted the CAPA to address a specific, demonstrable and continued need which contributed to the health care of CAF members.

The CAPA directly supports the CAF health care system by providing the necessary certification for military PAs. The CAPA also provides the necessary National Competency Profile standards for the accreditation of the PA curriculum used at the CFHSTC, which authorizes the training centre to deliver the PA education program.

Should the CAPA cease to exist, the supervision and monitoring of a unique and single competency profile necessary for accreditation of PA education programs in Canada would be difficult to achieve through alternative means, as would the required certification process for PAs in Canada, both civilian and military. Individual memoranda of understanding (MOU) might be more difficult to establish to allow CAF PAs access to civilian health care facilities in order to acquire, practise and maintain their clinical competencies, which could lead to gaps in the provision of health care to CAF personnel and impede CAF operations.

2.2 Relevance – Alignment with Federal Roles and Responsibilities

This section examines the extent to which the CAPA Contribution Program aligns with federal roles and responsibilities.

The following indicator was used in the assessment of the alignment of the CAPA Contribution Program with federal roles and responsibilities:

• evidence of alignment between federal roles and responsibilities (including legislative and policy obligations) and the delivery of the CAPA Contribution Program.

Observations and findings are based on evidence originating from official document reviews, in addition to on-line resources and to a validation of evidence through interviews with key informant stakeholders.

Key Finding 2: By contributing to the improvement of the CAF health care system and the Canadian health care system, the CAPA Contribution Program aligns with federal roles and responsibilities.

In Canada, responsibilities for “health” are shared between the federal government and the provinces and territories. Health Canada is accountable for the public health of Canadians in terms of concepts, policies and regulations, while the provinces and territories are responsible for the direct provision of health care to their citizens.⁸ As indicated on the Health Canada website: “The federal government, the ten provinces, and the three territories have key roles to play in the health care system in Canada. Health Canada’s role is to help Canadians maintain and improve their health.”⁹

2.3 Relevance – Alignment with Government Priorities and DND/CAF Strategic Outcomes

Section 2.3 examines the extent to which the CAPA Contribution Program’s objectives are consistent with federal government priorities and DND/CAF strategic outcomes.

The following indicators were used to make this determination:

• evidence of alignment between the CAPA Contribution Program’s objectives and current federal government priorities; and
• evidence of alignment between the CAPA Contribution Program’s objectives and DND/CAF strategic outcomes.

The observations and findings made in this section are based on evidence originating from official document reviews, in addition to a validation of evidence conducted through interviews with key informant stakeholders.

Key Finding 3: The CAPA Contribution Program was well aligned with government priorities.

The GC whole-of-government framework includes “Healthy Canadians” as one of the 16 outcome areas where the GC is aiming to create a responsible and accessible health system for Canadians. Support for this GC outcome area is reflected in the departmental Report of Plans and Priorities for Health Canada, DND, and Aboriginal Affairs and Northern Development Canada.¹⁰ It reinforces the concept that the health of Canadians represents a major priority for the federal government and many departments.

In addition, Defence Priorities and Elements 2014-18,¹¹ Priority 4 – Strengthening the Defence Team, includes the commitment to “Provide enhanced support to the ill and the injured and to the families of CAF members.” The Canada First Defence Strategy¹² underlines that “People are Defence’s most important resource. Both the Department and the Forces rely heavily on the work and expertise of dedicated personnel to ensure the operational effectiveness of the military.”

As such, a military health care program is provided for all members of the CAF. The certification of the military PA through the PACCC, a subsidiary of the CAPA, and the accreditation of the PA education program based on the competency profile established by the CAPA are important elements of this program. As an organization, the CAPA offers value to the PA profession and meets the needs of the CAF for military PAs, indirectly supports DND/CAF health care priorities, and indirectly contributes to DND/CAF strategic outcomes. However, one of the CAPA objectives is to become a nation-wide health care professional association through, inter alia, the accreditation of civilian university education programs, the certification of civilian PAs, and increased membership. However, this CAPA objective is outside of the DND/CAF mandate. The subsidization of a national organization supporting a new health care profession should be supported by, or shared with, other national health-related sponsors, such as Health Canada or Aboriginal Affairs and Northern Development Canada and facilitated through the Canadian Intergovernmental Conference Secretariat.

Performance

2.4 Performance – Achievement of Expected Outcomes (Effectiveness)

According to the TBS Policy on Evaluation, performance is defined as “the extent to which effectiveness, efficiency and economy are achieved by a program.”¹³ This section will review the effectiveness of the CAPA Contribution Program, while section 2.5 will cover the efficiency and economy of the Program.

The effectiveness of the CAPA Contribution Program was assessed by measuring the success of PA professional certification and PA program accreditation, and the extent to which the CAPA is financially viable and self-sufficient. These are identified on the logic model by immediate, intermediate, and ultimate program outcomes. The evaluation used two distinct approaches: the first one focused on the military PA profession in the CAF (2.4.1); the second reviewed the general PA profession in Canada (2.4.2).

The observations and findings for the Program’s effectiveness are based on evidence collected through: document and financial data reviews; questionnaires administered to PAs by the CAPA; and key informant interviews.

2.4.1 Outcomes Impacting the Military PA Profession in the CAF

The outcomes considered in this evaluation impacting the military PA profession in the CAF are as follows:

• Immediate outcomes: CAF PAs have the necessary professional skills, and CAF PAs have increased access to civilian hospitals to complete residency requirements;
• Intermediate outcomes: CAF members have confidence in military PA practitioners; and

• Ultimate outcomes: Ensure sufficient capacities of certified CAF PAs are available to meet operational and in-garrison health care needs.

2.4.1.1 Immediate outcome: CAF PAs have the necessary professional skills

The following indicators were used to assess the achievement of this outcome:

• evidence of improvement in the number of CAF PAs certified during the period FY 2009/10 to FY 2013/14; and
• evidence that certified CAF PAs can meet the required competencies.

The majority of CAF PAs have the necessary professional skills to assume their clinical duties. PA certification is awarded by the PACCC,¹⁴ a subsidiary organization of the CAPA, which administers and maintains the PA certification process. Certifications ensure that PAs meet the standards set out in the National Competency Profile for the PA profession. To obtain certification, military PAs must successfully write a formal examination that is administered once a year. PAs who are granted certification can hold the title of “Canadian Certified Physician Assistant.”

Table 4 indicates the number of CAF PAs and CCPAs from FY 2009/10 to FY 2013/14.

Table 4. Certified military PAs. This table indicates the number of Certified CAF PAs as a percentage of total CAF PAs for the period FY 2009/10 to FY 2013/14.

According to Table 4, 73 percent of existing military PAs achieved certification as of FY 2013/14, which is an increase of 250 percent compared to FY 2009/10. As the certification level is an indicator of professional skills, the evaluation assessed that the majority of military PAs have the necessary skills to do clinical work.

2.4.1.2 Immediate outcome: CAF PAs have increased access to civilian hospitals to complete residency requirements

The following indicators were used to assess the achievement of this outcome:

• evidence of increased access demonstrated by the number of CAF PAs having fulfilled their residency requirements for the period FY 2009/10 to FY 2013/14; and
• number of CAF PAs who did not complete residency requirements during the period FY 2009/10 to FY 2013/14 due to lack of MOUs/contracts with hospitals.

All CAF PA students must complete clinical rotation requirements in civilian health care centres, which is a necessary requirement to fulfill the PA education program. The CFHSG works closely with the CFHSTC to identify medical clinics and hospitals where formal arrangements are made. These agreements allow the military PA students to complete the one-year clinical exposure required by the education program. It must be noted that the CAPA has no role in this activity. Finding and organizing this employment requirement remains the responsibility of each PA education centre. The certification of military PAs through the PACCC, however, facilitates military PA employment in the civilian health care centres.

Since the creation of the military PA education program, all CAF PA students have completed their clinical rotation requirements.

2.4.1.3 Intermediate outcome: CAF members have confidence in military PA practitioners

The following indicator was used to assess the achievement of this outcome:

• The level of CAF members’ satisfaction with military PA practitioners (for the period FY 2009/10 to FY 2013/14).

Key Finding 4: Key stakeholders confirmed the emergent confidence of CAF Health Services customers in military PAs. However, the survey administered by the CFHSG did not provide a specific measurement for the PA profession.

The initiating documentation for the CAPA Contribution Program recommended that CFHSG administer an annual survey to CAF members to specifically measure client satisfaction with health services support provided by military PAs. Unfortunately, the survey was administered using a global and consolidated approach instead of isolating the health services support received from specific professionals within the CFHSG.

Interviews with key stakeholders indicated a high level of CAF members’ confidence in and satisfaction with the health care received from the CAF PA practitioners. Nevertheless, the CAF Health Services survey should be adapted to measure satisfaction of services received from specific clinical military health professions, such as the PA profession.

2.4.1.4 Ultimate outcome: Ensure a sufficient capacity of certified PA is available to meet operational and in-garrison health care needs

To assess the achievement of this outcome, relevant indicators focus on appointments, attraction, certification, and career development concerns of military PAs. The following indicators were used:

• evolution of the number of PA Military Occupational Structure Identification requirements and the number of PA positions filled (for the period FY 2009/10 to FY 2013/14);
• evidence of PA shortage in CAF deployed operations such as Afghanistan, Haiti, Alert and Vancouver (for the period FY 2009/10 to FY 2013/14);
• evolution of the military PA retention rate (for the period FY 2009/10 to FY 2013/14); and
• evidence of CAF ability to attract new military PAs (for the period FY 2009/10 to FY 2013/14).

Key Finding 5: With the sound management of scarce CAF CCPA resources, CFHSG has met all operational health care requirements without negatively impacting in-garrison health care needs. However, the CAF PA career path, as it is now, remains a concern and could affect operations in the future if not addressed.

Appointments of Military PAs. Interviews conducted with CFHSG staff involved in manning, tasking, and appointments of military CCPAs indicated that it has always been possible to fill operational CCPA positions. Even if some PA positions had to be kept vacant (e.g. 38 out of 198 positions (in 2013) and 53 out of the existing 198 position (in 2014)), the established priority system was not detrimental to any clinical military requirements.

Attraction of PAs. At present, CFHSG does not have many attractive elements to offer for the recruiting of new PAs. The actual career process is to join the CAF as a medical technician, spend up to eight years in this occupational specialty, then indicate interest in becoming a PA, be designated by a selection board, undergo the two-year education program in Borden, and be subject to a four-year obligatory service. However, CFHSG is conducting a study, with results expected to be delivered in 2016, to assess the proposal to commission military CCPAs. This could facilitate the attraction of new military PAs through the Direct Entry Officer Program.

Certification of Military PAs. In 2014, CFHSG determined that certification through the PACCC was compulsory for all military PAs in order to be selected to fill clinical appointments and to be promoted to the rank of Warrant Officer. This approach has contributed to the increase in the number of CCPAs shown in Table 4 (sub-section 2.4.1.1).

Development of CCPAs. As of 2014, no CAF PAs had less than 10 years of service (YOS). This situation is largely due to the present career progression process explained earlier. More than 63 percent of PAs have completed 20 YOS and are eligible for retirement programs, which places the viability of the CAF PA trade in jeopardy. The number of YOS has two main clusters: 11 to 20 YOS and 21 to 30 YOS with a peak of PAs in the 24 YOS area. The majority have between 21 and 30 YOS. The breakdown of the 164 serving military CCPAs as of October 31, 2014, showing their number of years of military service (as YOS) is presented in Table 5.

Table 5. YOS – Military CCPAs. This table presents the distribution of military CCPAs according to their years of military service, as of October 2014.

Attrition of Military PAs. The attrition rate of PAs can reach as high as 25 percent in some years. This seems to be a result of the attraction of available civilian PA jobs as the PA profession is expanding in the civilian health care system.

In conclusion, CFHSG must address the attraction, appointment, certification, retention and development of military PAs to ensure CAF clinical requirements continue to be met.

2.4.2 Outcomes Impacting on the General PA Profession in Canada

The outcomes considered in this evaluation, which impact the general PA profession, are as follows:

• Immediate outcomes: Academic programs and CFHSTC accreditations are renewed; number of university-based accredited programs for PAs has increased; and the CAPA is incorporated; and
• Intermediate outcomes: Single national PA certification standards exist.

The evidence collected to evaluate the effectiveness of these outcomes is mainly supported by: interviews of key informants; document reviews; and the use of quantitative data compiled by CFHSG and the CAPA.

2.4.2.1 Immediate outcome: Academic programs and CFHSTC accreditations are renewed

The following indicators were used to assess the achievement of this outcome:

• for the period FY 2009/10 to FY 2013/14, evidence that the accreditation of the CFHSTC program has been renewed; and
• for the period FY 2009/10 to FY 2013/14, evidence that the accreditation of university-based programs has been renewed.

During the period FY 2009/10 to FY 2013/14, all existing PA education programs accreditations were granted or renewed. The accreditation is primarily conducted directly between PA education institutions and the CMA, as the CMA is the sole approving authority in Canada for medical-related education programs.¹⁵ The role of the CAPA in this process is to monitor, review, and maintain current the National Competency Profile defined for the PA profession. CMA approves the National Competency Profile and uses it as the standard for the accreditation of university-based programs.

For the period covered, the CFHSTC military PA education program was renewed in 2010 for a period of six years. CFHSG is in the process of supporting the CMA accreditation renewal for CFHSTC in 2016. The three other university-based PA education programs were granted accreditation in 2010 and 2011, all of them for a period of six years. They will be subject to the renewal of their accreditation by the CMA in 2016 and 2017. Table 6 illustrates the status of accreditation for PA education programs in Canada.

Table 6. Status of Accreditation in University-based PA Education Programs. This table presents the status of accreditation of the academic institutions offering the PA education program in Canada.

2.4.2.2 Immediate outcome: Number of university-based accredited programs for PAs has increased

The following indicators were used to assess the achievement of this outcome:

• changes in the number of university-based accredited programs for the period
• FY 2009/10 to FY 2013/14 due to the CAPA expansion program; and
• evidence of effective marketing activities performed to gain professional endorsement on the accreditation of programs and development of the PA profession.

During the period FY 2009/10 to FY 2013/14, three new university-based PA education programs were implemented in two provinces. When the CAPA Contribution Program was initiated in 2009, only the CAF PA education program, created in 2002, was in place. Since then, the University of Manitoba, McMaster University, and the Consortium of Physician Assistant Education (University of Toronto, Northern Ontario School of Medicine, and The Michener Institute for Applied Health Sciences) have initiated PA education programs.

Based on interviews with key stakeholders, the implementation of relevant PA education programs by universities is part of the implementation of the PA profession in Canada. Once provinces or territories have put the legislation in place to recognize the profession, a university-based program can be established. Some universities interested in establishing the PA program are waiting for their provincial and territorial authorities to adopt the legislation. As a result, university-based PA programs cannot be implemented until provinces and territories adopt the legislation necessary to implement the PA model. Therefore, the CAPA continues to have its main efforts centered on advocacy and lobbying of provinces and territories to facilitate the adoption of this legislation. The CAPA, in its strategic plan for future activities, is preparing parallel marketing activities with universities in order to rapidly support the PA profession with academic programs.

2.4.2.3 Immediate outcome: The CAPA is incorporated

The following indicator was used to assess the achievement of this outcome:

• evidence of actions leading to potential incorporation.

The CAPA was incorporated on November 27, 2009. As soon as the Contribution Program commenced in 2009, one of the first activities completed by the CAPA was to achieve incorporation.

2.4.2.4 Intermediate outcome: Single national PA certification standard exists

The following indicator was used to assess the achievement of this outcome:

• evidence of the existence of a single national PA certification standard.

Since 2011, there exists only one national PA certification standard represented by the Scope of Practice and the National Competency Profile for PAs. Certification is granted by the PACCC, which develops certification exams based on the Scope of Practice and the National Competency Profile.¹⁶ Created in 2009 by the CAPA and approved in 2011 by the CMA, this standard constitutes the foundation of the clinical aspects of the PA profession. It also serves as the standard to determine accreditation of actual and incoming PA education programs in Canada.

2.5 Performance – Demonstration of Efficiency and Economy

This section will examine the performance of the CAPA Contribution Program related to efficiency ¹⁷ and economy.¹⁸ More particularly, it will address the extent to which the CAPA Contribution Program has used the resources in relation to the production of outputs and to the progress made towards the given outcomes, and the achievement of the objectives of the Contribution Program. In addition, outcomes identified in a program are usually measured in terms of effectiveness. In the present evaluation, since many outcomes were very closely related to the measurement of efficiency and economy, they are discussed in the present section.

The approach utilized to demonstrate efficiency and economy will cover the following outcomes and evaluation questions:

• Immediate outcomes: Other revenue-generating activities are undertaken, and membership revenues have increased;
• Intermediate outcome: The CAPA is financially viable and self-sufficient;
• Unintended outcome: Limitation of self-sufficiency options due to stacking of grants and contributions; and
• The specific efficiency and economy questions addressed are as follows:
  • What is the progress made by the Contribution Program towards the expected outcomes and the resources expended?
  • What is the trend of the output/input ratio towards achieving the aim of the Contribution Program?
  • How were the objectives established at the creation of the program met?

The evidence collected to assess the achievement of outcomes is supported by: document reviews; confirmation by interviews with key informant stakeholders; and the use of quantitative data compiled by CFHSG and by the CAPA.

2.5.1 Immediate outcome: Other revenue-generating activities are undertaken

The following indicators were used to assess the achievement of this outcome:

• variations in other revenues (for the period FY 2009/10 to FY 2013/14);
• evidence of specific activities aimed at generating permanent sources of funds (for the period FY 2009/10 to FY 2013/14);
• evidence of growth in the number of sponsorships (for the period FY 2009/10 to FY 2013/14); and
• evidence of growth in the revenues generated by sponsorship (for the period FY 2009/10 to FY 2013/14).

Key Finding 6: The CAPA made progress in developing some revenue-generating activities, but permanent sources of funding are presently not able to ensure the financial self-sufficiency of the organization.

The CAPA’s not-for-profit status allows the organization to generate revenue from various sources and activities. The only limitation to funding activities is where the CAPA cannot request or accept any funds for activities already covered by the Contribution Program and stated in the CFA. For all other activities not funded by the CFA, the CAPA can accept funds and declare the funds to the Minister of National Defence, who, in turn, will decide any impacts concerning funding received from the Contribution Program.

Typical revenue-generating activities include collecting membership dues, PA conference fees, examination fees for certification, advertising, sponsorship, and revenues from an exhibit during the conference. However, revenue can also be generated through commissions on sales from the CAPA boutique, advertising on the CAPA website, partnerships, and donations from supporters.

Table 7 shows the revenue reported for the period FY 2009/10 to 2013/14. As FY 2009/10 was the first year of operations for the CAPA, there were no other significant sources of revenue.

Table 7. Sources of Revenue. This table presents the revenues generated by CAPA activities for the period FY 2009/10 to FY 2013/14.

According to Table 7, ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

2.5.2 Immediate outcome: Membership revenues have increased

The following indicators were used to assess the achievement of this outcome:

• evidence of growth in the number of CAPA members (for the period FY 2009/10 to FY 2013/14); and
• evidence of growth in revenues generated by the different categories of membership (for the period FY 2009/10 to FY 2013/14).

The membership of the CAPA has increased. As shown in Table 8, the military and civilian PA membership of the CAPA progressed by 134 percent, increasing from 217 members in FY 2009/10 to 508 members in FY 2013/14. The growth of the number of members varied each year, maintaining an average progression of 26 percent per year during the last five years.

Table 8. CAPA Membership. This table presents the status of CAPA membership from FY 2009/10 to FY 2013/14.

In terms of revenues, income generated by membership dues has also progressed positively, showing improvements for each year between FY 2010/11 to FY 2013/14, ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ Table 9 presents the situation in detail. FY 2009/10 does not appear in the table as there were no membership dues collected in that fiscal year.

Table 9. CAPA Membership Dues. This table presents the income generated by the membership dues between FY 2010/11 and FY 2013/14.

2.5.3 Intermediate outcome: The CAPA is financially viable and self-sufficient

The following indicators were used to assess the achievement of this outcome:

• the degree of financial autonomy and self-sufficiency of the CAPA at the end of FY 2013/14;
• at the end of the extension period (FY 2015/16), the probability that the CAPA will become financially self-sufficient;
• improvements shown in the sources of funds external to DND since the inception of the Contribution Program;
• improvements shown in the number of sponsors; and
• improvements/increases shown in the amount of funding from other sources.

Key Finding 7: ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓¹⁹

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

2.5.4 Unintended outcomes: Limitation of self-sufficiency options due to stacking of grants and contributions

The following indicator was used to assess this outcome:

• evidence of unintended elements having impacted on the results of the CAPA Contribution Program.

Key Finding 8: The CAPA could have generated other revenues without jeopardizing funding from the CAPA Contribution Program.

There exists a “stacking of grant and contribution” clause in the CFA. The CAPA staff understood that the disbursement of funds from the Contribution Program would have ceased if the CAPA had acquired grants or contributions from other sponsors or monetary contributions from partnerships. Based on an assessment made by the Officer in Charge of the Grants and Contributions at Assistant Deputy Minister (Finance / Chief Financial Officer), the clause in the CFA covering the “stacking of assistance” might have been misinterpreted. It should have been read in conjunction with one of the aims of the Contribution Program, which was for the CAPA to become self-sufficient and to ensure survivability within the period of time allocated by the Contribution Program.

Therefore, any additional funding received from activities not covered by the CFA or from external entities would have initiated a process of disclosure to the Minister of National Defence. This process could have resulted in a decision in funding adjustment to reduce the actual funds received by the CAPA from the Contribution Program. The official documentation creating this Contribution Program aimed to make the CAPA self-sufficient within the allocated contribution period. If additional funding was to be provided, the stacking clause should not be seen as a barrier to obtain other revenues and should be interpreted more widely.

2.5.5 Progress towards Expected Outcomes and Resources Expended

To assess the progress towards outcomes and the resources expended, the following indicators were used:

• funds originating from other sources compared to the total expended for the period FY 2009/10 to FY 2013/14 to determine self-sufficiency and viability; and
• for the period FY 2009/10 to FY 2013/14, evidence of positive impacts and of value for money in the achievement of the immediate outcomes.

Key Finding 9: Progress was made towards the full achievement of Contribution Program outcomes.

The CAPA staff and the Board of Directors members interviewed indicated that during the first five years, the Contribution Program focused more on developing the Association and increasing the opportunities for advocacy, lobbying activities, and contacts with a view to having the provinces and territories adopt the PA model. Contribution Program funding covered all approved activities, so the necessity of finding external sources of funding was not assessed as a priority.

Table 10. Sources of Funds. This table outlines the sources of funds generated by the CAPA during the period FY 2009/10 to FY 2013/14.

Table 10 shows that DND ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

As indicated earlier, most immediate outcomes not related to the generation of funds were achieved. The evaluation assessed that there was value for money for those achievements. ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

2.5.6 Trend of the Output/Input Ratio towards Achieving the Aim of this Contribution Program

To assess the trend of the output/input ratio towards achieving the aim of the Contribution Program, the following indicators were used:

• perceptions of performance in consideration of resources/expenditures; and
• for the period FY 2009/10 to FY 2013/14, evidence of evolution in the number of accreditations by province, number of certification exams, number of Continuing Professional Development (CPD) training activities, number of PA programs in universities, number of members and membership dues, number of CAPA sponsors, number of participants for CAPA events (including the annual conference), level of public awareness about the PA profession, and number of certified military and civilian PAs.

Due to the lack of a baseline or comparisons with other existing programs, it is difficult to assess whether the levels of resources utilized and of expenditures made were sufficient to achieve the aim of this program. Nevertheless, some data support relevant observations. For instance, one element was isolated to assess its specific efficiency. Certification fees versus certification costs was the element selected.

Key Finding 10: The cost generated by the PA certification process is greater than certification process revenues.

Table 11 presents the percentage of the costs covered by the fees charged to operate the certification process put in place by the CAPA. ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

Table 11. Certification Fees versus Certification Expenses. This table shows the amount of expenses covered by the fees charged as a percentage of the cost of the certification process.

Key Finding 11: The efficiency of delivering expected CAPA Contribution Program outputs was inconsistent.

It was expected that the outputs of the CAPA Contribution Program would show progressive improvement and positive results. Table 12 indicates the results applicable to the main outputs.

Table 12. Evaluation of Outputs and Deliverables. This table shows result-based information pertaining to the main outputs of the Contribution Program for the period FY 2009/10 to 2013/14.

The limited expansion of the PA model in provinces and territories was a significant factor in the development of the PA profession that impacted on: membership levels; PA position development; availability of universities to offer new PA education programs; accreditation of new PA education programs; and certification of PAs. The complexity of dealing with 10 provinces and three territories might have been underestimated when the CAPA Contribution Program was created.

2.5.7 Meeting the Objectives Established at the Creation of the Program

Key Finding 12: The Contribution Program met many of the objectives required in the initiating documents, but did not succeed in achieving financial viability and self-sufficiency.

According to the initiating documents for the CAPA Contribution Program, an evaluation must be conducted to report on how the Program has met the Program objectives. To determine the degree of achievement of the objectives stated in the initiating documents, the evaluation used the conclusions reached in sections 2.4 and 2.5 of this evaluation report. A compilation of the CAPA Contribution Program objectives that were either achieved or not achieved is provided in Table 1 in the Executive Summary.

2.6 Other Issues

CFHSG, as the sponsoring organization of the Contribution Program, raised the following parallel issue during the conduct of the evaluation.

Key Finding 13: Limitations in policy restrict the ability of DND to reimburse CAF members for professional memberships in the CAPA. This acts as a financial disincentive for personnel to maintain membership.

CAF PAs are an important part of the CAPA organization, comprising a large proportion of current membership and thus contributing to the financial viability of the CAPA Program. One barrier to membership is the financial impact on the individual CAF members to maintain CAPA membership. While the Surgeon General has directed that a CAF PA shall be certified and therefore must be a member of the CAPA, CAPA membership fees are currently not reimbursed to the military PAs, while most other CAF Health Care Professionals’ memberships are reimbursed²⁰ (e.g. nurses, physicians). The current policy (source: CANFORGEN 125/13) limits reimbursement to CAF members of memberships that are legally required for the performance of their duties. In the case of most health care professions, the legal requirement is found in either federal law or provincial regulations. However, as the PA profession is not yet recognized and regulated in all provinces, this basic principle of the CAF membership reimbursement policy is not met. In contrast, the standard for reimbursement of public service employees does not require a legal authority but only membership to be considered by the employer as a requirement for the continuation of the performance of the duties of the employee’s position. The authority to make the determination for public service employees is delegated to Level 2 advisors (generally Brigadier-General rank or EX-02). CANFORGEN 125/13 is an interim policy until a new Compensation and Benefits Instruction is developed and issued. In relation to this approach, CMP should investigate mechanisms that would allow for the reimbursement of CAPA membership fees to military PAs as required for the performance of their duties.

3.0 Conclusion and Recommendation

In assessing the relevance and performance of the CAPA Contribution Program, the evaluation noted the importance of the CAPA as an independent organization that establishes and maintains the regulation and the curriculum of the PA profession both within the CAF and the provinces. However, evidence demonstrated that the CAPA failed to meet its main goal of financial self-sufficiency over the duration of the program.

The DND Contribution Program funding also supported the CAPA’s objective to become a nation-wide health care professional association and to promote the PA profession across Canada. Funding of this CAPA objective, however, is outside of the DND/CAF mandate and does not align with DND/CAF strategic outcomes. The subsidization of a professional association, which supports a national health care profession, should not be the sole responsibility of DND. Funding, if necessary, should be assumed by or shared with other national health-related sponsors.

Continued support to the CAPA, through a DND-sponsored Contribution Program, is not recommended. The termination of the Contribution Program at the end of the current terms and conditions as of March 31, 2016, necessitates that the CAPA become financially viable. If the CAPA does not become financially self-sufficient, it could lead to the Association’s eventual dissolution, which could impact the CAF PA profession. As such, the following recommendation is made:

Annex A—Management Action Plan

Management Action

The Surgeon General supports the ADM(RS) recommendation to not renew the DND-sponsored CAPA Contribution Program.

The CAPA was founded in 1999 by CFHSG with a view to create, legitimize, and advocate for PA as a recognized health occupation amongst other health care professions, as well as to heighten their already outstanding reputation within and outside the DND/CAF. The CAPA has now grown into a credible pan-Canadian association with members from various provinces, civilian stakeholders, government agencies, and other federal departments. The CAPA has successfully established professional development programs, certification/re-certification programs, medical education programs, and other initiatives.

During recent discussions with the CAPA, CFHSG was advised that the CAPA will address its long-term financial viability with a series of measures, including an increase in CAPA membership fees. The CAPA and CFHSG intend to continue their already well-established relationship via mutual presence and consultation on various committees and sponsorship events.

If the CAPA is not able to become financially self-sufficient and ceases to exist as an association, CFHSG has considered possible implications and mitigation strategies for the CAF PA profession.

The accreditation of the CAF PA education program should not be affected. The PA course curriculum taught at the CFHSTC was originally based on the University of Nebraska PA curriculum. If the CAPA ceased to exist as an organization, CFHSG would work with the provinces to align the competency profile of the PA profession.

• The training success of CAF PAs is subsequently certified by the CAPA via a certification process (PACCC). Currently only New Brunswick and Manitoba recognize the CAPA certification, while the other two provinces that have a PA program (Ontario and soon Alberta) do not presently require PAs to be certified. The CAPA is working towards having their certification recognized by all provinces, thus allowing the PA certification to be “portable” across Canada, which would benefit the CAF PA profession with easier access to on-going CAF PA clinical training opportunities. If the CAPA ceased to exist as an organization, CFHSG would identify a collaborative approach to work with the provinces that have established PA programs.
• Certification of CAF PAs by a recognized health association remains a main objective of the CAF Surgeon General. It is a sine qua non condition for any health system to have all of its health care professionals licensed/certified by a recognized professional body, to ensure patient safety, operational readiness, quality in health care delivery and mission success. Annual CAPA membership of each CAF PA is required to maintain professional certification status. Through CAPA membership, CAF PAs will contribute significantly to the CAPA’s financial stability. CFHSG will continue to work with the departmental office of primary interest (OPI) to obtain an appropriate financial authority for the reimbursement of CAPA membership for CAF PAs.
• The Medical Technician-PA occupation restructure process has resulted in the identification and reassignment of 109 clinical PA positions to the new PA officer occupation and 94 Medical Technician-PA leadership positions to the Medical Technician occupation. With a reduction in the total number of positions required within the PA officer occupation, the number of certified personnel presently available is considered sufficient to maintain CAF operational capabilities.

OPI: CMP

OCI: CFHSG

Annex B—Evaluation Methodology, Validation and Limitations

1.0 Methodology

1.1 Overview of Data Collection Methods

The findings and associated recommendations of this evaluation are supported by multiple lines of evidence collected through qualitative and quantitative research methods. Information and data collected from relevant sources were analyzed to inform conclusions on the relevance and performance (effectiveness and efficiency/economy) of the program.

Key activities of the evaluation were grouped together to determine the expected outcomes of the program as defined in the logic model. Performance indicators (PI) were identified for each outcome to assess the performance of the CAPA Contribution Program. Outcomes and their associated PIs are listed in the Evaluation Matrix in Annex D.

The data for each PI was collected using the following research methods:

• Literature review
• Document review
• Key informant interviews
• Site visits

• Administrative and financial data reviews

1.2 Details on Data Collection Methods

1.2.1 Literature Review

A literature review was undertaken to identify the existence of similar programs in other government departments or in other countries, to assess which type of national associations exist to support them, and to understand the evolution of the profession.

1.2.2 Document Review

A document review was completed to provide a thorough picture of the purpose, scope and mandate of the CAPA Contribution Program. Additionally, the document review enabled visibility into the larger organizational and operational context in which the program operates. Core program documents like the initiating documents were used to establish parameters regarding program delivery and as a source of data to support the performance evaluation of the economy and efficiency of the program.

The documents reviewed during the conduct phase of the evaluation included the following:

• official initiating documents for this Contribution Program;
• policy, legislation, and related accountability documents, such as: TBS Policy and Directive on Evaluation, Departmental Administration and Operation Directives, Departmental Performance Reports, Reports on Plans and Priorities applicable to the period covered by the evaluation (FY 2009/10 to FY 2013/14), and the DND Program Alignment Architecture;
• strategic documents and plans, such as: CAPA Annual Activity Reports on the contribution, CAPA strategic plans, results of the yearly (2010 to 2014) PA members’ marketing survey, and results of the 2014 public opinion survey;
• GC direction and related documents, such as: Canada First Defence Strategy, Speech from the Throne, Priorities of Defence, and GC whole-of-government framework;
• program specific documents, such as: reporting documents, internal reports, and extractions from the CAPA website;
• other government/military documents, such as: reports, service papers, and related documentation that enable or prevent comparisons between the CAPA Contribution Program and similar programs in allied militaries; and

• other audits and evaluations conducted in the past by the OAG or by ADM(RS) relevant to the present evaluation.

1.2.3 Key Informant Interviews

Most of the key stakeholders who contributed to interviews were located within the vicinity of Ottawa-Gatineau or were contacted by phone. Interviews were conducted with key senior stakeholders from the CAPA (the Board of Directors and the management staff); from CFHSG (senior officials); from CMP (senior official); and from the CMA (accreditation and advocacy officials).

The interviewees were given an interview guide in advance. Interview guides were tailored to each individual and interview questions were aligned with the PIs indicated in the evaluation matrix. The evaluator transcribed the notes taken during the interviews in an evidence matrix with a view to regroup all of them according to related PIs and to make a conclusive analysis.

1.2.4 Site Visits

The CAPA Headquarters, the CMA Headquarters, and the CFHSG Headquarters were visited.

1.2.5 Administrative and Financial Data Reviews

Financial and administrative data for the CAPA Contribution Program, obtained from the CAPA itself, and those created and maintained by the CFHSG Comptroller office were analyzed with respect to the defined PI provided in the Evaluation Matrix.

2.0 Validation

Validity and credibility of data is a concern in all evaluations. To address these concerns, peer review, saturation, triangulation, and evaluation advisory group mechanisms were employed.

2.1 Peer Review

At each step of the evaluation where documents were created (i.e. work plan, logic model, evaluation matrix, evidence matrix, interview guides, portion of draft report, etc.), an internal peer review was conducted to validate the documents and to provide multiple reviews of the evaluation.

2.2 Saturation

The data collection was concluded once the evaluator assessed that the facts and evidence collected were consistent and led to the same conclusions for a given PI. The saturation point was then considered reached and the analysis could be initiated.

2.3 Triangulation

This triangulation was achieved by the diversity of methods to capture the same information. For instance, information that had been gathered in the initiating document and confirmed in financial statements was discussed in interviews. Another triangulation method was to ask the same question relating to the same PI to different stakeholders. Obtaining the same data from multiple stakeholders was considered data validation.

2.4 Evaluation Advisory Group

An Evaluation Advisory Group was established with a view to place the evaluation in a participative mode with stakeholders and to give them an opportunity to validate the tools, observations and findings as the evaluation process proceeded. Meetings were scheduled at different phases of the evaluation: initial meeting, review of the logic model and of the evaluation matrix, delivery of preliminary findings, review of the draft report, and discussions on the management action plan.

3.0 Limitations

In this section, limitations affecting the evaluation are identified and assessed. A mitigation strategy is also included to acknowledge how the evaluation attempted to overcome those limitations. The limitations and mitigation strategies are summarized in Table B-1.

Table B-1. Evaluation Limitations and Mitigation Strategies. This table identifies the main limitations of the evaluation process and the possible mitigation strategies.

Annex C—Logic Model

Figure C-1. Logic Model for the CAPA Contribution Program. This flow chart shows the relationship between the program’s main activities, outputs, and expected outcomes.

Annex D—Evaluation Matrix

Note : According to the initiating documents for this contribution program, in addition to covering all issues determined by the TBS Policy on Evaluation (2009), this evaluation must specifically address the relevance, success, and cost-effectiveness of the contribution program in support of the CAPA.

Table D-1. Evaluation Matrix—Relevance – Continued Need. This table indicates the data collection methods used to assess the evaluation issues/questions for determining the CAPA Contribution Program’s relevance pertaining to the continued need for this program.

Table D-2. Relevance – Alignment with Federal Roles and Responsibilities. This table indicates the data collection methods used to assess the evaluation issues/questions for determining the relevance (alignment with federal roles and responsibilities) of the CAPA Contribution Program.

Table D-3. Relevance – Alignment with Government Priorities. This table indicates the data collection methods used to assess the evaluation issues/questions for determining the relevance (alignment with government priorities and departmental strategic outcomes) of the CAPA Contribution Program.

Table D-4. Evaluation Matrix—Performance (Effectiveness). This table indicates the data collection methods used to assess the evaluation issues/questions for determining the CAPA Contribution Program’s performance in terms of achievement of outcomes (effectiveness).

Table D-5. Performance (Efficiency and Economy). This table indicates the data collection methods used to assess the evaluation issues/questions for determining the performance of the CAPA Contribution Program and the objectives stated in the initiating documents.

___________________________________________________________________________________________________________________________

Footnote 1 The National Defence Act (Section 4) and Queen’s Regulations and Orders (Section 34.07(4)) – Medical Services Entitlements to Medical Care provide the authority for the creation of the CAPA.

Footnote 2 Physician Assistant Certification Council of Canada.

Footnote 3 http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=13525, article 6.5.5. Last consulted on February 12, 2015.

Footnote 4 Chief Review Services was the former designation of ADM(RS). The current designation came into effect on May 13, 2015.

Footnote 5 Certification is provided through the PACCC, which is a subsidiary of the CAPA.

Footnote 6 Source: The Contribution Funding Agreement (CFA) and other initiating documents.

Footnote 7 TBS. Directive on the Evaluation Function, April 1, 2009. http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=15681&section=text. Last consulted on February 1, 2015.

Footnote 8 Health Canada’s responsibilities are not oriented towards direct health care support. An exception to this is the provision of health care services to certain population groups for which it is constitutionally responsible (First Nations and Inuit people). Health Canada’s interest in health care is also reinforced by the creation, in 1999, of an Office of Nursing Policy embedded in the Health Canada Policy Directorate.

Footnote 9 http://www.hc-sc.gc.ca/hcs-sss/index-eng.php. Last consulted on March 15, 2015.

Footnote 10 Aboriginal Affairs and Northern Development Canada was the applied title of the department when this report was written; the name has been changed to Indigenous and Northern Affairs Canada, as of December 7, 2015.

Footnote 11 http://www.forces.gc.ca/en/about-reports-pubs-report-plan-priorities/2013-other-defence-priorities-corporate-risk-profile-paa-alignment-matrix.page. (Recommend removing the period at the end of this URL address, otherwise this link will not work – IM) Last consulted on March 15, 2015.

Footnote 12 http://www.forces.gc.ca/assets/FORCES_Internet/docs/en/about/CFDS-SDCD-eng.pdf. Last consulted on April 3, 2015.

Footnote 13 http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=15024. Last consulted on April 11, 2015.

Footnote 14 https://capa-acam.ca/paccc/. Last consulted on April 11, 2015.

Footnote 15 Discussions between the CFHSG and the CMA held at the beginning of 2016 revealed that, in the future, the CMA may no longer be involved in program accreditation. Another approving authority may eventually assume that responsibility.

Footnote 16 This document is available at: https://capa-acam.ca/wp-content/uploads/2012/12/NCP_en_sept20092.pdf.

Footnote 17 The term “efficiency” used in this evaluation is defined in the TBS Policy on Evaluation as “the extent to which resources are used such that a greater level of output is produced with the same level of input or, a lower level of input is used to produce the same level of output.”

Footnote 18 The expression “economy” is defined in the TBS Policy on Evaluation as “minimizing the use of resources.” The policy adds that “. . . economy is achieved when the cost of resources used approximates the minimum amount of resources needed to achieve expected outcomes.”

Footnote 19 ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

Footnote 20 Extracts from the Public Service Alliance of Canada Collective Agreement: 1) Health Services collective agreement: Article 21 – Registration Fees, paragraph 21.01: The Employer shall reimburse an employee for the payment of membership, registration or other related fees to organizations or governing bodies when the Employer is satisfied that the payment of such fees is a requirement for the continuation of the performance of the duties of the employee’s position. 2) Operational Services Group collective agreement: Article 66 – Trade Certification Fees, paragraph 66.01: The Employer shall reimburse an employee for the payment of registration, licensing or certification fees to an organization, governing body or government agency when the payment of such fees is a requirement for the continuation of the performance of the duties of the employee’s position.

Footnote 21 In the following tables, the word “yes” indicates the method employed by the evaluation to measure the indicators. To facilitate its reading, all “yes” cases have been yellowed. For the last column, the method expected to be used is indicated in the box and shaded accordingly.