Evaluation of Canadian Forces Liaison Council and Employer Support Program

ADM(Fin)
Assistant Deputy Minister (Finance)
ADM(IM)
Assistant Deputy Minister (Information Management)
ADM(PA)
Assistant Deputy Minister (Public Affairs)
ADM(RS)
Assistant Deputy Minister (Review Services)
CAF
Canadian Armed Forces
CAF LA
Canadian Armed Forces Legal Advisor
CDS
Chief of the Defence Staff
CEO
Chief Executive Officer
CERP
Compensation for Employers of Reservists Program
CFLC
Canadian Forces Liaison Council
CMP
Chief of Military Personnel
CRES
Chief of Reserves and Employer Support
DGE
Director General Evaluation
DGMPRA
Director General Military Personnel Research and Analysis
DND
Department of National Defence
ESDC/SC
Employment and Social Development Canada/Service Canada
FY
Fiscal Year
HR
Human Resources
JOPG
Joint Operational Planning Group
JPL
Job Protection Legislation
L1
Level One
LO
Liaison Officer
MND
Minister of National Defence
O&M
Operations and Maintenance
OCI
Office of Collateral Interest
OPI
Office of Primary Interest
QUEST
Questionnaire on Unit Employer Support Targets
RAP
Reserve Assistance Program
RLO
Regional Liaison Officer
RUSP
Reserve Unit Support Program
SOP
Standard Operating Procedure
SSE
Canada’s defence policy: Strong, Secure, Engaged
VCDS
Vice Chief of the Defence Staff
WG
Working group
WGH
With Glowing Hearts

Overall Assessment

  • CFLC Program activities fulfill an ongoing need to support DND/CAF priorities related to reservists
  • The CFLC Program increases awareness of the value and skills of reservists among Canadian employers and educators, and influences job protection legislation (JPL) across Canada
  • The CFLC Program appears to be efficient, with the exception of CERP which is not yet cost-effective due to low uptake levels
  • CFLC volunteers are key enablers of program delivery, but there is a need to clarify roles and responsibilities between DND/CAF and the volunteers
  • The CFLC Program does not systematically collect performance data necessary for program management and decision making
Table 1 Details - Key Findings and Recommendations
Table 2 Details - CFLC Program Expenditures ($ thousands)
Figure 1
Figure 1. CFLC O&M expenditures ($ thousands). This figure shows the breakdown of O&M expenses by key activities covered in this evaluation from FY 2013/14 to 2018/19.

Figure 1 Details - CFLC O&M expenditures ($ thousands)

“Executreks provide a priceless lasting impression. Participants usually come in with completely different perceptions of reserves which are usually far from the reality, and then we break these perceptions and show them the real value of Reserve Force. A lot of the participants are captivated by the professionalism and the skills of reservists that they as employers can benefit from.”

Figure 2
Figure 2. Number of CFLC Events and Guests, FY 2011/12 and 2018/19.
Figure 2 Details - Number of CFLC Events and Guests, FY 2011/12 and 2018/19
Figure 3
Figure 3. Reservist Engagement with CFLC, 2017 and 2018.Footnote 25  Percentage Reservist Engagement and Awareness based on responses to QUEST survey, potentially due to CFLC initiatives/activities.

Figure 3 Details - Reservist Engagement with CFLC, 2017 and 2018

ADM(RS) Recommendation

ADM(RS) Recommendation

ADM(RS) Recommendation

Figure 4
Figure 4. Canadian Reserve Retention Survey (percentage).  This figure shows the breakdown of Employer’s perception of the reserve force in 2015.

Figure 4 Details - Reserve Retention Survey (% percentage)

“As an employer of a reservist, it opened my eyes to what they learn and why it is important.”

ADM(RS) Recommendation

ADM(RS) Recommendation

“Last Monday, we were at a military cup, and the military and CFLC representatives were there, along with about 120 business leaders. We talked to each of them individually, we asked them questions and had a range of responses. We later followed up with phone calls to those that didn’t have [military leave] programs and policies in place. This is how we raise awareness and achieve improvements.”

“Volunteers are a built-in efficiency”

Figure 5

Figure 5. Direct cost per event and per guest, Executreks and Outreach, 2018/19. This figure shows the direct costs of Executreks and Outreach activities, divided by the number of events, and by the total number of guests attending, in FY 2018/19. The scale of the bars in the chart have been adjusted to present visually the large range of quantities.Footnote 37

 

Figure 5 Details - Direct cost per event and per guest, Executreks and Outreach, 2018/19

ADM(RS) Recommendation

ADM(RS) Recommendation

Figure 6
Figure 6. Proportion of the value of CFLC’s volunteer network.

Figure 6 Details - Proportion of the value of CFLC’s volunteer network

ADM(RS) Recommendation

Current data collection opportunities: 

  • Events held and cancelled
  • Event participation 
  • Employer/educator contact information, including employment of reservists and presence of reserve-friendly policies
  • Award nominations and attendance of award events
  • Expenditures per individual event

 

Past tools:

  • Directors’ log book
  • Annual reporting
  • Post Executrek surveys
  • Statements of endorsement
  • Collated suggestions
  • Database of event participants 
  • Employer Information System software

“Data does not meet decision-making needs.”

“There is no consolidated list of employers.”

ADM(RS) Recommendation

ADM(RS) Recommendation

ADM(RS) Recommendation

ADM(RS) Recommendation

ADM(RS) Recommendation

ADM(RS) Recommendation

Table B-1 Details - Evaluation Limitations and Mitigation Strategies


Back to Table of Contents

Annex C—Logic Model

Figure C-1
Figure C-1. This flowchart shows the relationship between the program’s main activities, outputs and expected outcomes.

Figure C-1 Details - CFLC Logic Model

Figure C-1 Summary:

This flowchart, entitled Draft Logic Model for the Canadian Forces Liaison Council (CFLC) and Employer Support Program, shows the relationship between the program’s inputs, activities, outputs and expected outcomes. On the left-hand side of the graphic, there are six boxes, one on top of the other, showing six stages as follows, from top to bottom: Inputs, Activities, Outputs, Immediate Outcomes, Intermediate Outcomes, and Ultimate Outcomes. 

At the top of the chart, to the right of the Input box, is a single box containing the program’s inputs, as follows:

  • Human Resources (Civilian & Military) & Financial Resources (Salaries, Operational & Maintenance, Grants & Contributions)
  • Enabling Infrastructure & Information/Data (including documentation: SSE, MND’s Guidance Letter, CDA ID on Strengthening the Reserves, CFLC Strategic Plan, CFLC National Planning Framework, QUEST Census Results)
  • Influences/Drivers (Leadership Direction, National/Provincial Volunteer Network, Employer Engagement)

Beneath that box, the next two rows of boxes denote the activities and outputs. There are eight columns of boxes, for each of which there is an activity box and, below it, a box listing the output(s) for that activity. The following are the activities and their associated outputs.

The first activity on the left is ExecuTrek. The output for this activity is: Hosting employers and other target audiences at military exercises.

The next activity is Outreach. Outputs for this activity are: Community outreach, engagement events; Local unit outreach events; Social media presence; Presentations, articles for professional journals.

The next activity is Awards & Recognition. Outputs for this activity are: Provincial & National awards ceremonies; Letters and certificates of recognition.

The next activity is Compensation for Employers of Reservists Program (CERP). The output for this activity is: Grants to employers.

The next activity is Reserve Assistance Program (RAP). Outputs for this activity are: Direct assistance to Reservists; Conflict resolution; Communication support.

The next activity is Reserve Unit Support Program (RUSP). Outputs for this activity are: Support to Reservists and Reserve Force units; Consistent engagement with the CAF CoC.

The preceding six sets of activities and outputs are surrounded by a larger rectangle labelled “CFLC Programs. The next two sets of activities and outputs are outside of that rectangle. 

The next activity is Contribution to Policy Development. Outputs for this activity are: Priority-setting documents, briefing notes, MCs, TB submissions, lobbying, communications & consultations.

The next activity is Clerk’s Initiative on Dual Employment. Outputs for this activity are: Mentorship program; Post-Secondary Recruitment (PSR); FSWEP; GCcollab; Communications outreach.

Below outputs is a row of six boxes containing the immediate outcomes, as follows:

  • Employers, educational institutions and the public are aware and informed of the value of the reserve service and the skills of Reservists
  • Enhanced engagement and communication with reservists, public and private sectors, and diverse communities 
  • Improved relationships between Reservists and their employers
  • Reservists are aware of the value of employer support and CFLC programs and services available to them
  • CFLC influences policy development on issues related to the Reserve Force and supports other reserve-specific initiatives
  • Promotion of dual public/reserve service employment

Below immediate outcomes is a row of four boxes containing the intermediate outcomes, as follows:

  • Enhanced support from employers and educational institutions through military leave policies and student-reservist policies
  • CFLC programs meet the needs of Reservists
  • Increased passive recruiting attraction to the Reserve Force
  • Legislation and policy at F/T/P levels are harmonized and reflect the needs of Reservists

Below intermediate outcomes, at the bottom of the chart, are two boxes containing the ultimate outcomes, as follows:

  • Increased attraction and retention of Reservists
  • Enhanced availability of Reservists for military duties


Back to Table of Contents

Annex D—Evaluation Matrix

Table D-1. Evaluation Matrix. This table indicates the data collection methods used to assess the evaluation issues/questions.
Evaluation Question Indicators Sources/Methods
Relevance – Does CFLC address actual needs?
1. Is there an ongoing need for CFLC program activities?

Evidence/perceptions of the continued need for CFLC activities

Presence/absence of other programs that complement or duplicate the objectives of CFLC

Document Review

Key Informant Interviews

Effectiveness – Has CFLC achieved its intended outcomes?
2. Does CFLC contribute to enhanced engagement and communication with its target audiences (e.g., reservists, public and private sectors, diverse communities) to increase passive recruiting attraction to the reserve force?

Evidence of/views on the extent CFLC programs have contributed to enhanced engagement and communication with its target audiences

Trend analysis of number of Outreach events per year

Levels of participation (i.e., percentage of target employers/academic institutions that attended an event/activity)

Percentage of activities executed by sector (e.g., strategic partners, internal DND/CAF, private sector business & labour organizations, private sector, public services and education)

Evidence of/views on the extent that CFLC contributes to increased passive recruiting attraction to the reserve force

Recruitment rates

Retention rates

Document Review

Key Informant Interviews

Review of Performance Data

Direct Observation

3. Do CFLC programs contribute to increased awareness of the value of the reserve service and the skills of reservists among Canadian employers, educational institutions and other target groups (e.g., communities and groups currently under-represented in the Reserve Force)?

Evidence that employers and educational institutions are aware and informed of the value of the reserve service and the skills of reservists

Levels of public support – public awareness and perception

Percentage of Canadian employers that are aware of the Employer Support Program

Percentage of the Canadian employers with a positive perception of the Reserve Force

Evidence of/views on the extent CFLC contributes to increased awareness of the value of the reserve service among its target groups in support of diversity and inclusion

Percentage of recruits from designated groups and communities

Document Review

Key Informant Interviews

Review of Performance Data

Direct Observation

4. To what extent does CFLC contribute to influencing policy development on issues related to the Reserve Force and support other reserve-specific initiatives?

Evidence of/views on the extent that CFLC contributes to policy development related to the Reserve Force and reserve-specific initiatives

Evidence of/views on the extent that CFLC contributes to harmonizing JPL at the federal/provincial/territorial levels

Document Review

Key Informant Interviews

5. To what extent do CFLC programs contribute to enhancing support from employers and educational institutions through military leave policies and student-reservist policies?

Evidence of/views on the extent that CFLC programs contribute to increased number of employers who have adopted military leave policies

Number of reserve-friendly HR policies

Evidence of/views on the extent that CFLC programs contribute to increased number of educational institutions who have adopted student-reservist policies

Evidence of/views on the extent CFLC programs contribute to enhanced overall support of reservists from employers and educational institutions, and improvements in existing leave policies

Extent that reservists are comfortable informing their employers/educators they are in the reserves

Extent that reservists are confident that their employer/educator would grant them time off for their military duties

Document Review

Key Informant Interviews

Review of Performance Data

Efficiency – Are the most appropriate, efficient and economic means being used to achieve outcomes?
6. Are CFLC programs implemented in an economical manner?

Evidence of/views on whether CFLC activities are delivered in a cost-effective manner

Views on whether the progress made towards expected outcomes is adequate for the resources expended

Proportion of CFLC spending on key Outreach activities according to current CFLC strategic priorities

Trend analysis of program resources (e.g., expenditures and full-time employees)

Amount of CFLC funding as a proportion of the overall Defence budget

Document Review

Review of Financial and HR Information

Review of Performance Data

Key Informant Interviews

7. Is the design of CFLC and its programs appropriate and efficient?

Views on whether CFLC uses the right mix of programs and activities to achieve its objectives

Evidence of/views on the appropriateness and efficiency of CFLC’s volunteer-based structure

Document Review

Review of Performance Data

Key Informant Interviews

8. Are CFLC programs delivered in an efficient manner?

Evidence of/views on overall efficiency of program delivery and administration

Evidence of/views on whether there is sufficient communication and coordination between program stakeholders

Clarity of roles and responsibilities for program delivery

Evidence of/views on whether appropriate internal policies and strategic directives are in place to guide the delivery of CFLC activities

Evidence of/views on how the efficiency of program activities can be improved and alternative ways of delivering program activities and outputs

Evidence of/views on external factors that affect program delivery and CFLC’s ability to adapt to these influences and remove barriers to successful delivery

Document Review

Key Informant Interviews

Review of Performance Data

Review of Financial and HR Information

9. Does CFLC have appropriate program performance measures and data collection in place?

Presence/absence of appropriate performance measurement framework and data collection tools

Evidence of/views on the extent that CFLC collects appropriate program performance data to assess the achievement of results in a systematic manner

Evidence of/views on whether CFLC’s performance measurement and data collection are used to support strategic decision making

Evidence of/views on efficiency of data sharing and dissemination among key stakeholders

Review of performance data

Key Informant Interviews

Document Review

Table D-1 Details - Evaluation Matrix
Figure E-1

Figure E- 1. CLFC Structure. This chart shows the relationship among the internal stakeholders involved in the CFLC Program.

Figure E-1 Details - CFLC Structure


Back to Table of Contents

Page details

Date modified: