24-08 – Cadet Instructors Cadre Program Evaluation

Cadet Administrative and Training Orders (CATOs)

References

1. This order should be read in conjunction with the following references:

1. CATO 23-03, Prior Learning Assessment and Recognition; superseded by CJCR Group Order 5031-3, Prior Learning Assessment and Recognition Process;
2. CATO 24-01, Cadet Instructors Cadre Professional Development System;
3. CATO 24-02, Cadet Instructors Cadre Individual Training and Education;
4. CATO 24-03, Cadet Instructors Cadre Training Organization;
5. CATO 24-04, Conduct of Cadet Instructors Cadre Individual Training and Education;
6. CATO 24-05, Service and Support for Cadet Instructors Cadre Individual Training and Education;
7. CATO 24-06, MITE Course Management System;
8. DAOD 5031-0, Learning and Professional Development;
9. DAOD 5031-2, Individual Training and Education Management Framework; and
10. A-P9-050 Manual of Individual Training.

Policy

2. This order prescribes the policies and procedures for program evaluation within the Cadet Instructors Cadre Training Organization (CIC Trg Org).

Definitions

3. The following definitions apply to this policy:

Canadian Forces Individual Training and Education System (CFITES) is a management framework designed to optimize the quality and quantity of IT&E, while minimizing the resources dedicated to IT&E programmes. The CFITES is composed of a quality control system, a quantity control process and resource management mechanisms. (Système de l’instruction individuelle et de l’éducation des Forces canadiennes (SIIEFC))

CFITES Quality Control System refers to the six-phase systems approach model that is applied throughout the development, implementation and maintenance of IT&E programs. The phases are: Analysis, Design, Development, Conduct, Evaluation, and Validation. (Système de contrôle de la qualité du SIIEFC)

Individual Training and Education (IT&E) all instructional activities provided to CF members that impart skills, knowledge and attitudes required to perform assigned duties (training) as well as exercise sound judgement and correctly interpret information (education). Individual training and education normally results in a CF qualification. (Instruction individuelle et éducation (II et E))

Program Evaluation assesses the merit or value of an instructional program. It is a systematic process designed to collect data to assess whether instruction has satisfied the objectives of the instructional programme in the most effective and efficient manner. (Évaluation de programme)

Purpose

4. The purpose of conducting program evaluation is to ensure the effectiveness and efficiency of Cadet Instructor Cadre (CIC) Individual Training and Education (IT&E).

5. The goal of CIC IT&E program evaluation is to allow for the collection of valuable and meaningful data that will be analysed in order to improve CIC IT&E.

Responsibilities

6. The Director Cadets and Junior Canadian Rangers (D Cdts & JCR) has been delegated the responsibilities for the overall management of CIC IT&E. To ensure CIC IT&E remains relevant, effective and efficient D Cdts & JCR has tasked the following personnel with evaluation of the program:

1. Director Cadets (D Cdts) 6 – Personnel Policy & Training is responsible for ensuring CIC IT&E program evaluation is conducted;
2. D Cdts 6-4 – Personnel Training Coordination is the officer of primary interest for CIC IT&E program evaluation; and
3. D Cdts 6-4-4 – Quality Assurance Officer is the officer in charge of program evaluation.

Stakeholders

7. Stakeholders in CIC Trg Org Program Evaluation are:

1. CIC Branch Advisor;
2. Commanding Officers (COs) of Regional Cadet Support Units (RCSUs) and their staff;
3. Officer Commanding (OCs) Regional Cadet Instructor Schools (RCISs) and their staff;
4. Corps / Squadron COs;
5. Officers Commanding Sailing / Gliding / Expedition Centres; and
6. CIC officers.

Back to top

Program Evaluation Process

8. D Cdts & JCR and Regional staff will conduct formative program evaluation in relation to the development of an instructional programme through course trials and course pilots.

9. Formative program evaluation will be conducted locally by RCIS staff during the conduct of an instructional program, with the aim of suggesting improvements, by monitoring:

1. the delivery of instruction; and
2. candidate learning.

10. Summative program evaluation will take place nationally by D Cdts staff after an instructional programme has been implemented to report on its:

1. effectiveness; and
2. efficiency.

11. Clearly defined criteria have been established to allow for unbiased judgments about the effectiveness and efficiency of the instructional programme. Relevant criteria for various key programme components are based on CFITES guidance for the effective and efficient design, development and conduct of instruction in the Canadian Forces (CF).

Program Evaluation Areas

12. CIC IT&E Program evaluation will focus on several key areas:

1. learner achievement;
2. learner feedback;
3. Directing Staff (DS) feedback;
4. course content;
5. course delivery;
6. instructional effectiveness;
7. program costs;
8. courseware documentation and distributed learning (DL) components;
9. support services;
10. staff; and
11. training facilities.

Program Evaluation Tools

13. The CIC Trg Org will use various tools in order to evaluate and report on CIC IT&E:

1. Candidate web based training (WBT) course evaluation tool;
2. Candidate in-house training course evaluation tool;
3. Distributed learning instructor (DLI) course evaluation tool;
4. Directing staff (DS) course evaluation tool;
5. WBT after action report (AAR);
6. In-house AAR;
7. Quality Assurance and Evaluation Checklist; and
8. Quality Assurance and Evaluation Report.

Sources of Data Collection

14. The following sources will be used to collect data during the program evaluation process:

1. school training documentation (administrative orders, course timetable, lesson plans, etc);
2. national training documentation (Specialty Specifications (SS), Qualification Standards (QS), Training Plans (TP), Instructional Guides (IG), Assessment Tools);
3. candidate and DS course evaluations;
4. After Action Reports (AARs);
5. interviews and direct observation of training establishment staff; and
6. interviews and direct observation of candidates.

Course Trial Evaluation System

15. A course trial is conducted with draft course documentation and allows for evaluation of instructional material prior to implementation of training nationally.

16. Course trials may be conducted with or without a subsequent pilot course. At times, design changes may be so extensive as to warrant the conduct of a pilot course prior to steady-state implementation.

17. The CIC Trg Org Course Trial Evaluation System provides important information needed by the Training Management Board (TMB) to decide whether or not to move a course to steady state.

Pilot Course Evaluation Protocols

18. A pilot course is a trial of the final course design prior to implementing training nationally. Pilot courses may be conducted with or without a previous course trial. Pilot courses will normally be regionally conducted on behalf of D Cdts & JCR.

19. The CIC Trg Org Pilot Course Evaluation System provides important information needed by the Training Management Board (TMB) to decide whether or not to move a course to steady state.

Back to top

OPI: D Cdts 6

Date:  Sep 09

Amendment:  Original