Joint Audit and Evaluation of the DND/CAF Safety Program

July 2021

1258-3-035/1259-3-0054 - ADM(RS)

Reviewed by ADM(RS) in accordance with the Access to Information Act.  Information UNCLASSIFIED.

Acronyms

Back to Table of Contents

Executive Summary

Program Overview

The Department of National Defence/Canadian Armed Forces (DND/CAF) Safety Program covered under this joint audit and evaluation refers to Program 6.13 in DND/CAF’s Program Inventory which includes the following five safety areas (hereinafter referred to as “domains”): General Safety, Occupational Health and Safety (OHS), Fire Safety, Respiratory Protection and Nuclear Safety. An additional 11 domains make up the full safety portfolio but are not included in Program 6.13, and thus were out of the scope of this project. These include Ammunition and Explosives Regulation and Safety, Diving Safety, Submarine Safety, Field Safety, Range Safety, Airworthiness, Flight Safety, Air Weapons Safety, Laser Safety, Operational Safety, and Road and Vehicle Safety.

The Safety Program ensures unity of effort and purpose to enhance the health, safety and well-being of the Defence Team. The Safety Program further strengthens prevention efforts and ensures compliance with legislative and other requirements.

Project Objective and Importance

The objective of the joint audit and evaluation of the Safety Program was to assess the relevance, efficiency and overall effectiveness of the DND/CAF Safety Program, and whether the associated risk management, controls and governance processes were in place and functioning as intended. 

Hazardous occurrences (HO) are impediments to Defence capability, readiness and mission success, and thus safety is critical in a department such as DND and the CAF that inherently exposes its military personnel and, to some extent, its civilian employees to hazards on a daily basis. 

The decentralized nature of  DND and the CAF, compounded by the diversity of the operations and geographical locations, requires that robust safety activities be in place, supported by an effective governance framework and processes to ensure safety risks are effectively managed. By extension, DND/CAF may be exposed to operational, reputational, litigation and financial risks should the safety program not deliver on its intended objectives.

Key Findings

The Safety Program domains and activities as defined do not provide a comprehensive view of the full safety portfolio which includes over 16 domains, all of which are critical to DND/CAF’s operations. This is partly due to the decentralized model, wherein various Level 1 (L1) organizations act as Functional Authorities (FA) to deliver the numerous safety domains. 

Operationally, safety procedures for hazard identification and mitigation, investigations and site inspections are established but are not consistently adhered to, with some variations in compliance across domains. In certain domains, non-compliance with legislation was found, in particular with regards to the requirement to investigate all hazardous incidents. Oversight mechanisms do not adequately ensure that all required procedures are being followed. Such processes, in turn, provide the data that enable effective decision making and visibility of key risks, in addition to demonstrating compliance to legislative and policy requirements. At present, the data is mostly incomplete and thus cannot be effectively used for analysis, reporting and decision making, and risks may be accepted due to limited awareness by L1s. 

Furthermore, improvements are required to better coordinate between safety domains, leverage knowledge and minimize duplication of effort given resources limitations. Finally, the completion and tracking of training was also found to be lacking, partly due to low availability, with the result of having some untrained personnel assuming safety responsibilities. 

The work for this project was substantially completed on August 26, 2020.

Back to Table of Contents

Key Findings and Recommendations

Results

Findings are presented by themes of relevance, oversight, reporting, monitoring and performance measurement, governance and training.

Back to Table of Contents

Key Findings by Safety Domains

Back to Table of Contents

Program Profile & Context

The safety of all personnel in DND/CAF is of primary concern in the execution of every operational, technical and administrative task. With the wide range of activities performed by DND/CAF, the protection of this well-being is shared with at least 16 different safety domains, falling into 8 different L1 organizations. This complex system of safety measures and protocols requires strong relevance, oversight, monitoring, reporting and performance measurement, governance and training.

Safety Domains

Program Stakeholders

The principal stakeholders for this joint project are: 

Lead organizations: 

• Vice Chief of the Defence Staff (VCDS)
• Assistant Deputy Minister (Infrastructure and Environment) (ADM(IE))

Collaborative Organizations Consulted: 

• Chief of Military Personnel (CMP)
• Assistant Deputy Minister (Materiel) (ADM(Mat))
• Royal Canadian Navy (RCN)
• Canadian Army (CA)
• Royal Canadian Air Force (RCAF)
• Canadian Joint Operations Command (CJOC)
• Strategic Joint Staff (SJS)
• Assistant Deputy Minister (Human Resources - Civilian) (ADM(Hr-Civ)

Program Resources

The Safety Program has an average annual expenditure of approximately $5.2 million. Note this total only reflects the amount dedicated to resources and initiatives carried out at the national policy level, and does not reflect the true cost of delivering safety domains and activities at the ground level. There is currently no system in place for comprehensively forecasting and tracking safety-related expenses across the Safety Program.

Back to Table of Contents

Key Themes

The Key Findings were aligned with five themes as follows:

1. Relevance
2. Oversight
3. Report, Monitoring and Performance Measurement
4. Governance
5. Training

Back to Table of Contents

Relevance

Why it’s Important

Safety domains and activities need to be aligned with Departmental and Government of Canada priorities to make coordinated and strategic decisions on Safety for DND/CAF members and to meet legislated and regulatory requirements. 

What We Found

• General Safety is mandated by the Canadian Labour Code Part II. The FPP and RPP are mandated by applicable laws and codes. The Nuclear Safety Program is mandated by DND/CAF directives on nuclear safety, that are consistent with the requirements for health, safety, security and the protection of the environment established under the Nuclear Safety and Control Act.
• General Safety is included in Canada’s defence policy: Strong, Secure, Engaged, under the Total Health and Wellness Strategy; however, there are no specific initiatives in the Policy for the Safety Program.
• The 2019 Departmental Plan does reflect safety with a commitment to the development of a common Defence Safety Management System that will be applied to all safety domains.
• While the activities included in the Safety Program are linked to Government of Canada outcome areas related to safety, it includes only 5 of 16 key safety domains in DND/CAF (please refer to Annex C for full list of other safety domains).
• The activities of the 11 other safety domains are embedded in various Programs across DND/CAF; this decentralized approach results in difficulty confirming the alignment of safety activities and initiatives to department and government-wide priorities, policies and legislations.

Back to Table of Contents

Oversight

Why it’s Important

Proper oversight of the Safety domains is important because:

• Hazards may not be identified, assessed and corrected in a timely fashion, resulting in extended safety risk exposure to members and civilians. Hazard analyses are important structured processes to identify, assess and then eliminate or mitigate work place hazards. They inform the design of equipment, processes, work methods and systems analysis.
• Without roll-up and oversight at unit level and base/wing level, there is a possibility that hazards may be accepted at a level without the appropriate authority and limited senior management awareness of safety risks. 
• There could be potential liabilities to DND/CAF if due diligence through inspections and hazard assessment and correction cannot be demonstrated. 

What We Found

• While processes to identify and assess workplace hazards are established, they are not always implemented across all safety domains. 
• Three of the four domains examined that require oversight have processes in place to identify national issues: CFFM, for FPP and RPP, and D N Safe perform multi-year compliance reviews at the bases/wings. Conversely, General Safety does not have a national process in place to identify, assess and correct hazards. 
• Interviews with Fire Safety key stakeholders indicated that hazards are not consistently remediated within acceptable time frames, and direct observation also supported this finding. For example, during a site visit, a roof leak above an electrical breaker panel in a widely used building observed by the project team could result in electrical shocks and hazards due to slipping. 
• There are also issues with the ability to track work orders to make required repairs and maintenance to mitigate hazards, as noted by Chief Fire Inspectors. Chief Fire Inspectors also noted that while they perform follow-ups regarding corrective action, they do not have authority to expedite corrective measures implementation, other than to escalate up the chain of command.
• Hazardous incidents, such as injuries, near misses and property damage are reported through the Hazardous Occurrence Investigation Report Form (DND 663 form) and all must be investigated regardless of gravity. The form is to be completed by the supervisor of the affected personnel and verified by the UGSO or the GSO at the base, wing or formation before an investigation takes place. Analysis of the Hazardous incident data shows that 50 percent of hazardous incidents from the sites visited were not investigated, and program data for several years show a lack of investigations being reported. While the gravity will dictate who investigates, it is not formally defined when the UGSO must be involved in the investigation.
• While inspections for the Nuclear Safety program are occurring, inspections for General Safety, Fire Protection and Respiratory Protection are not all occurring as required. Inspections are essential to maintain a satisfactory standard for safety and to ensure the protection, conservation and consequent minimization of risks to health, safety, life, property and financial liabilities. 
• As reported during interviews, each of the domains and safety officers on site stated that one of the causes for not completing all required duties was due to the lack of resources and capacity. An example from the base and wing level is a shortage of fire inspectors, resulting in the Chief Fire Inspector performing inspections instead of focussing on oversight and program management to mitigate key risks. 
• The effectiveness of DND/CAF Safety Policies and Directives in reducing, preventing and resolving safety hazards and risks could not be fully assessed due to incomplete reporting and monitoring systems by the Safety domains in this project (refer to slide 15).

Back to Table of Contents

Reporting, Monitoring and Performance Measurement

Why it’s Important

Proper reporting, monitoring and performance measurement are important because:

• Systemic issues may not be identified, resulting in unmitigated risks which could lead to future HOs that could have been avoided. 
• It is difficult to track and measure outcomes of the program and make strategic or evidence-based decisions if the data being collected is unreliable.
• Without consistent and complete monitoring information, the information cannot be leveraged for continuous improvement purposes and reporting on Program outcomes. 

What We Found

• The DND 663 Form, updated in October 2019, is used to report HOs and initiate the investigation process to identify and correct the cause of an occurrence. Base/wing personnel noted that the guidance provided was not sufficient to ensure accurate and timely completion of information. Not all required information from DND 663 forms is entered and verified in the Guardian system. Therefore, many HOs are not being reported accurately, and gaps were found. 
• HO forms and Safety Program Development and Evaluation Techniques (SPDET) are not being completed with all required information, in a timely manner and sent to D Safe G. As per internal policy, D Safe G should receive and compile statistics from GSOs, conduct trend analyses on a national basis and initiate appropriate actions based on the results. Currently, no analytics is being conducted on HO data. 
• Analysis of this data should be reported in the Defence Occupational Health and Safety Annual Report; however, it has not been completed for the last two years due to lack of reliable HO data collected through the DND 663 form. In addition, in previous versions of the report, the main source of information used for the reports were self-assessments conducted by L1s which may have provided non-objective results.
• At the Program level, the Safety Program does not adequately capture safety performance data for DND/CAF. The PIP for the Safety Program includes only three indicators: Number of HOs, Injury Frequency Rate (IFR) and Injury Safety Rate (ISR). IFR and ISR have not been collected since Fiscal Year (FY) 2014/15. Furthermore, several of the Program’s key outcomes and outputs are missing performance measures. 
• This lack of Program-level data that is limited in scope has resulted in an inability to report on DND/CAF Safety Program effectiveness. This is of concern in that valid and reliable data across the 16 safety domains is not available to support senior management evidence-based decision making.

This chart outlines HOs incurred between November 2014 and October 2018 downloaded from the DND/CAF Guardian system in February 2020. Consolidated data was not available after 2018. HOs are defined in the DND 663 form and entered into the Guardian human resources database. The total number of HOs includes both verified and unverified data.
 

Back to Table of Contents

Governance

Why it’s Important

• The overlapping mandates of domains at the safety committees may result in duplication of work, or work not being actioned.
• Dual responsibilities between D Safe G and OHS Secretariat FA may lead to improper implementation of their respective mandates.
• The lack of coordination amongst safety domains, and between OHS Secretariat and the safety domains, prevents the sharing and leveraging of best practices and limits potential efficiencies between the domains and the Secretariat.

What We Found

• While safety committees such as the OHS Oversight Committee can play a role in the dissemination of information, there is a risk of gaps and ineffective information sharing among all the safety domains. This may be caused by the fact that the same group is responsible for both the OHS Secretariat and the administration of D Safe G.
• Findings suggest that given the D Safe G is also the chair of the OHS Secretariat, matters related to other safety domains may not be equally considered. It was noted in several interviews with committee members that the General Safety domain had more exposure at the meetings. This same issue was observed by the project team during the meetings they attended. Many safety domains have considered General Safety and OHS Secretariat requirements to be separate from their own policy and reporting requirements.
• Several base-level and unit-level committees do not have terms of reference. A terms of reference document is a written road map that contains clear and specific information on how a committee is organized, the committee accountabilities, roles and authorities, what the committee is trying to achieve, who the members are and when they meet. 

Back to Table of Contents

Training

Why it’s Important

• ​Base and wing personnel may not have the knowledge to address workplace hazards to ensure the safety of defence team personnel and assets. This can potentially result in hazards occurring. 
• Staff and supervisors may not know or understand their safety responsibilities if training is not delivered to all required personnel. 
• Proper record keeping allows for better tracking of who has and has not completed required training.
• Effective training is essential, contributing to the achievement of the Safety Program outcomes, in particular to ensure that professionals are equipped to fulfill key safety roles.

What We Found

• D Safe G is responsible for managing the centralized training courses for the General Safety domain. These in-class courses are delivered by contractor to all base/wing personnel who require general safety training for their position or role. For example, while UGSOs are required to complete certain centralized training courses to ensure they have the skills and knowledge to meet their responsibilities, UGSOs stated that training is not always available to meet the demands of bases/wings, causing delays in training delivery. For courses to take place on a base/wing, a minimum number of participants is required. If a base/wing is not able to reach the minimum required number of participants, the course is postponed. In addition, UGSOs noted that some bases/wings have too many personnel to train and an insufficient number of offered courses. As a result, there are UGSOs who have assumed their role but have yet to take the required training, which is a requirement in the Canadian Labour Code Part II. Further, in light of the COVID-19 environment and the need to work remotely, more flexibilities are required to meet training needs.
• The DND/CAF RPP requires that supervisors be trained in their RPP responsibilities, and that training records be maintained. The CFFM compliance reviews identified that not all supervisors are receiving training. Delivery of the RPP supervisor training is the responsibility of the local RPP Administrator. Additionally, document analysis identified that RPP training record keeping is not always completed. RPP training records are to be maintained by the Base, Wing and Unit RPP Administrators. 
• Consistent data on general safety training from base/wing to Headquarters level are not readily accessible because training is not tracked at the domain level. Interviews with the base/wing GSOs identified that tracking of training is done at the unit level. As a result, it is difficult to determine how many personnel and DND staff have up-to-date safety training.

Back to Table of Contents

Overall Conclusion

The DND/CAF Safety Program is designed to protect the 130,000 military members and civilian staff. While OHS is mandated by law, and program relevance is confirmed, it is limited to the domains within the Safety Program. Furthermore, performance data is not consistently collected, analysed and reported, which has resulted in information gaps to support informed decision making, resulting in DND/CAF being further exposed to operational, reputational, litigation and financial risks. 

Due to the lack of consistent performance data, the joint audit and evaluation was unable to conclude on the extent to which the Program is achieving its intended outcomes and thus could not conclude on its overall effectiveness. 

It was determined that while risk management, controls and governance processes are established, they are not functioning fully as intended. For example, while several of the safety domains were able to demonstrate that inspections and workplace hazards are being assessed and mitigated, insufficient oversight means some safety procedures are not being followed, with instances of non-compliance both to departmental policy and legislation including the Canadian Labour Code. At the governance level, coordination gaps amongst and between safety domains and the OHS Secretariat poses governance challenges, impacting efficiency and the ability to share best practices. Finally, safety training is another area needing attention due to limited availability, resulting in personnel not always receiving the required training to equip them in their safety roles and responsibilities.

Ensuring a comprehensive view of departmental safety activities would improve alignment with key departmental and Government of Canada priorities. Improvements to training shortfalls, ensuring better oversight of inspection and workplace hazards, while simultaneously collecting data and reporting on performance, will further strengthen efforts to ensure a robust Safety Program is in place and functioning as intended.

Back to Table of Contents

Notable Practices

Back to Table of Contents

Annex A – Management Action Plan

Management Action

VCDS/D Safe G will work with the OCIs to identify the most suitable approaches to monitoring and measuring the performance and activities of FAs of Safety across Defence.

• By March 31, 2021, convene a meeting with the FAs of Safety to:
  1. discuss approaches with C Prog on monitoring and measuring performance, including the VCDS OHSMS Framework Directive and the Safety Information Management System; and
  2. determine actual Safety Programs and Safety Domains for the Defence Team.
• By May 30, 2021, have agreed upon key performance measurements and implementation plan for FAs of Safety
• By August 31. 2021, have agreed upon key performance measurements and implementation plan by L1s and Commands
• By March 31, 2022, key performance measurements implemented by FAs of Safety and L1s/Commands.

OPI: VCDS
OCI: ADM(HR-Civ), ADM(IE), ADM(Mat), RCN, CA, RCAF, CJOC, SJS, ADM(Fin), VCDS/C Prog
Target Date: March 31, 2022

Management Action

ADM(IE) as the FA for the FPP and RPP, through the CFFM, will review the policy for both the FPP and RPP to ensure clarity in requirements for hazard assessments, investigation processes, record keeping and compliance mitigation completion. ADM(IE)/CFFM will develop an annual communications plan, to be distributed after each Active Posting Season, that will be sent to applicable base/wing leadership regarding inspection and record-keeping requirements. 

OPI: ADM(IE)
Target Date: March 31, 2022 

VCDS will ensure: 

1. To complete a review of GSP Volume 1 and Volume 2 in combination with an update of overarching policies for a Defence Team OHS Program.
GSP Volume 1 and Volume 2 will be renamed Defence Team OHS Policies and Standards.

• By July 31, 2021, draft Directives, Policies and Standards to be completed and ready for review with L1s/Commands and consultation with National Heath and Safety Policy Committee.
• By March 31, 2022, all updated Directives, Policies and Standards approved by the VCDS as the overarching FA for OHS.

2. Promulgation of VCDS Directive or CANFORGEN to L1s and Commands for the updated General Safety/ Defence Team OHS Program including requirements for the employer, managers, supervisors, all public servants and military members.

• By January 30, 2022, draft VCDS Directive to L1s and Commands on the new General Safety/Defence Team OHS Policies and Standards.
• By March 31, 2022, final VCDS Directive to be issued to the Defence Team.

3. To begin L1/Command compliance and conformance reviews and finalize the development of a national process for unit-level hazard identification, assessment and corrective measures.

• By April 30, 2021, contract in place for OHS Specialist to conduct baseline review of L1s and Commands – current practices for hazard identification, assessment and corrective measures.
• By October 30, 2021, all baseline reviews completed with interim Defence Team report and used to create internal processes for improved hazard identification, assessment and corrective measures.

4. Publication of first Defence Team OHS Annual Report.
This Annual Report is based on all activities carried out by FAs of Safety and Implementation by L1s and Commands of the various Safety Program requirements.

• By December 30, 2021, discuss with FAs of Safety what common data will need to be captured – i.e., through the common elements within the OHS Management System and which Safety Program specific elements are to be reported.
• By June 30, 2022, discuss with L1s and Commands on how they will report on common data that is required by each Safety Program requirement.
• By March 31, 2023, publish Defence Team OHS Annual Report.

OPI: VCDS
OCI: ADM(HR-Civ), RCN, CA, RCAF, CMP, CJOC , ADM(Mat), SJS
Target Date: March 31, 2023

Management Action

VCDS will: 

1. Complete Business Process for the development of the Safety Information Management System with ADM(IM) and begin Phase 1 of programming/development of high-level functional requirements (e.g., Management of Committees; Safety Training and Reporting).

• Option 1: VCDS and DM to prioritize the development of the Safety Information Management System with ADM(IM).
• Option 2: Update/Revision to Monitor Mass to be able to house OHS data, with record keeping, reporting and analytics capabilities.
• Option 3: Discussion with third party vendor of OHS Safety Information Management System software – off the shelf and modified for use by the Defence Team, upwards to $2 million (includes modifications, maintenance, licensing and cloud storage).

Target Date: March 31, 2021 

2. Reconvene the OHS Safety Board with all FAs of Safety to discuss common program elements to improve upon the OHSMS Framework.

• By March 31, 2021, convene a meeting with the FAs of Safety to:
  1. discuss approaches with C Prog on monitoring and measuring performance, including the VCDS OHSMS Framework Directive and the Safety Information Management System; and
  2. determine actual Safety Programs and Safety Domains for the Defence Team.
• By May 30, 2021, have agreed upon key performance measurements and implementation plan for FAs of Safety

3. Review the OHSMS Framework requirements with L1s and Commands for implementation and execution.

• By August 31, 2021, have agreed upon key performance measurements by L1s and Commands

4. Continue development of the Safety Information Management System addressing mid-level functional requirements (e.g., Management of Assurance of Voluntary Compliance, Directions and Refusals and Record Keeping).
Target Date: March 31, 2022

5. Complete development of the Safety Information Management System addressing low-level functional requirements (e.g., Total Health requirements, User Access/Info and General Requirements)
Target Date: March 31, 2023 

OPI: VCDS 
OCI: ADM(HR-Civ), ADM(IE), RCN, CA, RCAF, CMP, CJOC, ADM(Mat), SJS
Target Date: March 31, 2023

Management Action

VCDS will:

1. Review and update DAOD 1000-8, Policy Framework for Safety and Security Management; DAOD 2007-0, Safety; and DAOD 2007-1, GSP.
GSP Volume 1 and Volume 2 will be renamed Defence Team OHS Policies and Standards.

• By July 31, 2021, draft Directives, Policies and Standards to be completed and ready for review with L1s/Commands and consultation with the National Heath and Safety Policy Committee.
• By March 31, 2022, all updated Directives, Policies and Standards approved by the VCDS as the Overarching FA for OHS.

2. Create new or revise existing DAOD for overarching Defence Team OHS Program.

• By July 31, 2021, draft Directives, Policies and Standards to be completed and ready for review with L1s/Commands and consultation with the National Heath and Safety Policy Committee.
• By March 31, 2022, all updated Directives, Policies and Standards approved by the VCDS as the overarching FA for OHS.

3. Complete D Safe G organizational change to Director of Defence Team Occupational Health and Safety and realignment internally under four program areas with OHS Secretariat existing under the Directorate of Occupational Health and Safety.
To be completed in conjunction with all revisions and updates to Directives, Policies and Standards.

• By March 31, 2022, organizational change complete in association with new Directives.

OPI: VCDS
OCI: ADM(HR-Civ), ADM(IE), RCN, CA, RCAF, CMP, CJOC, ADM(Mat), SJS
Target Date: March 31, 2022

Management Action

A draft VCDS Directive on Mandatory Safety training will be finalized by March 31, 2021 for the Defence Team. The Directive will include the list of all courses, when they will be made available and on what platform (e.g., virtual classroom or Defence Learning Network (DLN)), and

the intended audience. The UGSO course will require Technical and Writing Boards in FY 2021/22 and should be made available by March 31, 2022. In the interim, the General Safety Individual Course (GSIC), the General Safety Supervisors Course (GSSC), the General Safety Leaders Course (GSLC), and the Health and Safety Committee in the Federal Jurisdiction course will specifically minimize the risk to the Defence Team.

VCDS will ensure:

1. Release of the VCDS Directive on General Safety (OHS) Mandatory Training requirements for managers, supervisors, public servants and military members to be tracked by L1s and Commands in HRMS/Guardian.

• By February 19, 2021, draft Directive on Mandatory Safety training drafted for review by Deputy VCDS – this includes the GSIC, the GSSC, the GSLC, Workplace Hazardous Information Management System, and Health and Safety in the Federal Jurisdiction Course training for Workplace Health and Safety Committees, with SPDET and HO Investigation and Reporting course delivered virtually.
• By February 28, 2021, draft Directive on Mandatory Safety Training for review and approval by VCDS.
• By March 31, 2021, VCDS Directive on Mandatory Safety Training promulgated to the Defence Team.

2. Release of the GSIC, the GSSC and the GSLC.

• By February 28, 2021, GSIC to be available on the DLN, initial warning order to be sent to L1s and Commands on availability.
• By May 30, 2021, GSSC to be available on the DLN and will be included in the VCDS Directive on Mandatory Safety Training.
• By July 31, 2021, GSLC to be available on the DLN and will be included in the VCDS Directive on Mandatory Safety Training.

3. Release of the UGSO Course and the General Safety Advanced Investigator Course. The UGSO Course is a requirement and will continue. The General Safety Advanced Investigators course is an optional course, and its requirements will be debated and discussed by March 31, 2021.

• By March 31, 2021, establish Writing Board and Technical Boards for UGSO content development.
• By October 30, 2021, provide all relevant data for programming to Canadian Defence Academy Borden.
• By March 31, 2022, UGSO course available on the DLN.

OPI: VCDS 
OCI: ADM(HR-Civ), ADM(IE), RCN, CA, RCAF, CMP, CJOC, ADM(Mat), SJS.
Target Date: March 31, 2022

Management Action

As the FA for the RPP, ADM(IE), through CFFM, will ensure online training is available for users and supervisors with respiratory protection requirements and will pursue a method to record the training results, in consultation with other stakeholders. CFFM will review the RPP policy to ensure clarity in requirements for training and record keeping. CFFM will develop an annual communications plan and generic briefs, to be distributed after each Active Posting Season, to build awareness for Respiratory protection.

OPI: ADM(IE)​
Target Date: March 31, 2022

Back to Table of Contents

Annex B – Methodology

Data Sources

The findings and recommendations of this report were informed by multiple lines of evidence collected throughout the conduct phase of the project. These lines of evidence were triangulated with each other and verified with program officials to ensure their validity. The research methodology used in the scoping and conduct of the project were as follows:

• Literature Review:
  A document, benchmarking and literature review of over 150 safety documents were examined and used to develop a foundational understanding of the Safety domains within DND/CAF to determine the scope of the project, which was also expanded upon during the conduct phase of the project as the documents were examined to find data that would help in the program assessment. 
• Interviews: 
  The audit and evaluation teams conducted over 35 interviews with organizations internal to DND/CAF. These responses were aggregated to inform opinion and perspectives. 
• Focus Group: 
  The project undertook direct observation of OHS Working Group Meetings to capture data from targeted populations within the safety program areas.
• Site Visits: 
  The project team visited 3 sites, including Canadian Forces Base (CFB) Winnipeg, MARLANT, and CFB Kingston. A site visit to CFB Valcartier was cancelled due to the COVID-19 travel restrictions.
• Survey: 
  The evaluation team conducted a survey. The survey was administered to targeted individuals (Safety Officers) who had been identified as being directly responsible for ensuring the effective oversight of mandated Governmental and Departmental safety domains across DND/CAF.

The finding themes were based on employing the following audit criteria and examination of the ensuing evaluation questions:

Audit Criteria: 

• The DND/CAF Safety Program policy aligns with legislation and is communicated, and safety committees are established. 
• The Safety Program has a risk-management process in place. 
• Investigations and inspections are performed according to policy. 
• Safety Program controls are implemented in compliance with regulations.

Evaluation Questions: 

• Are Safety Program activities relevant?
• Are Safety domains and policies effective?
• Is the governance and design of DND/CAF general safety and Safety Program efficient and effective?
• To what extent are safety management systems harmonized and performance measures effective?

Project Scope and Timeframe

The project examined the performance of the DND/CAF safety domains by the Evaluation team over a 5-year period spanning FYs 2014/15 to 2018/19 and by the Audit team over a two-year period spanning calendar years 2018 and 2019.

Back to Table of Contents

Annex C – Safety Specialty Programs

Scope Exclusions

The scope of this project is limited to those five domains included in Program 6.13 as Evaluations are planned based on Departmental Results Framework (DRF) design; the audit followed suit to ensure consistency. In addition, other safety domains not included in Program 6.13 may be included indirectly under other DRF segments and programs and potentially covered under other evaluations and audits.

The project did not assess the content of the training material or curriculum. 

The project did not examine OHS practices in operations or preparations for deployment or recent COVID-19 preparations.

Back to Table of Contents

Annex D – Safety Program Logic Model

Source: Chief of Programme January 2019

Back to Table of Contents