Evaluation of the Cyber Forces
Table of Contents
Assistant Deputy Minister (Review Services)
Table 1 Summary
This table has two columns. The left-hand column lists the twelve key findings and the right-hand column lists the associated recommendations. Key findings and their corresponding recommendations are located across each row.
Ready Cyber Forces
Cyber Force Development
Figure 1. Cyber Forces Expenditures FY 17/18- FY 19/20 ($ millions).Footnote 1
Figure 1 Summary
This chart represents fiscal year expenditures of the Cyber Forces between FY 17/18 and 19/20 with numbers in millions of dollars. It is a stacked bar chart for each Program with figures for 17/18, 18/19, 19/20 and the totals of all three programs per fiscal year.
- 1.5 Cyber Operations: 92.7
- 2.6 Ready Cyber Forces: 78.6
- 4.6 Cyber Force Development: 56.6
- Total: 227.9
- 1.5 Cyber Operations: 73.9
- 2.6 Ready Cyber Forces: 88.4
- 4.6 Cyber Force Development: 18.8
- Total: 181.1
- 1.5 Cyber Operations: 58.5
- 2.6 Ready Cyber Forces: 70.5
- 4.6 Cyber Force Development: 15.8
- Total: 144.8
Program expenditures for each Program only go back to FY 2017/18, as illustrated in the chart. To note, FY 2019/20 figures represent planned expenditures.
In FY 2019/20, the Cyber Forces were supported by 1309 full-time equivalents in total, with 671 for Program 1.5 (blue), 602 for Program 2.6 (orange), and 36 for Program 4.6 (green).
- DND/CAF: Assistant Deputy Minister (Human Resources - Civilian), Assistant Deputy Minister (Infrastructure and Environment) (ADM(IE)), Assistant Deputy Minister (Materiel) (ADM(Mat)), Assistant Deputy Minister (Policy), Assistant Deputy Minister (Public Affairs), ADM(RS), Assistant Deputy Minister (Science & Technology) (ADM(S&T)), Chief of Force Development (CFD), Chief of Military Personnel, Judge Advocate General, Strategic Joint Staff (SJS), Vice Chief of the Defence Staff (VCDS), Canadian Army (CA), Royal Canadian Air Force (RCAF), Royal Canadian Navy (RCN), Canadian Special Operations Force Command (CANSOFCOM), Canadian Forces Intelligence Command (CFINTCOM), Canadian Joint Operations Command (CJOC).
- Other Government Departments (OGD): Canadian Security and Intelligence Service, Communication Security Establishment (CSE), Defence Construction Canada, Department of Finance, Innovation, Science and Economic Development, Privy Council Office, Public Safety, Public Services and Procurement Canada, Public-Private Partnerships Canada, Royal Canadian Mounted Police, Shared Services Canada (SSC), Treasury Board Secretariat (TBS).
- Allied & International Partnerships: Include: CYBERCOM – United States of America (USA), National Security Agency – USA (NSA), North American Aerospace Defence Command (NORAD), North Atlantic Treaty Organization (NATO), Five Eyes Partners (USA, United Kingdom, Australia and New Zealand) .
- Private Industry & Academia: There are approximately 275 firms that have been identified as related to cyber defence and cyber security in Canada, and 91 academic institutions with computer science degrees.
Figure 2 Summary
This figure represents the evolution of the Cyber Forces since its inception in 2010. It is a horizontal timeline from 2010 to 2017. The left-hand side of the figure notes that in 2010, the Cyber Task Force was established under ADM(IM). In 2011, the Cyber Task Force transferred to CFD and renamed DG Cyber. The right-hand side of the figure displays that in 2017, the Creation of the Cyber Forces occurred. At this time, DG Cyber transferred to ADM(IM), 6 SSE initiatives were linked to cyber (63, 65, 87, 88,89, 90), the Chief of Defence Staff (CDS) Cyber Initiating Directive was released, and the cyber operator trade was created.
The Cyber Mission Assurance Program
The evaluation assessed the development of the Cyber Mission Assurance Program (CMAP), which falls within the authority of the Cyber Forces. The CMAP forms a large portion of the development of the Cyber Forces and was created in response to SSE initiative 87. CMA incorporates the concepts of “cyberspace” and “mission assurance,” which is the ability of an organization, service, infrastructure, platform, weapons system or equipment to operate in contested cyberspace and accomplish its mission.Footnote 2
The CMAP is a DND/CAF-wide endeavour led by DG Cyber to create a cyber resilient defence team. It is specifically designed to address the cyberspace resilience of people, process and technology from cyber-associated threats with five lines of activities:
- Stakeholder Engagement & Collaboration
- Education & Training
CMAP is working to establish a structure of centralized oversight with decentralized execution for cyber mission assurance to enable and empower Level 1s (L1) with the appropriate Accountabilities, Responsibilities and Authorities (ARA) to do their part to ensure cyber mission assurance and cyber resilience at every level of DND/CAF. In short, the concept of CMA is everyone’s responsibility.
CMAP ObjectivesFootnote 2
- Enable informed, timely and effective risk-management decisions and action at both the pan-DND/CAF level and within the supporting Programs
- Establish standing information feeds from a diverse range of sources to inform risk-management activities
- Enhance collective action with allies and OGDs and agencies
- Improve cyberspace resilience of CAF force elements
- Improve cyberspace protection of DND/CAF critical infrastructure and services
- Enhance materiel acquisition and support (including supply chain and operational sustainment) assurance
- Close risk gaps within and between existing programmatic boundaries
- Establish standing surveillance for the development of vulnerabilities and indications that adversaries are seeking to access vulnerabilities
“Protect critical military networks and equipment from cyber-attack by establishing a new Cyber Mission Assurance Program that will incorporate cyber security requirements in the procurement process”
This report presents the results of the evaluation of the Cyber Forces, conducted during FY 2019/20 by ADM(RS) in compliance with the 2016 Treasury Board Policy on Results. The evaluation examines the performance of the Cyber Forces over a three-year period, FY 2017/18 to 2019/20 and was conducted in accordance with the DND/CAF Five-Year Departmental Evaluation Plan. The findings and recommendations in this evaluation may be used to inform management decisions related to program design, delivery and resource allocation, as well as serve as a baseline for future evaluations.
The Cyber Force was newly created upon the establishment of the Departmental Results Framework (DRF) in 2017.
- The Cyber Forces have not been evaluated previously; however, DND/CAF was part of the Horizontal Evaluation of Canada’s Cyber Security Strategy * (2017) completed by Public Safety. “The Strategy is built on three pillars: securing Government of Canada systems; partnering to secure vital cyber systems outside the Government of Canada; and helping Canadians to be secure online.”Footnote 3 The Horizontal Evaluation examined:
- The extent to which the horizontal governance structure was effective in overseeing the Strategy’s implementation;
- The extent to which participating departments and agencies implemented the Strategy’s funded activities; and
- The extent to which planned activities contributed to achieving the Strategy’s main objectives.Footnote 3
*Link last accessed on Oct 23, 2020
Program Implementation & Management
The Cyber Forces lack direction without clear ARAs and a sufficient voice to inform decision making.
Finding 1: Unclear ARAs, the current organizational construct of ADM(IM), and the lack of a Cyber Champion ██████████████████████████ of the Cyber Forces for the CAF and the ability to inform senior management decision making.
- The majority of program stakeholders believed that ARAs associated with cyber are still unclear in terms of direction and responsibilities.
- The concepts of Information Technology (IT), Operational Technology and Platform Technology (PT) have been used to delineate cyber Functional Authority (FA) to ADM(IM), ADM(IE) and ADM(Mat). However, in practice, stakeholders acknowledged that not all technology can easily fit into these concepts and further clarification is needed.
- DGIMO and DG Cyber have been working to further define ARAs, for example, determining responsibilities between cyber defence (Canadian Forces Network Operations Center (CFNOC)) and cyber security (Director IM Security (DIM Secur).
- Senior program management acknowledges that many ARAs concerning cyber are still unclear; however, they anticipate that new Defence Administrative Orders and Directives (DAOD) will help clarify them as the Cyber Forces mature.
Figure 3 Summary
This figure is a horizontal bar graph indicating a range of strongly disagree, disagree, agree strongly agree. Results reflect the percentage of respondents who align with the two statements in the chart.
There are clear ARAs that empower leaders to achieve strategic outcomes.
- Strongly disagree: 21%
- Disagree: 44%
- Agree: 31%
- Strongly Agree: Negligible
There is clear policy that direct program actions to achieve strategic outcomes
- Strongly disagree: 12%
- Disagree: 41%
- Agree: 39%
- Strongly Agree: 8%
The Organizational Context
- Having cyber resident within ADM(IM) has resulted in challenges for the cyber forces:
- ADM(IM) exists outside the operational Chain of Command, and as such, does not have command authority.
- ADM(IM), with its corporate, department-wide, proactive and long-term focus ███████████████████████████████████████████████████████████████████
- Although oversight of the CMAP is conducted within ADM(IM), the Functional Authority is held by VCDS. This has resulted in delayed approvals, according to program managers.
- Interviews with senior program managers and survey responses highlighted the important synergy between cyber and networking activities, indicating that separating them would have negative repercussions.
- Senior managers have suggested establishing a military commander within ADM(IM) with the appropriate ARAs. However, to prevent negatively impacting cyber operations, this would require additional staffing resources similar to the staffing levels of other operational commands.
The Lack of a Cyber Champion
- Interviews with program managers have raised concerns that there is an insufficient voice representing cyber at the senior-most levels.
- Review of meeting minutes from the last year (2019) of the Information Management Board, the IM/IT Capability Development Board, the Programme Management Board and the Defence Capability Board revealed little to no evidence of cyber discussion at these high-level committees.
- The CFC ARAs have not been formalized, nor have they been exercised.
- The CFC role is one of five roles of COS(IM) that is without staff support. As a result, cyber initiatives have not been able to be prioritized or raised at senior decision-making meetings.
- Although the CFC is supposed to have a direct link to the CDS, this role is not commonly exercised, which causes delays in actioning of military cyber initiatives.
- Interviewees have suggested that the role of the CFC is that of an advisor and not a commander, which is why they do not have the same ARAs as a commander.
- Interviewees believe the ARAs of a full commander are required to effectively operationalize the Cyber Forces. This could be compared to the establishment of the Intelligence Commander in CFINTCOM or the Space Commander in the RCAF.
Cyber is currently integrated into existing governance bodies, but has none of its own.
Review the current governance framework to determine whether the cyber domain requires a distinct structure.
There is evidence of strategic planning and insight into program development.
Finding 3: Although there are a number of strategic documents for the planning of the Cyber Forces, awareness and use are not evident across DND/CAF.
A number of foundational documents concerning cyber, Cyber Forces development and Cyber Mission Assurance were reviewed. Some of these included the following:
- CDS Initiating Directive (2017);
- Cyber Joint Doctrine Note (2017);
- CMAP Mandate (2018);
- Defensive Cyber Operations Concept Note (2019);
- [Draft] CMAP Charter (2019); and
- Defence Terminology Database updates.
Additionally, program managers have identified considerable work that is underway in the development of other concepts and doctrines, such as the renewal of the Joint Doctrine Note, as well as the drafting of new DAODs.
DG Cyber has produced and necessarily continues to produce multiple sources of foundational doctrines and concepts for the cyber domain...
…However, in spite of doctrines and concepts, there is still a lack of clarity.
Figure 4 Summary
This figure is a pie chart. Results reflect the percentage of respondents who agree with the pie chart statement.
- 71% agree
- 29% disagree
During interviews, senior program managers agreed that there is a lack of clarity with regard to cyber. DG Cyber has plans to more thoroughly include cyber awareness and understanding into Officer and Non-Commissioned Members professional development. However, they acknowledge that developing training could take a long time.
- Additionally, increased dissemination measures could be explored to ensure that a wider audience in DND/CAF has received and is aware of these strategic documents.
Implementation of the Cyber Mission Assurance Program has been delayed.
Finding 4: Implementation of SSE 87 will █████████████████████ as support for Cyber Mission Assurance is not universally prioritized across DND/CAF.
As mentioned previously, the CMAP is a key component of the cyber domain for all of DND/CAF. The importance of CMA was noted and acknowledged by all interviewees; however they also █████████████████████████████████████████
Program managers agreed with these concerns, referring to a number of challenges in the implementation of CMAP initiatives. In particular, ██████████████████████████████████████████████████
Figure 5 Summary
- Cyber responsibilities are often assigned to staff who have a number of other pre-established roles. As a result, CMA cannot always be a priority task to be undertaken.
- Cyber groups across DND/CAF are usually ███████████████████████████████████████████████████████████████████████████████████
Signs of Progress
- The recent establishment of a permanent team lead for the CMAP has led to improved progress and stability.
- Work is underway for the CMAP Charter to enable more reliable funding levels as the program is formalized.
- ADM(Mat) has played a very significant role in undertaking development and implementation activities for CMA and materiel procurement processes for supply chain resilience.
“CMA requires not only the IM Group, but also the VCDS and other affected L1s to take it on board”
Figure 6 Summary
This figure is a pie graph. Results reflect the percentage of expenditures dedicated to CMAP.
- 8% expenditures dedicated
- 92% expenditures elsewhere
The Cyber Forces have not identified required levels of preparedness.
Finding 5: The responsibility for defining cyber readiness ███████████████████████████████████████████ with the exception of the Navy which has made some progress.
- Defined cyber readiness levels are becoming increasingly important as the demand for cyber activities continues to grow, indicating future readiness challenges.
- Readiness will not work the same for cyber as for other operational domains. The terrain is constantly changing and a Cyber War concept is challenging to formally identify. Cyber preparedness may be a more descriptive terminology.
- There is disagreement among cyber managers as to whether cyber readiness standards need to be defined before personnel qualifications are established or vice versa. Currently this is very ad hoc, identified individually or as a team, and reported upwards instead of receiving top-down direction. This is discussed further in Finding 12.
- While it is important to ensure cyber components are integrated in Force Posture & Readiness (FP&R), it is too early to do this since the Cyber Forces are still in development. ██████████████████████████ as discussed in Finding 9, it was reported that the Cyber Forces █████████████████████████████████████████████████████████████████████████
- With respect to the responsibility for the setting of readiness standards, the strongest arguments are for the Cyber Forces to set them to ensure technical standards are established and technical connections to external organizations remain current. This is as a challenge for the IM Group to do as they were perceived by senior managers during interviews to be more focused on corporate demands versus operational.
- In the ████████████████████████████████ the Navy is in the process of developing its own cyber readiness levels within three combat readiness requirements.
Suggestion for follow-up:
Examine the formalization, direction and compliance of cyber readiness
Lack of cyber-ready workspace is impeding both operations and training.
Finding 6: █████████████████████████████████████████████████████████████████████████████████████████████████████████████████
- The draft CMA Program Charter (2020) states, “It is anticipated that suitable accommodation spaces for required staff can be found within existing DND facilities.” However, no further details were given, ██████████████████████
- Certain personnel have to travel between five separate locations across the National Capital Region in order to access the appropriately cleared networks, as the main buildings housing cyber personnel outside of Canadian Forces Information Operations Group (CFIOG) ██████████████████████████ This can result in up to $200 in taxi fares a week for one person.
- Interviewees indicated that ████████████████████████████████████████████████████████████████████████████████████████████████████████ Survey responses echoed this concern over physical infrastructure limitations.
- While the fit-for-purpose National Defence Secure Campus may eventually be a solution, in the meantime, operations, personnel and daily activities are experiencing inordinate inefficiencies.
- At the January 2020 Defence Capability Board, the CFD indicated that “the National Defence Secure Campus is a critically important capability and it must be expedited wherever possible.” The topic was discussed at the Project Management Board in March 2020.
- Survey respondents were █████████████████████████████████████████████████████████████████████████████████████████████████████████
- The Cyber Collaborative Imperative by Canadian Association of Defence and Security Industries (CADSI) noted that the leading collaboration functions, policies and practices from Canada’s Allies include Cyber Experimental Ranges and Capability Testing Environments. This “offers an environment where emerging solutions can be tested against known challenges…”
Figure 7 Summary
This figure illustrates the distances between various DND locations.
- NDHQ and Shirley’s Bay: 27 km
- Shirley’s Bay and Carling Campus: 8 km
- Carling Campus and CFS Leitrim: 35 km
- CFS Leitrim and Star Top: 11 km
- Star Top and NDHQ: 10 km
Create an institutionalized, centralized, serviced cyber range with classification adaptability and remote access.
Research & Development
DND/CAF Cyber projects are not sufficiently nimble to cope with changes in the threat environment.
Figure 8 Summary
This figure is a horizontal bar graph indicating a range of strongly disagree, disagree, agree strongly agree. Results reflect the percentage of respondents who align with the three statements in the chart.
Cyber projects are coordinated to ensure the optimal use of resources.
- Strongly disagree: 25%
- Disagree: 28%
- Agree: 47%
- Strongly Agree: 0%
There are no instances of duplication in the development of cyber projects
- Strongly disagree: 34%
- Disagree: 50%
- Agree: 14%
- Strongly Agree: negligible
The cyber project development process has mechanisms to ensure efficient project delivery
- Strongly disagree: 20%
- Disagree: 41%
- Agree: 37%
- Strongly Agree: negligible
- Force Generators and Force EmployersFootnote 5 indicated that ██████████████████████████████████ due to the current procurement process.
- All interviewees noted the ████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████
- Cyber is now part of every defence capability – there is a need to have governance bodies across DND/CAF engaged and energised. Platform CMA is tied to the capital procurement process; if this process is unclear with respect to cyber, ███████████████████████████████
- Many projects currently underway began before cyber requirements were considered, necessitating timely and costly amendments to include the cyber requirements.
Possible Mitigation Strategies
- Projects that remain under $5 million can move through the procurement process more quickly and lower the threats and risks associated with changing technologies, as well as elevate the opportunity to capitalize on opportunities that technology provides.
- ADM(IM) is discussing options with TBS in order to review capital project process with the goal of finding efficiencies, according to interviewees.
- ADM(S&T) can inform what can be done throughout prototype development to show stakeholders the type of functionalities they should be looking for when buying off the shelf.
- Implementing a funded program in order to allow the procurement of necessary tools quickly rather than project by project. The crypto program in ADM(IM) is one example of this, and CANSOFCOM is implementing capability-based procurement to address this same issue.
A properly integrated CMAP could mitigate the impact of changes in technology and the inclusion of cyber considerations.
The Cyber Forces are extensively engaged with numerous stakeholders.
Finding 8: The program design has incorporated relevant knowledge from DND/CAF, OGDs and allied stakeholders; however, engagement with private industry and academia has been a challenge.
There is significant evidence of internal engagement and involvement from other L1 organizations and Environmental Commands:
- All of the 12 internal DND/CAF cyber stakeholder organizations who were interviewed indicated that they were engaged in the development of the Cyber Forces in their respective areas and continue to be engaged with DG Cyber through Working Groups. In particular, the Cyber Steering Committee facilitates knowledge transfer; however, it is not a governance body.
- ADM(Mat) is sufficiently engaged in talks concerning PT and leverages ADM(S&T) to integrate cyber support systems.
- ADM(S&T) identifies, tests and prioritizes cyber requirements for capabilities that cannot be bought off the shelves. It also demonstrates linkages between IT and PT for the Environmental Commands.
- ADM(IE) is engaged in talks concerning Operational Technology as they work to develop and implement cyber strategies for defence infrastructure.
Figure 9 Summary
This figure is a horizontal bar chart indicating a range of strongly disagree, disagree, agree strongly agree. Results reflect the percentage of respondents who align with the four statements in the chart.
My organization has sufficient engagement with program managers.
- Strongly disagree: Negligible
- Disagree: 29%
- Agree: 55%
- Strongly Agree: 13%
My organization has sufficient voice on cyber committees
- Strongly disagree: 10%
- Disagree: 23%
- Agree: 47%
- Strongly Agree: 20%
My organization has regular communication with DGIMO
- Strongly disagree: 10%
- Disagree: 21%
- Agree: 34%
- Strongly Agree: 35%
My organization has regular communication with DG Cyber
- Strongly disagree: negligible
- Disagree: 16%
- Agree: 52%
- Strongly Agree: 29%
Overall, interviews with program managers highlighted the various relationships with OGDs. DND/CAF is present at a number of interdepartmental committees, such as the DG Cyber Strategic Committee.
- CSE was identified as a significant OGD partner with DND/CAF as the key deliverer of the Canadian Centre of Cyber Security.
- There are promising initiatives for increased collaboration between the organizations, such as training. However, the differences in corporate cultures between CSE and DND sometimes lead to miscommunication that may hinder collaborative opportunities.
- DND/CAF is regularly engaged with Public Safety, which leads a number of Government of Canada forums on cyber.
- ADM(S&T) sits on a number of cyber research bodies, such as the one run by ISED concerning workforce in training.
- A few interviewees noted, however, that there is a need for increased clarification of ARAs with SSC.
DGIMO and DG Cyber have numerous relationships with international partners and allies, which have been a source of best practices to incorporate into the Cyber Forces. In some instances, allied relationships have been more active than OGD relationships:
- OUTCAN liaison positions act as conduits with allies for information-sharing; however, ████████████████████████████████████
- Particular to the US, there are regular Cyber Coordination Committee briefings between the CAF Cyber Forces and US CYBERCOM to enable effective information-sharing and the identification of best practices.
- The US Cyber Safe Program was identified as a model for the protection and security of information, which could be emulated.
- DND/CAF participates in US Cyber Flag group training exercises to learn and exchange strategies in the development of Cyber Force training with Five Eyes partners.
- DND/CAF is also a participant in various NATO cyber initiatives and other multinational conferences.
There are 275 cyber firms in Canada, of which 250 are related to cyber IT security and 25 are specific to cyber defence.Footnote 6 Industry has a lot to offer, such as innovation and agility, which may be used to further enable the Cyber Forces. The DND MINDS program brings DND and industry to the table and may be the quickest way to incorporate cyber into DND/CAF operations and activities. However, there are a number of barriers which limit increased partnerships:
- A lack of research funding, according to industry;
- DND’s delayed engagement with industry due to current procurement processes;
- The lack of capacity for a permanent liaison position between DND and industry;
- Differing scopes of vision between DND and industry;
- Rules dictating departmental involvement in regard to contracts for capability development and capability procurement; and
- Overall security concerns due to:
- Ownership and protection of Intellectual Property of DND cyber defence capabilities; and
- Risks of firms being sold or failing.
There has been very little engagement with academia except with the Royal Military College (RMC), where engagement is robust, providing science and technology, and research and development perspectives in the cyber domain. Interviews with senior program managers identified some challenges with engaging academia:
- Inherent security risks in partnering with academia; and
- A lack of available Vote 10 grant funds to use for research.
“RMC is filling the gaps but it is not a scalable solution”
The Cyber Forces have a sufficient number of positions; however, filling the positions has proved to be difficult.
- Attracting and retaining cyber personnel for both military and civilian personnel was █████████████████████████████
- Additionally, ██████████████████████████████████████████████ by ADM(IM) corporate resource requirements.
- In a cyclical fashion, ███████████████████████████████████████████████████████████████████ as indicated in the survey.
- Progress has recently been made in developing stronger relationships with educational institutions, by attending attraction events and using the Subsidized Training Education Plan as an opportunity to entice applicants.
- CFNOC reported that they were at ████████████ at the time of the evaluation.
Civilian Cyber Force
- Civilian personnel are particularly challenging to attract and retain due to the competitive nature of cyber employment opportunities.
- 93 percent of survey respondents and the majority of interviewees agreed that DND/CAF would benefit from the hiring of (more) civilian cyber personnel, stabilizing the institutional memory.
- Hiring civilian cyber personnel will remain a challenge due to current financial compensation models, tied to classification, which has constrained the growth potential for technical personnel. This has also led to retention issues of military personnel once they become cyber-trained. The Computer Science (CS) classification is not thought of as adequate for cyber-related activities.
- “…[I]f you promote them to increase compensation, then by the job definition, they will not be doing the actual work you hired them to do.” Interviewees also indicated that many technical personnel do not want to be in management roles.
Figure 10 Summary
This figure is a pie chart. Results reflect the percentage of those who align with the pie chart statement.
- 84% ███████████████
- 16% █████████████
Figure 11 Summary
This figure is a horizontal lollipop chart, which is similar to a horizontal bar chart, but with a bulb at the end. Each bar illustrates the different salaries made by those working in the cyber domain between, CAF, DND, CSE and the private sector.
- CAF – Corporal: $67,392
- DND – CS-02: $ 75,027
- CSE – UNI-7: 78,001
- Average Canadian Cyber Analyst SalaryFootnote 7 : $97,500
A lack of cyber personnel is a risk to successful program implementation.
63% of survey respondents indicated that there is no effective career management for the cyber operator occupation.
In order to keep pace with both Allies and adversaries, DND/CAF need to ensure cyber personnel remain current and employed within the cyber domain.
Opinions remain mixed on the need for a Cyber Officer occupation.
Case Study: Cyber Officer
The evaluation conducted a case study of a quasi-Cyber Officer, after the topic arose during scoping interviews, to determine whether the expectations and technical requirements of a Cyber Operator warranted the possibility for a Cyber Officer position. As it is too early to make an informed opinion to this regard, the evaluation analyzed present opinions and context around a potential Cyber Officer Position.
The evaluation interviewed the case subject, queried survey and questionnaire respondents and discussed with interviewees.
- The role of an officer has additional responsibilities that revolve around management as well as the strategic planning and implementation of initiatives.
- There is an expected level of cyber technical knowledge and understanding which takes years to develop, to be functional in the cyber domain.
- The case subject reported that current responsibilities of Cyber Operators, specifically on Active Cyber Operations, require higher levels of initiative and resourcefulness than those usually associated with roles of typical operators but more similar to roles typically associated with officers.
- If cyber is a domain of its own, to be fully developed as all other domains, such as land, sea and air, it was argued that a Cyber Officer should exist to enable further development of the Cyber Forces.
- Even with projected growth over the next few years, there may not be enough personnel to warrant a specific Cyber Officer role to oversee the Cyber Operator cadre, where career progression would be limited by available positions.
- A more generalized officer role has been argued by some interviewees, to ensure that they are not too focused on “cyber” and losing the big picture of Command, Control, Communications, Computers, Intelligence, Surveillance and Reconnaissance, and network operations. This pool would likely draw from the overarching Communications Electronics Engineering and Signals Officer occupations.
84% of survey respondents believe that DND/CAF would benefit from the creation of a Cyber Officer occupation. The majority of these respondents were at the tactical and operational levels.
However, the majority of interviews with senior management disagreed.
- 66 percent of interviewees thought there was no need for a Cyber Officer position.
- Regardless of the opinion on the need for a Cyber Officer opinion, 53 percent of interviewees referenced a need for better career management within the cyber domain, discussed further in Finding 10.
Suggestion for Follow-up:
Examine the results of the Cyber Leadership study conducted by D Cyber Ops FD, slated to begin in March 2020, which will examine this issue ██████████████████████
Delays in security clearances negatively affect the Cyber Forces.
Finding 11: Security clearances processes and timelines ███████████████████████████████████████████████████████
- Overwhelming evidence indicates that the security clearance process and timelines for both civilian and military members ███████████████████████████████████████████████████████████████████████
- Interviewees reported that they are unable to attend necessary meetings due to security clearance requirements.
- Senior managers report being unable to recruit through certain channels such as Federal Student Work Experience Program students and co-op students, as they cannot show them the actual work they might be interested in doing due to the limitations of not having the proper security clearance requirements in place, ██████████████████████████ However, interviewees reported that OGDs are able to bring in several hundred students each summer, and have them screened within a month to the Top Secret level with polygraph. This could be further investigated by the Cyber Forces.
- The issue is two-pronged – security clearances are slow, and the work needs to be accurately classified. Interviews indicated that a lot of activity is purportedly done at the Top Secret level that does not need to be, creating a self-imposed problem. Secret and unclassified work could be maximized to reduce the burden on limited security-cleared infrastructure, discussed in Finding 6.
82% of senior manager survey respondents indicated that security clearance approvals ███████████████████████████████████████
72% of survey respondents noted that security clearance approvals ████████████████████████████████████
CAF training is unable to keep pace with cyber needs; however, third-party cyber training lacks standardization and validation.
Finding 12: As cyber training continues to be developed and formalized, █████████████████████████████████████████████████████████████
- Interviewees agreed that due to the complexity and constant evolution of the cyber domain, CAF training is not agile enough to keep pace. Third-party training is preferred by 91 percent of survey respondents but requires standardization and validation.
- One interviewee noted that “[t]he Canadian Forces School of Communication and Electronics (CFSCE) alone cannot sustain the training of cyber.” This is supported by the Cyber Operator Occupation Briefing (2017) which noted that the Cyber Forces currently have a “Just enough just in time” training model.
- Third party training is thought to be more cost effective and may better enable the Cyber Forces to react to the rapid advancement seen in this realm.
- Comments from survey respondents further highlighted this gap in training, as respondents were concerned that the lack of standard expected skillset paired with the absence of third-party course validation poses a risk that operators may not be meeting the level of skill required.
- As discussed in Finding 9, ███████████████████ This presents the CAF with the alternative to choose between conducting cyber operations or providing training to the cyber force to build up necessary skill sets. Evidence shows that operations were prioritized.
The draft CMA Program Charter (2020) states “It is anticipated that some specialist training will be required to develop the necessary skill sets to enable this program - the nature of which will be determined during options analysis.”
In FY 2020/21, the CMAP is expected to complete a training needs analysis, formalize training and awareness framework, and issue common training guidance, all of which are reported to be on track.
Without knowing whether cyber operator training is achieving an expected standard, cyber effects may face unintended consequences.
Assess the feasibility of standardizing third-party training with training validations.
As a new group, the Cyber Forces are actively engaged in numerous activities to establish Canada’s position in the cyber domain of warfare. DG Cyber and DGIMO, as the leads for cyber in DND/CAF, have produced and are currently working on various initiatives to set in place the foundational components needed for an effective Cyber Force in the future. However, unless appropriate attention is given to the cyber domain, the rate of implementation will continue to be constrained.
The Cyber Forces’ program design theory is robust. Extensive strategic planning and insight into the development of the Cyber Forces is apparent through the production of cyber concepts and doctrines, as well as the formalization of terminologies. Further, DG Cyber and DGIMO have maintained extensive engagement to ensure that relevant knowledge and stakeholder engagement were included in the Cyber Forces’ design. Continued regular engagement with stakeholders as well as the incorporation of best practices and lessons learned will enable an effective Cyber Force.
The Cyber Forces need DND/CAF-wide support and investment to ensure a holistic and effective implementation of CMA and other cyber initiatives. DND/CAF must undergo a cultural shift to recognize the importance of CMA, as it is critical to all DND/CAF activities and operations. Pan-DND/CAF cyber initiatives cannot be implemented without the support of L1s and their respective cyber teams, and processes may need to evolve in order to effectively enable the Cyber Forces. Presently, cyber implementation activities are at risk of delays and constraints without the appropriate resources to facilitate their efforts across the Department.
Cyber stakeholders require greater strategic guidance. In spite of a robust program theory, cyber stakeholders have indicated that increased strategic direction is needed to guide their respective implementation of cyber initiatives. Instead, cyber stakeholders are acting without guidance (e.g., cyber readiness). During the evaluation period assessed, there was no overarching DND/CAF Cyber Vision.
Early program developments indicate signs of progress for training of the Cyber Forces. The creation of the Cyber Operator trade required rapid development of courseware and professional development to support the new trade. Additionally, initiatives to increase awareness and knowledge of the cyber domain among military and civilian members have been rolled out across the country.
Annex A — Management Action Plan
ADM(RS) Recommendation 1
To make Cyber Forces management more effective, ADM(IM) should review, update and promulgate cyber relevant ARAs across DND/CAF to ensure their awareness.
Management Action 1
- ADM(IM) recognizes the Armed Forces Council decision of February 2018 that endorsed the ARAs for the creation of CFC, Chief of the Cyberspace Staff (C Cyber) and the Joint Force Cyber Component Commander (JFCCC). The C Cyber position was established by DM/CDS in NDHQ Organization letter in February 2018.
- Three part action plan as follows:
- 1.1: Determine the status of the Cyber Force leadership positions such as CFC, C Cyber, JFCCC, and take necessary steps to ensure that the positions are properly established;
- 1.2: Upon further strategic direction, work with L1 stakeholders to determine the proper ARAs for each role (Force Management, Force Development, Force Generation, Force Employment) within the Cyber Force at the senior leadership level; and
- 1.3: Promulgate ARAs using appropriate organisational instruments including DAODs, doctrine and DND policies.
OPI: C Cyber/DGICFD
Target Date: March 2021
ADM(RS) Recommendation 2
Review the current governance framework to determine whether the cyber domain requires a distinct structure.
Management Action 2
- C Cyber and Directorate General Information Capability Force Development (DGICFD), in consultation with the VCDS and other L1 Authorities will develop, evaluate and make recommendations on whether or not the cyber domain requires a distinct governance structure. The recommendations will consider:
- 2.1: Matching governance mechanisms for the cyber domain with land, maritime, air and space domains in line with the CAF approach to Pan Domain Force Employment Concept;
- 2.2: Ensuring that the governance of the CMAP reflects the pan-CAF/DND nature of cyber resilience in people, processes and technology leading to CAF mission success in any cyber contested domain; and
- 2.3: The organizational design of governance systems to ensure they are practical and sustainable.
OPI: C Cyber/DGICFD
Target Date: June 2021
ADM(RS) Recommendation 3
Create an institutionalized, centralized, serviced cyber range with classification versatility and remote access.
Management Action 3
- The C Cyber and DGICFD acknowledge the concerns with the limited access to cyber ranges used for simulations and training, as well as the physical infrastructure to access them.
- Three part action plan as follows:
- 3.1: Formally accept the Collaborative Security Test Environment/Interim Cyber Training Environment as the interim CAF cyber-immersive training environment;
- 3.2: Determine the CAF’s cyber-immersive training environment requirements in coordination with all L1s; and
- 3.3: Determine if a capital project is required as the cyber range permanent solution.
OPI: C Cyber/DGICFD
Target Date: March 2022
ADM(RS) Recommendation 4
Assess the feasibility of standardizing third-party training with training validations.
Management Action 4
- As C Cyber and DGICFD develop the cyber training and where third-party training is considered, it is being validated in conjunction with the Cyber Training Authority (Chief of Military Personnel/Military Personnel Generation Training Group (CMP/MPGTG)). Some existing third party training is the industry standard (certification or qualification), which does not require validation.
- Three part action plan as follows:
- 4.1: In conjunction with the Cyber Training Authority, DGICFD will continue to validate third-party training. We will leverage third-party training, including allies as required where there is a training gap;
- 4.2: Continue working the CAF-ACE program to recognize admissible Post-Secondary Institutions; and
- 4.3: Continue working with the Government of Canada (GC) Cyber Skills Workforce Development Working Group that is examining holistic cyber training across the GC, utilize common areas of interest for cyber individual training and education, and, where possible, share validation and lessons learned.
OPI: C Cyber/DGICFD
Target Date: Validation process ongoing / GC Cyber Skills Working Group output, July 2021
Annex B — Gender-Based Analysis Plus (GBA+)
As per the TB Directive on Results (2016), Mandatory Procedures for Evaluation, this evaluation took into account the government-wide policy on GBA+ as it was deemed relevant.
The Evaluation Matrix included a Key Performance Indicator dedicated to GBA+: “GBA+ considerations are included in the hiring process”
The Cyber Forces Survey that was distributed included various GBA+ questions:
- Within the Self-Identification section, for the purpose of disaggregation of data, we asked whether respondents were military, civilian or ex-military; their age; primary and secondary language; sex; gender-identity; and ethnic origins.
- The questions below were included in the Training section:
|Cyber Forces Survey Questions||% Agree|
|The planning and implementation of the Cyber Forces takes into consideration DND/CAF diversity and inclusion initiatives.||96%|
|Cyber Forces policies incorporate GBA+ considerations.||85%|
|GBA+ considerations are included during the hiring process for cyber personnel.||87%|
|Gender and diversity initiatives are taken into consideration in the development of training for cyber operators.||77%|
Table B-1 Summary
This table represents a sample of questions from the Cyber Forces Survey that related to GBA+. It has two columns and five rows. In the first row are titles for the “Survey Question” column, and the “% Agree” column.
- The planning and implementation of the Cyber Forces takes into consideration DND/CAF diversity and inclusion initiatives. – 96%
- Cyber Forces policies incorporates GBA+ considerations. – 85%
- GBA+ considerations are included during the hiring process for cyber personnel. – 87%
- Gender and diversity initiatives are taken into consideration in the development of training for cyber operators. – 77%
- While the survey results were positive regarding GBA+ considerations, interview responses were varied. Many voiced that there were no unique GBA+ factors incorporated during planning or hiring, and no metrics are being tracked.
- One questionnaire response noted that DG Cyber is an equal opportunity employer and will hire anyone with various identity factors as long as they meet the educational and technical qualifications and can obtain the required security clearance. That same response indicated that they are currently assessing whether access to their services have any barriers to equality for those of differing abilities.
Status of Women in Canada defines GBA+ as:
an analytical process used to assess how diverse groups of women, men and non-binary people may experience policies, programs and initiatives. The “plus” in GBA+ acknowledges that GBA goes beyond biological (sex) and socio-cultural (gender) differences. We all have multiple identity factors that intersect to make us who we are; GBA+ also considers many other identity factors, like race, ethnicity, religion, age, and mental or physical disability.
Only approximately 5 percent of respondents to the Cyber Forces Survey identified themselves as women, and the majority of those were military members. In reflection of the CDS’ initiative to increase the presence of women in the CAF, certain initiatives could be explored to align with that intent. In the Office of the CDS Canadian Armed Forces Diversity Strategy (2016), it states “it is imperative that the Canadian Armed Forces (CAF) reflect the society it serves if we are to connect with Canadians and retain our relevance as a national institution. … Moreover, our operational experiences have demonstrated that diversity is a force enabler that enhances our operational effectiveness.” According to SSE Initiative 10, DND/CAF will fully leverage Canada’s diversity by promoting diversity and inclusion as a core institutional value across the Defence team.
Of respondents to the Cyber Forces Survey, for primary language statistics, 81 percent indicated English, 18 percent listed French, with 1 percent noting Mandarin. Approximately 15 percent of the respondents indicated an ethnicity other than white, illustrating a certain level of diversity in the Cyber Forces.
Awareness of differing identities should be actively included in all facets of the Cyber Forces to ensure an inclusive and diverse force structure.
Annex C — Evaluation Methodology
The findings and recommendations of this report were informed by multiple lines of evidence collected throughout the conduct phase of the evaluation. These lines of evidence were triangulated with each other and verified with program managers to ensure their validity. The research methodology used in the scoping and conduct of the evaluations are as follows:
As part of the planning phase of the evaluation, a preliminary document review was conducted to develop a foundational understanding of the Cyber Forces and Cyber Mission Assurance to determine the scope of the evaluation. This was expanded upon during the conduct phase of the evaluation, as other documents were examined to find data that would help in the assessment. Documents included: government websites; departmental administrative reports, program documents, both in draft and finalized; and external reports.
During the conduct of the evaluation, particular issue areas were identified for further clarification and information. As a result, a number of short form questionnaires (2 – 3 questions) were sent to key points of contact through email. Organizations contacted included: DGIMO, ADM(IE), VCDS, as well as the Communications Electronics Engineering and Signals Officer career managers.
The evaluation team conducted a case study concerning the necessity of a cyber officer trade. This study drew upon military documentation, survey data, interview notes and administrative data. Further information concerning the case study can be found in the report.
The evaluation team conducted over 30 interviews with organizations internal and external to DND/CAF. These responses were aggregated to inform opinion and perspectives in support of the evaluation. Unless otherwise noted, reference to “senior program managers” only refers to those who are at the director level and above in DG Cyber and DGIMO. Organizations interviewed included:
- DG Cyber
- Director Project Delivery Command and Control
- DIM Secur
External to DND/CAF
The evaluation team conducted a survey over the month of October 2019. Certain questions were targeted for key individuals, such as senior managers, military members or cyber stakeholder organizations external to DG Cyber and DGIMO. Charts throughout the report reflect these population nuances in their titles. Unless otherwise stated, “survey respondents” refers to the entire survey population.
In order to engage a wide population for opinions, perspectives and GBA+ data, the evaluation developed a survey in English and French. Certain questions in the survey were developed for targeted members of the population, for example, senior managers, CAF members only, or responses from cyber stakeholders external to the Cyber Forces. These nuances are reflected in the charts illustrating the data throughout the report.
The survey was administered to targeted organizations and individuals who work within the cyber domain or have connections to the Cyber Mission Assurance Program across DND/CAF. Survey distribution relied on the Points of Contact identified through research of the DND Directory. These individuals would then pass the survey to other relevant individuals or subordinates in their Chain of Command.
The survey remained online for approximately one month, and had a total of 120 responses from ADM(IM), ADM(S&T), ADM(Mat), ADM(IE), JAG, SJS, VCDS, CFINTCOM, CJOC, CANSOFCOM and the Environmental Commands (CA, RCN, RCAF). To note, additional respondents from OUTCAN positions submitted surveys through Microsoft Word, as they did not have access to the DWAN.
The evaluation team visited CFS Leitrim and conducted five interviews with individuals working at CFIOG, this included the Commander of CFIOG. Due to limited security clearances, the team was unable to get a tour of the site, but a number of presentations and briefings by CFIOG units were given to the evaluation team.
The evaluation team conducted a comparative analysis by benchmarking Canada’s Cyber Force with the military cyber groups of the United States, the United Kingdom and Australia. Using various indicators, data was collected from various online sources, such as government websites, to enable comparison.
The evaluation undertook a number of focus groups to capture data from targeted populations within the program areas of the population. In particular, a focus group was held with Director Information Management Engineering and Integration.
Annex D — Evaluation Limitations
The limitations encountered by the evaluation, and mitigation strategies employed in the evaluation process.
|Security Clearances||The nature of the Cyber Forces puts much of its business in the Secret and Top Secret realm.||The evaluation was kept at the unclassified level, scoping out areas that would not meet this criteria, specifically C4I content, as described in the Evaluation Scope.|
|Survey Access||The survey that was distributed was not able to be accessed electronically by all cyber personnel, in particular those in OUTCAN postings.||Upon receiving information about such challenges, the evaluation team sent Microsoft Word versions of the survey and manually inputted their response data to be included in analysis.|
|Nascent Programs||Due to the fact that the Cyber Forces are new and thus a formative evaluation was conducted, many documents were amended and updated as the evaluation progressed.||The evaluation team kept in regular contact with program stakeholders to obtain current drafts of pertinent documentation.|
|Survey Selection Bias||Bias could arise based on the selection of the individuals or organizations chosen for the survey, which could skew survey results.||All organizations that are connected to Cyber were contacted for the purposes of the survey. Respondents were selected from units from within the respective member organizations.|
|Interview Bias||Bias could arise based on the subjective impressions and comments of interviewees, which could lead to biased views.||Interview comments were corroborated with other sources to ensure validity. Interview notes were conducted by more than one individual to confirm understanding of discussions and decrease the likelihood of bias.|
|Global Pandemic||Due to the outbreak of COVID-19 on a global scale and within Canada, the evaluation was unable to complete the final round of high-level stakeholder engagement.||Evaluation team members conducted the final phase of the project remotely.|
Table D-1 Summary
This table represents evaluation limitations and mitigation strategies. The table has two rows and two columns, including titles. The first column contains information regarding limitations and the second column contains information regarding mitigation strategies.
Report a problem or mistake on this page
- Date modified: