Back to Table of Contents
Assessment - Governance Structures and Mechanisms
A. Are governance structures and mechanisms in place to provide effective oversight, control, integration and decision making to support the CSOR equipment project?
Yes, for CSOR, with opportunities for improvement for OBP - To effectively support the OBP, governance should be enhanced in the areas of committee membership, oversight capacity, change management and documentation practices within CANSOFCOM
Strengths:
SRB membership will be tailored to include subject matter experts relevant to the OBP.
CANSOFCOM employs a prioritization tool to assist in decision making for minor capital projects.
OBP is seeking approval through the standard process (DCB, PMB), providing a documented process for similar program approaches in the future.
Risks:
Given the SRB’s increased authority to approve under the OBP, its membership may not include all relevant stakeholders, or be at the appropriate level to support effective oversight and decision making, which may lead to project delays.
Recommendations for OBP:
To support effective oversight and decision making within the OBP, CANSOFCOM should:
Define the proper membership and establish appropriate representation at the SRB to ensure that key departmental stakeholders (such as Chief of Programme (C Prog) and ADM(Fin)/CFO) are involved, and that project interdependencies are identified and managed appropriately;
Strengthen the capacity of the challenge function within CANSOFCOM to reduce potential negative impacts on projects within the OBP, with consideration given to including external partners as required; and
Implement a formal change management process to support the consistent management and documentation of project changes, to ensure transparency regarding decisions made and actions taken.
Back to Table of Contents
Assessment – Risk Management
B. Have risks that may preclude the achievement of the project objectives/outcomes been identified/updated, assessed, mitigated, reported and monitored in accordance with policies and guidelines?
Somewhat – While risks are managed, risk management and documentation practices should be strengthened to support the OBP
Strengths:
A risk register is maintained for the CSOR equipment project, enabling risk management throughout the life of the project.
Risks pertaining to scope, schedule and cost are discussed at each SRB, including changes in risk levels.
Risks:
If risk information is not properly documented, opportunities to effectively risk manage and mitigate issues may be missed. Additionally, risk information supports transparent decision making and demonstrates the Department’s project management capacity.
Recommendations for OBP:
To ensure relevant risk information is captured and managed to enable the attainment of OBP objectives, CANSOFCOM should strengthen its documentation practices to capture and manage risk information for decision making, including the development of:
A full risk-management plan, aligned with the Project Approval Directive requirements; and
The required risk-management tools such as the risk tolerance threshold, risk profile sheets, risk indicators and risk mitigation plans.
Back to Table of Contents
Assessment – Performance Measurement
C. Do the performance measurement and reporting processes support the achievement of the objectives and outcomes of the CSOR equipment project and the organizational business outcomes of CANSOFCOM?
Somewhat – While a performance measurement process is in place from a project management perspective, it is not sufficient to measure the outcomes of the OBP
Strengths:
Project performance, scope, schedule and cost are measured and reported annually to SRB.
The performance of the CSOR project is measured, analyzed and evaluated internally; timelines and funding levels are measured on a regular basis, consistent with the quarterly financial reporting cycle.
Risks:
Without adequate performance measures that capture the threat-based capability planning requirements of OBP, CANSOFCOM may not be able to demonstrate how program outcomes are being met.
Recommendations for OBP:
To ensure the effective performance management of OBP, CANSOFCOM should, in consultation with Evaluation and C Prog, develop the OBP performance measurement framework. This would include:
Developing, measuring and monitoring key performance indicators aligned to strategic program objectives;
Using performance results to inform lessons learned and promote continuous improvement; and
Using performance baselines, for comparison and reporting purposes.
Back to Table of Contents
Assessment – Lessons Learned
D. Are mechanisms in place to capture information to support continuous improvement (lessons learned) for program and project management?
Somewhat – Lessons learned are captured for projects. The enhanced application of lessons learned would support continuous improvement within the OBP
Strengths:
The Project Management Office is co-located within CANSOFCOM which facilitates the capture and application of lessons learned.
Lessons learned are captured via the project completion report at the end of the project.
Project Directors review close-out reports of similar projects in order to apply lessons learned to new projects.
Risks:
Opportunities to implement good practices or gain efficiencies throughout a project’s lifecycle may be missed if lessons learned are not identified, analysed and applied in a formal manner.
Recommendations for OBP:
To support continuous improvement throughout the lifetime of the OBP, and to support robust information for decision making, CANSOFCOM should develop and formalize the OBP lessons learned process, which should include:
Documenting the process to identify, collect, analyze and consolidate lessons learned;
Strengthening the measures to promote and support application of lessons learned; and
Continually documenting lessons learned throughout the lifecycle of a project, in an easily accessible, centrally located repository, which includes risk mitigation information and change proposals.
Back to Table of Contents
Conclusion
The CSOR equipment project (2007) was assessed against the most recent policies (2015 Project Approval Directive), guidance and standards as a method to provide a forward-looking assessment and to provide recommendations for the implementation of the OBP.
The CSOR governance in place provides effective oversight and control for this project. In support of the OBP development and implementation, the opportunity exists for CANSOFCOM to further enhance this governance by:
Ensuring appropriate membership at SRB meetings;
Strengthening capacity of the challenge function;
Formalizing processes and improving documentation practices in the areas of change management, risk management and lessons learned; and
Developing its performance measurement framework beyond project management.
In doing so, CANSOFCOM will strengthen oversight, promote transparency and improve information for decision making for the OBP.
Back to Table of Contents
Considerations
While outside the scope of this advisory engagement, the following risk areas were identified, which will require OBP consideration as it continues to proceed for approval:
Independent Review Panel for Defence Acquisition review
Consideration should be given to seeking Independent Review Panel for Defence Acquisition (IRPDA) input, as its mandate is to review projects > $100 million, and the proposed annual budget of the OBP is $115 million.
Management Response: Agreed. IRPDA engagement was already planned, it will be engaged for IRP 2, currently scheduled for September 2019.
Target Date: September 2019
Business Case Analysis (BCA)
The BCA document for the OBP includes analysis and justification for one option, which is not in alignment with the Treasury Board Business Case guide - the BCA “should not be used as a justification for a decision already made or for an option already selected as a foregone conclusion.”
Management Response: Already corrected. The BCA includes a full analysis of the third option (delegation). Note that the BCA was approved at DCB 2 on April 29, 2019.
Target Date: Completed
Approval authorities
Further analysis may be required to determine which mechanism should be used for approval of the OBP (Treasury Board Submission versus Memorandum to Cabinet).
Management Response: Treasury Board Secretariat (TBS) will determine the level of approval authority based on the OBP (Special Operations Forces (SOF)) Pilot PCRA and the Program Brief. Note that any capabilities that would require a Memorandum to Cabinet could be included only after the Memorandum to Cabinet is ratified.
Target Date: PCRA acknowledgement by TBS has to be completed prior to the PMB (before February 28, 2020)
PCRA Level
As original OBP documentation stated all projects under the OBP would be PCRA Level 2 or lower; and the BCA suggests some PCRA Level 3 projects could be included in OBP, consideration should be given to seeking subsequent approval of the OBP’s PCRA level.
Management Response: Corrected. The BCA does not make reference to PCRA Level 3 projects. Note that the BCA was approved at DCB 2 on April 29, 2019. There will be a single PCRA for the OBP (SOF) Pilot. The current draft PCRA indicates a Level 2.
Target date: Completed
Financial Management
The financial controls surrounding the appropriate use of Vote 5 and Vote 1 funding will have to continue to be respected.
The use and determination of project contingency funding may need to be clarified within the OBP.
Management Response: Agreed.
The financial controls surrounding the appropriate use of Vote 5 and Vote 1 funding will continue to be respected.
The use and determination of project contingency funding will be clarified during the Costing process with Director, Cost Estimate Delivery.
Target Date: The Cost Concurrence and the Financial Input Committee have to be completed prior to the PMB (before February 28, 2020)
Back to Table of Contents