Review of Project Management and Oversight

From: Employment and Social Development Canada

Alternate formats

On this page

1. Background

1.1 Context

Project Management is the systematic planning, organizing, monitoring, and control of allocated resources to accomplish identified project objectives and outcomes. It is different than operations such that it is non-repetitive, time limited, and renders a specific product or service. The management of projects is key to providing value for money and demonstrating strong stewardship in project delivery.

In recent years, project management was acknowledged as a high risk in Employment and Social Development Canada (ESDC)’s Corporate Risk Profile, and Branches recognized project management as one of their top learning needs. Overall, the Department has implemented various project management related processes that are at different stages of maturity to help the organization manage the more complex projects. In spring 2017, Ernst & Young conducted a Project Management Maturity Assessment for ESDC. The assessment concluded that good progress has been made by the Enterprise Project Management Office (EPMO) to establish the foundation for standard project management practices within ESDC, but more time is needed to refine the processes and gain full adoption across the branches. This will require substantial investment in people and training, and a commitment to changing current behaviors and attitudes that are interfering with effective project management.Footnote 1

Each year, the Department invests hundreds of millions of dollars in projects to introduce new capabilities, transform existing capabilities, and improve service delivery to Canadians. In 2014, a new Project Management Framework as well as EPMO were introduced to provide a consistent approach to delivering projects across the Department. In the future, the Department will be undertaking many projects that represent a high degree of risk and complexity, which could require a higher level of project management maturity. Therefore, optimally managing projects in terms of their scope, schedule, budget, resources and strategic delivery can benefit the organization.

1.2 Review Objective

To assess to what extent the Department manages and delivers projects on time and on budget while respecting the project scope and realizing the stated benefits.

1.3 Review Scope

The scope of this review included a horizontal assessment of project management practices, and assessments of individual projects. Our sample of projects included completed projects that have succeeded, failed, and projects that are still in-flight. Project management and delivery was reviewed in the following three areas:

Processes

  • Integration: Review and analyze the effectiveness of cross divisional planning and execution of projects during initiation, execution and monitoring processes.
  • Human Resources: Review how management assesses and manages human resource capacity to support project planning and delivery.
  • Cost Management: Review and analyze the effectiveness of the project cost, budget development and monitoring processes.
  • Scope Management: Review and analyze the project scope, requirements definition, validation, control processes and techniques being used by project managers and governing bodies.
  • Schedule: Review and analyze the project schedule development and control processes (including change requests) and techniques.

Knowledge base

  • Review the skills assessment, training, competency profiles of individuals leading and supporting projects across the organization.

Culture

  • Review and analyze current challenges and barriers to further the maturity of project management as a distinct discipline and practice area.
  • Review and analyze the enterprise approaches and techniques that are in place to enable a learning oriented organization.

1.4 Methodology

A sample of projects was tested during the review to confirm if the required artefacts per the Project Management Framework were completed. The projects were selected independently by the review team and validated with the co-chairs of the Major Projects and Investments Board (MPIB) to confirm that they were representative of the project management processes that were under scope. Given that only investment projects were included in the MPIB listing, non-investment projects, including projects led by Branches and Regions, were out of scope for this review.

Samples were selected based on a cursory review of the MPIB portfolio of projects for fiscal years 2016–17 and 2017–18. The scope of the sampling was expanded beyond the current fiscal year to ensure that both “in-flight” and completed projects were reviewed. MPIB approved a total of 82 projects during 2016–17, 8 of which were selected for testing. MPIB approved a total of 69 projects during 2017–18, 18 of which were selected for testing. Of the 18 projects, 10 projects were from 2016–17 as carry over projects and 8 were from 2017–18 as new projects. Therefore, a total of 26 projects were selected for testing.

The 26 projects included in the sampling were in various stages in their project life cycle:

  • 7 in Stage 5, Close-out
  • 4 in Stage 4, Execution
  • 6 in Stage 3, Planning
  • 6 in Stage 2, Initiating; and
  • 3 in Stage 1, Justification

The review team also assessed the accuracy of the project status dashboards to confirm if they adequately reflect the health of the project in terms of the scope, budget and timelines.  

2. Review Findings

2.1 Project Management Processes

In 2014, the Department developed the Project Management Framework based on Treasury Board (TB) policy to improve its project management maturity and capacity, including the development of its project management related processes, the establishment of the mandate, formal accountabilities, and roles and responsibilities for key project stakeholders. The adoption of this Project Management Framework provided a consistent approach to delivering projects across the Department.

The Department manages a rolling portfolio of major (estimated cost of over $1,000,000), minor (estimated cost under $1,000,000) and small projects (estimated cost under $250,000) with an estimated value of $465,000,000 based on the 2017–18 list available at the time of this review.

Governance

Various governance committees provide guidance and decision making on the Department’s projects over their life cycle: the Portfolio Management Board, MPIB, Management Board Steering Committee, Service Management Committee, Service Delivery Committee, Service Transformation Committee, various Directors General level committees, Project Steering Committees, and others. The Investment, Procurement and Project Management Directorate within the Chief Financial Officer Branch (CFOB) provides secretariat support to governance committees, especially MPIB, as well as an independent project function to support the decision-making process. As per their Terms of Reference and records of decisions from Committee meetings, MPIB has a mandate to support rigorous and transparent investment decisions, project planning and project management for major projects.

During the review, it was noted that senior management, at the Director General level and above, was engaged within governance committees. However, overlap and redundancies were found within these governance committees. There are multiple governance committees with similar roles and overseeing the same projects. Furthermore, the same committee members are involved with different committees. While this may increase oversight, it causes delays in obtaining stage gate approvals when projects must be approved by multiple governance committees. This also creates confusion for project teams with respect to the appropriate courses of action for approvals. Interviewees indicated that there are some governance committees where approvals are not provided. We were also informed that delegates are often sent to replace absent committee members. In October 2017, EPMO developed a project governance “journey map” as a means for project managers to understand the routing and requirements for the various oversight and governance groups.

Processes and templates

The departmental Project Program Management Framework (PPMF) (Version 2.0), published in January 2017, is an overarching framework for project delivery aligned to TB PolicyFootnote 2. The EPMO developed process maps for various stages in a project life cycle from initiation to lessons learned. Existing guides have been updated and new guides have been introduced for key aspects of project management that address the targeted areas of improvement in project integration and project management standards. Within the project management process, material high level risks that could significantly impede the Department’s ability to deliver on strategic projects are identified, prioritized and monitored through Stages 2 to 5 of Project Portfolio Management Process for all projects.

ESDC named the Project Management Information Solution (PMIS) as the project management system of record in 2014. PMIS is a mandatory departmental information system for project management which includes Microsoft Project Server and SAP. PMIS is used to standardize templates and connect databases across the Department to better manage project information. PMIS is intended to assist project managers with their project management activities. However, interviewees indicated that PMIS can be tedious to use and update. This may explain the inaccuracy of some dashboards produced.

The Department has recently invested in project management tools, provided training and support on project management to employees and continues to monitor all projects within its portfolio. It was observed that only projects governed through the MPIB process (Investment or Technology enabled projects) are currently being managed within the PMIS and follow the PPMF. Interviewees indicated that the EPMO is viewed as being process heavy which deters projects from being managed through this channel. A project classification system that categorizes projects into three distinct groupings (small, minor and major) has been created by EPMO but the process is not currently scalable and all projects require the same level of rigour, regardless of their size and complexity. This may be causing some of the resistance to the adoption and use of the EPMO and PMIS.

It was also observed that there is not a clear universal definition or set of criteria to determine if a series of activities or an initiative is categorized as a project and subject to the defined project management processes and framework. ESDC’s Strategic Framework 2017–20 includes a significant number of strategic initiatives, which may or may not be projects, which are not being managed as projects within the PMIS or PPMF.

Stakeholder Engagement, Integration and Capacity

From the review of project files it was observed that often Innovation, Information and Technology Branch (IITB) conducts user acceptance testing (UAT) on behalf of the business branch as the skills needed to conduct the testing do no reside within the business branch. This practice does not align well with best practices in project management as the UAT should be conducted by the client, not the project team.

Benefits Realization

As part of the EPMO’s objectives to continually improve processes, a Benefits Management Framework was developed in March 2017 and is currently being piloted with several projects. Full launch will commence after lessons learned from the pilots are completed (including process enhancements). The Framework includes a working group supporting benefits management as well as gathering and disseminating project management best practices.

Performance Reporting

In reviewing and testing for the completeness of the sampled projects obtained from the 2016–17 and 2017–18 MPIB project portfolios, it was observed that certain required project documentation was either incomplete or absent from the PMIS (see Appendix B for details). It was also observed that formal records (Records of Decision) and other project management documentation (Meeting Agendas and Minutes) were not retained within the PMIS. We were informed that a Project Management Quality Assurance Framework was in place and that CFOB has proposed changes to enhance the existing framework effective June 30, 2017. The Quality Assurance Framework has not been assessed as part of this review, but it is intended to ensure that all pre-requisite project stage gate activities and deliverables have been completed in advance of pursuing approvals to pass the project gates. This approach could significantly improve the information provided for decision making purposes and strengthen compliance with the project management related processes and templates. Careful consideration should be given to the change management methodologies when implementing this new assurance process to ensure support and timely adoption by stakeholders. Awareness sessions and training on the assurance process should be considered for Project Managers during implementation.

Project issues are managed through an Issues Management Framework that was implemented in April 2016. We were informed that a process for identifying and managing risk has been developed and implemented, but we found the understanding of the application of the process and of project management risk principles varied greatly among interviewees.

Scheduling, Risk and Issue Management

Project scheduling has historically been a challenge within the Department that has resulted in poor quality project schedules being developed during the project planning phase. This has resulted in significant project changes (change requests) to re-baseline schedules as a result of projects often being reported as behind the planned schedule. To support the need for improved planning, EPMO intends to hire eight professional schedulers to assist the departmental project management community with project planning and scheduling activities.

Project Management Process Maturity

The Department has made noteworthy efforts on process standardization in project planning and management. ESDC has also initiated the formalization of a Quality Assurance Framework including the quantitative measurement of project deliverables.

The table below demonstrates the maturity of the Department in its project management processes based on the results of our review.

Figure 1: Project Management Process MaturityFootnote 3
Graphs comparing the proportions of Project Management Process Maturity. Text version below.
Description of figure 1

The table below demonstrates that there are five levels of project management process maturity along with a description of each level of maturity as well as the produced results. Based on the results of our review, the Department is currently at level 3 – Defined.

Project Management Process Maturity Level Description of Maturity Level Result
Level 5. Optimizing Continuous Process Improvement Productivity and Quality
Level 4. Quantitatively Managed Quantitative Management Productivity and Quality
Level 3. Defined (ESDC position) Process Standardization Balance between Productivity and Quality and  Risk and Waste
Level 2. Managed Basic Project Management Risk and Waste
Level 1. Initial Heroic Efforts Risk and Waste

2.2 Project Management Knowledge Base

Skills and Competency

The Department has identified key roles in project management such as Project Executives, Project Schedulers, and Project Managers. Project Program Portfolio Management Competencies Skills and Abilities Matrix, by role for all project management staff, have been developed. However, skills and experience for project management are specialized and interviewees indicated that often individuals assigned to projects do not have the necessary understanding of project costing, financing, scheduling and resource management. Project Managers were assigned multi-million dollar projects without any prior project management experience or training. They are often placed into project management roles without adequate training or skills to ensure success of assigned projects. There is currently no policy or standards in place to ensure that managers assigned to projects possess the competencies to initiate, plan and execute projects.

Some interviewees noted that they hired a consultant to manage projects who had the experience and was responsible for completing the necessary project artefacts as they did not possess the skill to develop project documents and plans.

Training

The EPMO has developed training material and content for Project Managers and other key roles. However, there is no formal mandatory training in place for new or existing project managers. Furthermore, there is no specific plan or course of action to ensure that all project management resources are appropriately trained prior to assuming project management roles.

Project Management Knowledge Maturity

The Department has made efforts in developing localized and repeatable practices in applying project management related skills and knowledge. However, there are still recognizable opportunities for improvement.

The table below demonstrates the maturity level in the project management knowledge based on the results of our review.

Figure 2: Project Management Knowledge MaturityFootnote 4
Graphs comparing the proportions of Project Management Knowledge Maturity. Text version below.
Description of figure 2

The table below demonstrates that there are five levels of project management knowledge maturity along with a description of each level of maturity as well as the progression gained. Based on the results of our review, the Department is currently at level 2 – Develop.

Project Management Knowledge Maturity Level Description of Maturity Level Gained Progression
Level 5 - Innovate Continuously improving practices Scalable knowledge
Level 4 - Optimize Measured and adaptive Enable Knowledge
Level 3 - Standardize Common processes and approaches Applied Knowledge
Level 2 - Develop (ESDC position) Localized and repeatable practices Ad Hoc knowledge
Level 1 - Initiate Growing Awareness

2.3 Project Management Culture

The Department has made significant improvements towards maturing the project management framework including tools and technology, knowledge base, skills and competencies. Efforts were also made to align behaviors with best practices in project management and to provide support at the Project Manager and Senior Executive levels. However, interviewees indicated that some areas and levels of management are not fully engaged and supportive of the value and discipline associated with professional project management. In some cases, the project management framework and processes can be perceived as an administrative burden adding little value to their respective business units.

Communication and Change Management

It was found that in some projects, such as the ERP-SAP project, best practices in change management were used to expedite change and minimize the resistance of implementing a significant change. This included the design and delivery of facilitated information sessions to gain insights and an understanding of new processes and systems. However, there was no evidence of change impact assessments being conducted for all projects.

Lessons Learned

There is an established community of practice for sharing information, knowledge and lessons learned with respect to project management. It was observed that individual projects retained “lessons learned” reports in PMIS; however, this information is not easily accessible or centralized for other project managers or staff to review.

Project Management Cultural Maturity

The table below demonstrates the maturity of the Department in its project management culture based on our review.

Figure 3: Project Management Cultural MaturityFootnote 5
Graphs comparing the proportions of Project Management Cultural Maturity. Text version below.
Description of figure 3

The table below demonstrates that there are five levels of project management cultural maturity along with a description of each level of maturity. Based on the results of our review, the Department is currently at level 2 – Exploratory.

Project Management Cultural Maturity Level Description of Maturity Level
Level 5. Adaptive Culture of innovation, continuous learning and sharing
Level 4. Adoptive Models integrated, roles clear, performance metrics in place
Level 3. Defined Actions for improvement defined and shared at all levels of management
Level 2. Exploratory (ESDC position) Leaders defined, executive sponsorship
Level 1. Unaware Culture not addressed and processes not in place

3. Areas for Improvement

While there have been significant enhancements with regards to project management and oversight since 2014, the following areas for improvements have been identified.

3.1 Project Management Processes

  • Clear criteria and definitions for both “Project” and “Initiative” should be developed and communicated across the Department. The boundaries of the PPMF should be expanded beyond the portfolio of investment projects to capture and govern all projects (non-routine operational activities) within ESDC. It should be noted that the capacity of PMIS and the EPMO should be assessed with respect to impacts related to changes made as resolving the definition of “Project” could potentially result in a large volume of projects requiring oversight.
  • The EPMO should consider modifying some of the project management processes to allow for a more scalable approach to planning and executing projects of varying levels of risks, complexity and monetary value. Many organizations use a scalable methodology to meet this requirement and refer to it as “Project Management Light”.
  • Business branches should provide the needed resources to conduct their own independent UAT instead of the testing being conducted by IITB.
  • As the Benefits Management Framework was developed in March 2017 and currently being piloted with several projects, and changes were made to the Quality Assurance Framework effective June 30, 2017, the success and performance of both initiatives should be evaluated in 18 to 24 months after implementation.

3.2 Project Management Knowledge Base

  • A comprehensive training and skills framework should be established that builds upon the training content and skills matrix that have already been developed. Policies or standards that would prohibit unskilled or untrained project resources from assuming responsibility and or accountability for project management functions should be put in place. It is common practice that organizations would only allow professional project managers to assume the lead role for projects of significant complexity or beyond defined financial thresholds.

3.3 Project Management Culture

  • A baseline of knowledge should be developed across management with respect to the values and core principles of project management. This could be achieved through the development and delivery of mandatory awareness training for some managers and for all Executive level positions.
  • Change management impact assessments and communications plans should be developed for all significant projects to ensure support and expediency and to minimize impacts when implementing changes.

4. Conclusion

Since the inception of the EPMO in 2014, the Department has made significant improvement in all aspects of project management. As project management continues to evolve and mature within the organization, management can anticipate more consistent results and higher performance from the portfolios of projects.

The review concludes that the Department has designed controls to manage and deliver projects on time and on budget while respecting the project scope and realizing the stated benefits. However, the following areas for improvement have been identified:

  • Even if Project Management Processes are in place, further evolution of the Project Management Knowledge Base through structured skills and competencies development is required to improve project performance.
  • To improve the consistency of project outputs and minimize deficiencies, continued evolution and focus on the Project Management Quality Assurance Framework as well as the Benefits Management Framework that were both deployed in 2017 is required.
  • Project management competencies and associated skill sets could be improved. This would support further maturity of project management as a discipline.
  • Even if the current project governance structure has been designed to bring a significant level of oversight, it could impede project progress due to the time associated with obtaining approvals, often from multiple governance committees.
  • Although there has been a noteworthy positive shift with respect to the culture of project management at the management level and above (Director), there is not a unified level of support for project management best practices and rigour.

5. Next Steps

This review assessed the design of the project management processes and knowledge base practices as well as the maturity of the project management culture. Senior Management has requested that Internal Audit Services Branch conducts a follow-up review in twelve months to assess to what extent project management processes including governance, Benefits Management Framework and the Quality Assurance Framework are implemented and operating as intended. This follow-up will also assess the progress made towards the implementation of the approved Management Action Plan.

6. Statement of Assurance

In our professional judgement, sufficient and appropriate review procedures were performed and appropriate evidence was gathered to support the accuracy of the conclusions reached and contained in this report. The conclusions were based on observations and analyses at the time of our review. The conclusions are applicable only for the period under review for the projects sampled and the related project management processes as well as knowledge and culture base practices in scope. The evidence was gathered in accordance with the Treasury Board Policy on Internal Audit and the International Standards for the Professional Practice of Internal Auditing.

Appendix A: Summary of Areas for Improvement

Application of PPMF

  1. Develop and communicate clear criteria and definitions for both “Project” and “Initiative” across the Department.
  2. Expand the boundaries of the PPMF beyond the portfolio of investment projects, to capture and govern all projects within ESDC.
  3. Modify relevant project management processes to allow for a scalable approach to planning and executing simple or low value projects.

Project Management as professional practice

  1. Develop a baseline of knowledge across managers with respect to the value and core principles of project management.
  2. Establish a comprehensive training and skills framework that builds upon the training content and skills matrix that has already been developed.
  3. Put in place controls to prohibit unskilled or untrained project resources from assuming responsibility and or accountability for project management functions.

Project Management Culture

  1. Develop change management impact assessments and communications plans for all significant projects within ESDC to ensure support and expediency, as well as to minimize impacts when implementing changes.

Appendix B: Summary of Testing Results

Several artefacts (mandatory project documentation) were required per the Project Governance Framework (PGoF) and were missing for the 26 project samples as follows:

  • 21 of the 24 had an investment proposal of which only 3 were signed and dated
  • 23 of the 25 had a business case of which 14 were signed and dated (please note that based on the stage gate of the project, not all the projects sampled required a business case per the PGoF)
  • 18 of the 18 had a project completion risk assessment completed
  • 21 of the 22 had a project charter of which 14 were signed and dated
  • 20 of the 22 had a project management plan of which 12 were signed and dated
  • 26 of the 26 had a project schedule
  • 9 of the 11 had approval to deploy forms of which 7 were signed and dated
  • 4 of the 10 had an acceptance report, 3 were signed and dated
  • 8 of the 8 had a project closure report of which 5 were signed and dated
  • 9 of the 11 had lessons learned of which 3 were signed and dated; and
  • 26 of the 26 had a costing workbook, however, they were not signed or dated

The ESDC Project Executive Dashboard Report was reviewed for the 26 projects sampled. This is a snapshot of the health of the project in terms of its financial performance, scope and timelines, including risks. Details are as follows:

  • 4 of the 26 projects misreported their financial status (2- reported green instead of yellow and 2- reported yellow instead of red)
  • 10 of the 26 projects did not report on any risks in the project dashboard which requires risk reporting; and
  • Only 2 of the 26 projects reported a change in scope, along with an estimated impact on cost and timelines.

Appendix C: Glossary

CFOB: Chief Financial Officer Branch

CMMI: Capability Maturity Model Integration

EPMO: Enterprise Project Management Office

ESDC: Employment and Social Development Canada

IITB: Innovation, Information and Technology Branch

MPIB: Major Projects and Investments Board

PGoF: Project Governance Framework

PMIS: Project Management Information Solution

PPMF: Project Program Management Framework

TB: Treasury Board

UAT: User Acceptance Test

Report a problem or mistake on this page
Please select all that apply:

Thank you for your help!

You will not receive a reply. For enquiries, contact us.

Date modified: