Audit of the Project Management Framework

December 2014

For readers interested in the PDF version, the document is available for downloading or viewing:

Audit of the Project Management Framework (PDF document - 846 KB- 24 pages)

Table of Contents

Executive summary

The focus of the audit was on the Public Health Agency of Canada's (the Agency) project management framework. The management of projects is key to providing value for money and demonstrating sound stewardship in program delivery. In June 2007, the Treasury Board ministers approved the Policy on the Management of Projects. The revised policy represents a significant change in how government manages projects.

The objective of the audit was to assess the effectiveness of the project management framework and controls to support the delivery of projects, including compliance with the Policy on the Management of Projects. The audit was conducted in accordance with the Internal Auditing Standards for the Government of Canada and the International Standards for the Professional Practice of Internal Audit. Sufficient and appropriate procedures were performed and evidence gathered to support the accuracy of the audit conclusion.

Over the past four years, the Strategic Policy, Planning and International Affairs Branch has made good progress towards improving the management of projects through a framework and governance structure that reflects the revised policy and best practices. Through the Corporate Project Management Office (CPMO), the Agency has started to receive project monitoring from a quarterly dashboard, which also helps in identifying risks that could impact the portfolio of projects. In keeping with best practices, CPMO has also developed guidance and templates to align with the project management gating process.

The current framework was found to be well-established and in compliance with the Treasury Board policy. However, the audit identified areas where the project management process should be enhanced. Specifically, the project management framework should be updated. As well, the process would be strengthened by adding an oversight reporting requirement related to project closure. CPMO should also work in partnership with the other areas that are implementing the majority of the Agency's projects and harmonize the templates that are presently in use.

Finally, while CPMO reviews the project documentation prior to gating approval, it was noted that project management plans did not always include all the required information. As well, the costing information could be improved if it was linked to the project's work breakdown structure. Lastly, CPMO should develop, monitor and regularly report to senior management on any risk items identified in the Organizational Project Management Capacity Assessment (OPMCA).

Management agrees with the four recommendations made in the audit and has prepared a management action plan that will serve to further strengthen the project management process at the Agency.

A - Introduction

1. Background

In June 2007, the Treasury Board ministers approved the Policy on the Management of Projects. This policy replaced the Project Management Policy, the Policy on the Management of Major Crown Projects and the Project Approval Policy for departments and agencies, as defined in Section 2 of the Financial Administration Act. This new policy represents a significant change in how government manages projects. As a result, the Treasury Board Secretariat (TBS) adopted a phased implementation approach, starting in 2007, which began with a group of departments that agreed to participate in a pilot. Groups of remaining departments were gradually brought onboard, so that by April 1, 2012, all departments and agencies were to have the systems and processes in place to meet the policy's requirements.

Project management is defined in the policy as the systematic planning, organizing and control of allocated resources to accomplish identified project objectives and outcomes. Project management is normally reserved for focused, non-repetitive, time-limited activities with some degree of risk and for activities beyond the usual scope of program (operational) activities.

The policy notes that the management of projects is key to providing value for money and demonstrating sound stewardship in program delivery. Moreover, a comprehensive approach to managing projects, which is integrated across the Public Health Agency of Canada (the Agency) and is appropriate for the level of project risk and complexity, will enhance the likelihood of realizing project outcomes. The approach is designed so that accountability for outcomes is clear, appropriate controls are in place to minimize risk and limit project duplication and overlap, key project stakeholders are consulted and outputs and outcomes are monitored and reported.

The policy is framed by the principles set out in the Policy Framework for the Management of Assets and Acquired Services and is supported by the Standard for Organizational Project Management Capacity and the Standard for Project Complexity and Risk. These policy instruments emphasize the people, systems and processes required to effectively manage projects. The policy allows departments the flexibility to tailor project management processes and oversight to business operations. Additionally, it aligns TBS oversight with an assessed level of project risk and a holistic understanding of the capacity of an organization to manage its planned portfolio of projects, rather than focusing solely on project costs.

The Treasury Board (TB) Policy on the Management of Projects delegates project authority to departments, commensurate with the project management capacity of the department relative to the level of project risks. This requires that the complexity and risk of each project be assessed to determine if it is within the organization's capacity to manage projects. The Organizational Project Management Capacity Assessment (OPMCA) is a 92-question self-assessment used to determine an organization's capacity to manage projects. Individual project complexity and risk is assessed using the government's Project Complexity and Risk Assessment (PCRA) Tool. This 64-question self-assessment is completed by the project manager. Both OPMCA and PCRA scores are submitted and reviewed by TBS. The project's score is then compared with the organization's capacity score to determine authority. Project PCRAs with scores that exceed the organization's OPMCA capacity score require TB approval and expenditure authority in order to proceed, while projects that are equal to or less than the organizational capacity score are likely within the approval limit and can be subject to either none or minimal TB oversight.

Agency project management

Each year, the Agency manages a range of projects to deliver priority programs and services to Canadians. The Agency has an approved Framework on Project Management and related project management templates (June 2011) to support this project management work. The Agency's standard project management processes prescribe the necessary project management approvals and governance for different project categories. It is based on categorizing projects according to dollar value as well as project risk and complexity. Typically, projects that are valued over $250K with a medium risk score can follow an approval/governing process. As the dollar value and risk increase so does the rigour of the approval/governing process.

In July 2012, the Agency's organizational project management capacity was approved at a capacity class of 2 – Tactical, on a scale from 0 to 4 (see Appendix D). At a capacity class of 2, organizations are considered to have a tactical project management capacity which includes successful delivery of projects, adjusting operations and meeting planned objectives. According to TB requirements, projects with costs exceeding $1 million require a PCRA. For the Agency, projects that obtain a score above 2 require TBS oversight; however, TBS can choose to review any project information for approval, even if it is within the Agency's authority.

As of June 30, 2014, the Agency is managing 10 projects, with a total planned cost of approximately $103 million (see Appendix C).

Figure 1 - Risk-based Project Approval and Oversight

Text equivalent for Agency's project management lifecycle

Figure 1 illustrates the Public Health Agency of Canada's risk-based project and approval framework in relation to organizational project management capacity and project complexity and risk.

The figure shows that the Public Health Agency of Canada has delegated authority for projects that are at level 1 or 2 for both organizational project management capacity and project complexity and risk.

Projects that obtain a score of more than 2 require Treasury Board Secretariat oversight, although TBS oversight can be extended to any project.

2. Audit objective

The objective of the audit was to assess the effectiveness of the project management framework and controls to support the delivery of projects and compliance with the Policy on the Management of Projects.

3. Audit scope

The scope of the audit included the Agency's project management framework and its controls for managing projects at both the portfolio and project level. Projects initiated after the effective date of the Policy on the Management of Projects (April 1, 2012) were included in the scope of the audit.

4. Audit approach

The audit examined the design of the Agency's project management framework for compliance with the Policy on the Management of Projects and the Policy on Investment Planning – Assets and Acquired Services. It also examined the governance structures, risk management practices and controls aimed at ensuring that projects are implemented successfully and in accordance with plans, including set timelines and budget. A sample of eight projects (seven that began after April 1, 2012 and one that began in January 2012) was examined to assess compliance with the organization's project management framework. A more detailed review was conducted for six of these projects, to assess the overall quality of the information contained in the project documents. Finally, the audit examined the Agency's support, methods and tools available to project managers and the effectiveness of oversight and accountability mechanisms for monitoring progress and reporting results achieved.

The audit criteria outlined in Appendix A were derived from the following sources:

  • The TB policy suite for the management of assets and acquired services;
  • the Office of the Comptroller General Internal Audit Sector's Audit Criteria Related to the Management Accountability Framework: A Tool for Internal Auditors (2011); and
  • Project Management Institute – A Guide to the Project Management Body of Knowledge (PMBoK® Guide, fifth edition).

Methodologies included a review of documentation, policies, standards, guidelines and frameworks; interviews with key personnel involved in project management; and detailed testing of a sample of projects for compliance with TB policy and the Agency's project management framework.

5. Statement of conformance

In the professional judgment of the Chief Audit Executive, sufficient and appropriate procedures were performed and evidence gathered to support the accuracy of the audit conclusion. The audit findings and conclusion are based on a comparison of the conditions that existed as of the date of the audit, against established criteria that were agreed upon with management. Further, the evidence was gathered in accordance with the Internal Auditing Standards for the Government of Canada and the International Standards for the Professional Practice of Internal Auditing. The audit conforms to the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program.

B - Findings, recommendations and management responses

1. Governance

1.1 Framework on project management

Audit criterion: The Public Health Agency of Canada's Framework on project management is documented and maintained to support accountability for projects.

The Treasury Board (TB) Policy on the Management of Projects requires deputy heads to have a department-wide governance and oversight mechanism that is documented, implemented and maintained. The mechanism is to be used to manage the initiation, planning, execution, control and closing of projects. In addition, the mechanism is to be designed so that opportunities can be considered for integrating projects across the Public Health Agency of Canada (the Agency) and the Government of Canada.

In June 2011, the Agency approved the Framework on Project Management. Projects are guided through five process groups: initiation, planning, executing, monitoring and controlling and closing. This process includes four decision gates (at the end of initiation, planning, executing and closing), where Agency management must approve the project to go forward. These project management principles are applied to the needs of any type of Agency project and are widely used to control both complex and straightforward projects. The Framework is based on guidance developed by both Treasury Board Secretariat (TBS) and the Project Management Institute (see Appendix E).

The Agency's framework is well developed and established but should be updated to reflect organizational changes that have been made since its development and to give the oversight bodies an increased role in the close-out of projects.

In the period since the Agency's framework was finalized, the Agency and Health Canada have entered into the Shared Services Partnership (SSP) agreement for the delivery of corporate administrative services. Currently, seven of the Agency's projects are delivered by an SSP branch. While the Agency chose to follow Health Canada's five gate process and accept project documentation using Health Canada's forms, both organizations would benefit from a common approach that draws on the best practices from each. This would also provide senior management with assurance that all portfolio projects are being managed similarly.

There have also been changes to the oversight committee structure since the Framework's development in 2011. New executive-level committees have been formed and old committees have been renamed or merged with one another. The framework should be updated to reflect these changes.

Finally, there is room for improvement in the oversight structure defined in the current Framework. Currently, the Executive Committee (EC) is responsible for reviewing and approving projects for both the project initiation (Gate 1) and project planning (Gate 2) phases. Oversight during project execution and project close-out falls to the sponsoring branch. The Science, Policy and Management Committee (SPMC) monitors projects through quarterly project portfolio dashboards. Given this practice, oversight committees only receive a report on the final project results and lessons learned at the discretion of EC or the sponsoring branches. Systematic sharing of close-out information may better enable senior management to refine the project management practices.

In reviewing the Framework on Project Management, the audit notes that the process is well-defined and implemented but that it should be updated to reflect organizational changes and to include a more active role for the Agency's oversight bodies in the close-out of projects.

Recommendation 1

It is recommended that the Assistant Deputy Minister of the Strategic Policy, Planning and International Affairs Branch update the Framework to reflect the changes in governance and to include a formal role for the oversight committees in managing the close-out of projects.

Management response

Management agrees with the recommendation.

The Corporate Project Management Office will update the Framework on Project Management to reflect changes in governance and will present to senior management options related to the oversight committees' role in managing the close-out of projects. Following senior management's decision, the oversight process will be modified to incorporate oversight for the close-out of projects.

1.2 Oversight committees

Audit criterion: Oversight bodies are established to govern the Agency's portfolio of projects.

The Agency has established a governance structure to oversee the management of projects. Large projects within the organization are governed by EC (chaired by the Chief Public Health Officer) and by SPMC, which recommends gating decisions to EC. Medium and small projects receive oversight from the branch assistant deputy ministers and the respective director generals.

The Executive Committee consists of the Agency's senior management team. EC is mandated to serve as a forum for guidance, leadership and final decision-making for the organization. Guided by the principles of accountability and transparency, among other issues, EC advises on horizontal issues, including approving and monitoring projects throughout their lifecycle.

The Science, Policy and Management Committee is a forum for discussion and recommendations to EC on the management and integration of the Agency's policy management and science and research activities. SPMC meets at least monthly. Membership is determined by the Chief Public Health Officer and the Associate Deputy Minister and is renewed every two years. Members are selected for their particular experience and expertise and as such, are expected to contribute to the collective management of the Agency's agenda from a corporate perspective. All SPMC decisions are subject to final ratification by EC, as appropriate. SPMC receives project planning information and based on its review, makes recommendations for final approval.

The audit reviewed the terms of reference, agendas and minutes for both EC and SPMC and found that there is an effective governance structure in place to support the oversight of the management of projects. The oversight bodies approve project baselines for scope, cost and timeline at the project initiation and project planning gates. Additionally, SPMC actively monitors project performance through a quarterly dashboard report that was initiated at the end of the last fiscal year. This report also gives SPMC an overview of how the organization is performing in terms of its portfolio of projects.

Overall, appropriate oversight bodies have been established to govern the Agency's portfolio of projects. The oversight bodies meet regularly and meaningfully address project management issues.

1.3 Roles and responsibilities

Audit criterion: Project management roles and responsibilities are clearly defined.

Project management is a shared responsibility between the Corporate Project Management Office (CPMO) in the Strategic Policy, Planning and International Affairs Branch and the individual project's sponsoring branch.

CPMO is located in the Governance, Planning and Reporting Directorate. CPMO is the centre of excellence for project management within the Agency. It is responsible for increasing project management capacity and knowledge across the Agency by providing guidance, consultation and training to project sponsors, project managers and project teams. It also establishes and maintains the Agency's Framework on project management and its supporting tools and templates. CPMO staff coordinates the gate approval process, ensuring that the oversight committees receive the information they require to make sound project management and investment planning decisions. They provide advice and support to project managers, review project gate documentation and provide a challenge function prior to gate reviews by the oversight committees. Finally, they monitor and report on project performance and conduct an Organizational Project Management Capacity Assessment (OPMCA) on a regular basis.

Project managers are responsible for achieving the defined project objectives within the approved cost, timeline and resources. They are responsible for the day-to-day management of the project, including the project team, schedule, budget, stakeholders, risks and issues. They manage the project throughout all stages, including planning, execution and close-out. Depending on the complexity of the project, different major contributors may each have a project manager.

Specific project manager responsibilities are defined in the project charter, as they may vary from project to project. The audit notes that all projects sampled had a project charter and project management plan that included a section outlining the roles and responsibilities of the project manager.

The roles and responsibilities for CPMO and branch project managers are clearly defined.

2. Risk management

2.1 Portfolio project risk management

Audit criterion: Portfolio project management risks are identified, assessed, mitigated and monitored.

Portfolio risk management

One of the most effective tools available for assessing risk at the portfolio level is the OPMCA. The assessment provides the Agency with a high-level assessment of its capacity to manage projects. The assessment is based on the risks associated with the management of projects and when completed, it identifies areas of strength and weakness in an organization's project management practices. In 2012, the Agency completed its first OPMCA and was approved at a capacity class of 2 on a scale of 0 to 4. This capacity class is acceptable to senior management, given the types of projects that the Agency undertakes, and it accurately represents a project management capacity that will meet the Agency's requirements.

The other benefit of the OPMCA is that it identifies areas potentially requiring improvement and the possible risks to managing the portfolio of projects. The Framework on project management outlines CPMO's responsibility to conduct regular OPMCAs and based on those results, to provide recommended action plans to the Agency's senior management. The Framework on project management was designed to manage portfolio risks through the gating process. This process minimizes the financial and resource risks to the organization, as only well-planned, strategic business cases are approved.

Regular dashboard reporting by the individual projects is another way that CPMO is able to mitigate project portfolio risk. The quarterly dashboard not only highlights issues with individual projects, but can also identify trends that could impact the Agency's portfolio of projects.

Project risk management

As described in the background section of this report, a Project Complexity and Risk Assessment (PCRA) is required for all projects over $1 million. The PCRA tool identifies project risks and is used to determine whether the project is within the organization's assessed project management capacity. In the sample selected, all projects had a PCRA score of 2 or lower and were within the Agency's assessed capacity.

CPMO has taken additional steps to manage risks at the project level by requiring each project to prepare a risk management plan as part of the project management plan that is a Gate 2 requirement. Project managers must also maintain a risk log throughout the execution phase of the project and review and update the risk register on an ongoing basis.

In conclusion, CPMO has implemented a number of measures and controls to mitigate the risks to the Agency's portfolio of projects. These include the implementation of the gating process for the approval of projects, along with the creation of the quarterly monitoring dashboard and the completion of the OPMCA. However, CPMO would benefit from using the risk results of the OPMCA to further strengthen the Agency's project management capacity.

Recommendation 2

It is recommended that the Assistant Deputy Minister of the Strategic Policy, Planning and International Affairs Branch use the results from the Organizational Project Management Capacity Assessment to further strengthen the project management process.

Management response

Management agrees with the recommendation.

The Corporate Project Management Office will revise the guides and tools to strengthen content, using the results from the Organizational Project Management Capacity Assessment.

3. Internal controls

3.1 Guidance and supporting tools

Audit criterion: Guidance and supporting tools are developed and up to date to support project management.

Guidance and supporting tools are important in ensuring that project managers understand and comply with organizational expectations for sound project management and meet the requirements at each gate of the process.

The audit found that the gating process is well defined and documented. Expectations for each gate are clearly defined and templates are provided to guide project managers in the development of key documentation. CPMO is active in supporting project managers in completing these templates and in assisting with the preparation for gate reviews by the oversight bodies. It provides project managers with expert advice and consultation on any aspect of project management, from procurement planning to risk management and mitigation and change management, functioning as a key internal resource to help bridge project management skills, knowledge and capacity gaps across the Agency.

The templates themselves are comprehensive and generally self-explanatory. They guide project managers through an effective planning process that is consistent with the guidance provided by both TBS and the Project Management Institute. In addition to the Agency's set of templates, it was noted that Health Canada's Real Property and Security Directorate and Information Management Services Directorate also developed project management templates. In order to manage the different templates for joint projects and increase efficiencies, the Agency has made a decision to use the Health Canada templates. Although all templates were developed with a similar intent–to introduce rigour and consistency in the project management process–they were developed independently of each other and often require the same information to be presented in a different manner. The Agency, in conjunction with SSP, would benefit from updating the templates to ensure a common level of project management practice while accommodating any unique project management needs between the two organisations and their branches.

The audit assessed the adequacy of the guidance and tools provided by CPMO and found both to support project management capacity, but noted some areas for improvement in the supporting tools.

Recommendation 3

It is recommended that the Assistant Deputy Minister of the Strategic Policy, Planning and International Affairs (in partnership) maintain a set of minimum content and documentation requirements for all aspects of the Framework across all areas of project management.

Management response

Management agrees with the recommendation.

The Strategic Policy, Planning and International Affairs Branch will undertake this deliverable, in consultation with Health Canada and other key internal stakeholders, to identify a set of minimum content and documentation requirements for all aspects of the Framework and will update the templates accordingly.

3.2 Project initiation and planning

Audit criterion: Projects are initiated and planned.

The gating process is designed to ensure that projects are worthy of investment and are aligned with Agency priorities, and that sufficient planning, stakeholder engagement and organizational commitment are in place to provide the necessary resources to implement the project.

Overall, projects were generally compliant with the requirements of project initiation (Gate 1), but improvement is required in project costing and certain other elements of project planning (Gate 2).

Gate 1 – Project Initiation

The project initiation stage is focused on ensuring that there is sufficient business justification and organizational support for the project before further organizational resources are committed to detailed planning. Key project deliverables for this stage include the project charter, business case and a PCRA. TBS approval may also be required if the risk score exceeds the Agency's project authority.

The audit sampled eight projects and found that they were generally compliant with Gate 1 requirements. The majority of the projects reviewed (with only a few minor exceptions) had appropriately completed a detailed project charter, business case and PCRA.

Gate 2 – Project Planning

During the project planning stage, project management and technical activities are defined, resourced and sequenced, and the proper management processes are put in place to successfully deliver the product and/or service. This project phase also establishes the official baselines against which the project will be measured. The key project deliverable at this stage is the project management plan, which defines how and when a project's objectives are to be achieved by showing the major products, milestones, activities and resources required. The PCRA is also refined at this stage, as more is known about the project's parameters. All planning should be completed before passing this gate, as the next phase is focused on implementing the project plan.

Based on the sample of projects selected, the analysis found that while all the required documentation was prepared, the quality of information varied by project. More specifically, there should be better information related to project costing, scheduling, change control, quality control, human resources and communication plans. While each of these is important, project costing is one of the most important project management activities to ensure that projects are delivered within the cost expectations documented in the project management plan. Cost can be affected by many factors, but some of the more common pressures are scope and scheduling changes (either acceleration or slowdowns) and estimating errors.

Project costing is the responsibility of the sponsoring branch. It identifies major products, milestones, activities and the related resources required. It is also expected that this information be maintained over the course of the project lifecycle. To support this activity, CPMO provides guidance on the level of detail and accuracy of the costing information required at each gate, and further supports the costing activity with comprehensive templates. As well, stewardship of the project gate review process is further strengthened, since CMPO is to provide an independent verification that gate requirements have been satisfied.

Despite having a well-designed process with the appropriate gating controls, the audit found that there was a range of maturity in the costing information provided as part of the gating and approval processes. Costing information varied by project and was often not sufficiently detailed, such that cost estimates were not tied or assigned to specific project activities or work packages, making it difficult to understand how they were calculated or when they would be incurred. More detailed costing information would better enable project managers and others to more accurately forecast and monitor project progress and financial performance.

The detailed examination also found that in two of the six projects reviewed in detail, the project schedules should have provided more detail related to tasks, critical path and milestones to support effective project monitoring. The same projects also did not have adequate change control plans to define how project changes would be identified, recorded, approved and actioned. Although interviews with project managers indicated that changes were discussed at branch level oversight committees, these processes were not formally documented and it was unclear when or if significant changes would be brought to the Agency's oversight bodies for approval.

Finally, for a majority of the projects sampled, the project management plans did not include adequate human resource, quality or communications plans. Interviews with project managers indicated that many of these plans had been considered informally but were not documented as required by the Project Management Framework. As these plans comprise important elements for an effective planning process, it is important that they be included.

Since April 2014, CPMO has been conducting a more formal review of the project documentation prior to presenting it to the oversight committees. Although this analysis entails expert judgement of the information by CPMO staff, it does not necessarily ensure that the information required in the templates is complete.

Recommendation 4

It is recommended that the Assistant Deputy Minister of the Strategic Policy, Planning and International Affairs:

  • work with sponsoring branches to ensure that project documentation is more closely aligned with the requirements outlined in the Agency's Framework on Project Management; and
  • work with the Chief Financial Officer to develop options to improve project costing.

Management response

Management agrees with the recommendation.

The Corporate Project Management Office will develop and implement a control tool to assess the information required in project management plans.

In partnership with the Office of the Chief Financial Officer, the Corporate Project Management Office will update the Framework on Project Management to include standard costing templates.

3.3 Project execution

Audit criterion: Projects are completed in terms of authorized time, budget and scope.

Project execution is when the project plans are put into action, with the purpose of achieving its expected results within the planned time and resources. Project managers are tasked with executing the plan, monitoring progress against the plan and identifying and addressing any issues or unforeseen events as they arise.

A total of eight projects were reviewed (one has been completed, six are in the execution phase and one is in the planning phase) to determine the degree to which they were on scope, on time and on budget relative to the plan, as approved at Gate 2. All but one project self-reported as being on scope, on time and on budget, but the audit was unable to verify these assertions. As discussed in other sections of this report, projects did not always report against the milestones or costs approved in the project management plans, making it difficult to determine their performance against the initial plan. Further, the deficiencies in project costing discussed earlier meant that cost targets were not always representative of the true project costs, and meaningful milestone-based cost targets were not available.

In conclusion, while project managers reported their projects as generally being on scope, on time, and on budget, the audit was unable to verify these assertions due to deficiencies in project plans and inconsistencies in project reporting (addressed in Recommendation 4).

3.4 Project closure

Audit criterion: Projects are closed out.

Project closure is the process of ensuring that all the necessary activities have been performed, so that the project can be officially terminated within the organization. Assessing the project's success and identifying lessons learned are two of the most important activities of this stage. Determining whether the project was successful in achieving its goals within the expected time and cost and identifying what worked and what did not are important elements that contribute to the continuous improvement of the Agency's project management practices. The key project document required at this gate is the project close-out report.

As discussed earlier, the current Framework does not require EC approval prior to closure. Providing the oversight committees with this type of information, including any lessons learned, would assist them in decision making for future projects and provide them with summary data on projects they had approved (addressed in Recommendation 1).

Of the six projects examined in detail, only one had reached the project closure phase at the time of the audit. Therefore, based on the lack of a reasonable sample of projects to review, it is not possible to accurately assess this criterion. However, to ensure the continuous improvement of project management practices, the Agency should ensure that project close- out activities for all projects are conducted promptly following project execution. The results should then be reported to the Agency's project oversight bodies so that they are kept apprised of project results and opportunities for improvement.

3.5 Project monitoring and controlling

Audit criterion: Projects are monitored and controlled, and change requests are managed.

To successfully deliver on a project’s objectives within the approved time, cost and scope, project managers must continuously monitor its performance and carefully manage change requests. While oversight bodies also have a responsibility to monitor project performance and manage change (discussed in section 1.3), this section focuses on monitoring and change control at the project manager level.

Monitoring

In March 2014, CPMO introduced its quarterly dashboard reports to SPMC for the ongoing monitoring of projects. Beginning in September 2014, this report will also be tabled at EC on a quarterly basis. The dashboard uses red, yellow and green indicators to highlight areas of concern for each active project in terms of schedule, cost and scope, and includes a short narrative for each project, highlighting any issues that could be impacting it. The information contained in the dashboard is supported by more detailed and comprehensive data submitted to CPMO by project managers. The detailed templates developed by CPMO contain sections to capture information on the project's costs, key milestones and deliverables, risks, project issues and an overall assessment of the project. The detailed reports generally captured the appropriate information to support effective project management and in interviews, project managers reported that the templates are effective tools for monitoring their projects. However, there were many instances where the budget and milestones did not match those identified in the approved project management plan. This made it difficult to objectively assess the project status relatively to its approved plan.

Change control

Change control is an essential aspect of any project management process, as most projects will encounter some type of change as they proceed from the planning phase to close-out. Unapproved changes can often result in cost or schedule overruns or projects that do not meet the needs of the business owner. In order to address this issue, project managers are required to describe how they will manage project changes.

Once a project receives Gate 2 approval, it is up to the branch to ensure that changes are being properly logged and approved. In our sample review of individual projects, the audit found that branches had developed processes to review and approve changes made at the project level. Effective August 2014, significant changes requiring a rebaselining of the project's schedule, cost or scope are reported to the oversight committees in the quarterly dashboards.

Overall, the audit found adequate controls for project monitoring and change control at the project manager level.

C - Conclusion

The Framework on project management adopted by the Public Health Agency of Canada (the Agency) in 2011 has been successful in putting more structure and accountability into the management of projects. However, moderate improvements can still be made to address weaknesses identified during the audit.

The Agency's Framework establishes clear roles and responsibilities for project management and supports generally effective oversight and project management practices. However, it requires updating to reflect organizational changes since 2012, including the Shared Services Partnership and changes to the Agency's oversight committee structure. Further, a more active role for the oversight bodies in the project close-out phases would ensure that senior management is receiving the information required to effectively manage the Agency's portfolio of projects and would continuously improve the organization's project management practices.

The Corporate Project Management Office (CPMO) recently established effective reporting practices to ensure that the oversight bodies receive the information they need to achieve their mandate and to support project managers in their day to day management. Agency level risks are also being managed, although a more formal assessment and mitigation of risks identified in the Organizational Project Management Capacity Assessment (OPMCA) would further strengthen this area.

Finally, the organization has established a good balance of internal controls to manage and oversee the management of projects. Although these controls are functioning as planned in most cases, CPMO should work with the branches to improve the costing of projects and ensure that gating requirements are being fulfilled. The templates must also reflect the present needs of the Agency's branches and projects that fall under the Shared Services Partnership agreement.

Appendix A - Lines of enquiry and criteria

Audit of Project Management Framework
Criteria Title Audit Criteria
Line of Enquiry 1: Governance
1.1 Framework on project management The Public Health Agency of Canada's Framework on project management is documented and maintained to support accountability for projects.
1.2  Oversight committees Oversight bodies are established to govern the Agency's portfolio of projects.
1.3 Roles and responsibilities Project management roles and responsibilities are clearly defined.
Line of Enquiry 2: Risk Management
2.1  Portfolio project risk management Project management roles and responsibilities are clearly defined.
Line of Enquiry 3: Internal Controls
3.1 Guidance and supporting tools Guidance and supporting tools are developed and up to date to support project management.
3.2 Project initiation and planning Projects are initiated and planned.
3.3  Project execution Projects are completed in terms of authorized time, budget and scope.
3.4  Project closure Projects are closed out.
3.5  Project monitoring and controlling Projects are monitored and controlled, and change requests are managed.

Appendix B - Scorecard

Scorecard –Audit of Project Management Framework
Criterion Rating Conclusion Rec #
Governance
1.1 Framework on project management Needs minor improvement  The Framework on Project Management requires revision to address projects related to the Shared Services Partnership and to actively involve oversight bodies in project close-out. 1
1.2 Oversight committees Satisfactory Oversight bodies meet regularly and meaningfully address project management issues.  
1.3 Roles and responsibilities Satisfactory Project management roles and responsibilities are clearly defined.
Risk Management
2.1 Portfolio project risk management Needs minor improvement  Portfolio level risk management should be improved by addressing areas of risk identified in the Organizational Project Management Capacity Assessment. 2
Internal Controls
3.1 Guidance and supporting tools Needs minor improvement  The templates are comprehensive and support good project management practices. However, the templates should be harmonized and updated to align with Framework updates. 3
3.2 Project initiation and planning Gate 1 - Satisfactory Good compliance at the project initiation phase.  
Gate 2 – Needs moderate improvement Project management plans should include all of the required documentation. More detailed costing information should be linked to work packages. 4
3.3 Project execution Needs moderate improvement  More detailed project schedules and costing is required in order to be better able to meaningfully assess project performance in mid-stream. 4
3.4 Project closure Needs minor improvement  Close out activities should be conducted promptly for all projects following project execution. 1
3.5 Project monitoring and controlling Needs Minor Improvement Recent changes to the project management dashboard have improved the monitoring and change control for the Agency's projects.  

Appendix C - Summary of the Agency's projects

As of June 30, 2014

  Name Description Project Phase Estimated Cost ($,000)
1 JC Wilt Lab (Winnipeg Lab Space Optimization) Construction of a level 2 lab in Winnipeg, Manitoba. Close-out $68,560
2 Energy Management Controls System Replace all building automation controls at the Canadian Science Centre for Human and Animal Health. Execution $5,121
3 130 Colonnade Workplace 2.0 Optimization Implement Workplace 2.0. Execution $3,849
4 100 Colonnade Workplace 2.0 Optimization Implement Workplace 2.0. Close-out $4,746
5 391 York Accommodation Project  Implement Workplace 2.0. Execution $1,347
6 Single Window Initiative Canada Border Services Agency-lead, with 7 other federal departments and agencies participating in the establishment of an integrated solution for the importation of products. Execution $5,090
7 Email Transformation Initiative The project involves Shared Services Canada implementing a common email solution. Execution $335
8 Information Management (IM) Readiness IM Readiness provides the preparatory recordkeeping information management and electronic file identification work needed for the Public Health Agency of Canada (the Agency) to be compliant with the Treasury Board 2015 Directive on Recordkeeping. Planning $448
9 Windows 7 OS Enterprise Upgrade The project involves the replacement of Health Canada and the Agency’s desktop infrastructure (the operating system) that will no longer be fully supported by its vendor after April 2014. Execution $1,121
10 ISTOP – It solution that support the Human Pathogens and Toxins Act Implementation Note. The project involves the tasks and activities required to implement a new regulatory framework, as set out in the new Human Pathogens and Toxins Act (2009). Execution $12,394 
      Total  $103,011 

Note – The Treasury Board submission supporting the implementation of the Human Pathogens #and Toxins Act (2009) included a total cost of $34,160,000, the information systems implementation portion of the total was $12,394,000.

Appendix D - Definitions of complexity and risk

Project Complexity and Risk Level Rating Definitions
1. Sustaining Project has low risk and complexity. The project outcome affects only a specific service or at most a specific program, and risk mitigations for general project risks are in place. The project does not consume a significant percentage of departmental or agency resources.
2. Tactical A project rated at this level affects multiple services within a program and may involve more significant procurement activities. It may involve some information management or information technology (IM/IT) or engineering activities. The project risk profile may indicate that some risks could have serious impacts, requiring carefully planned responses. The scope of a tactical project is operational in nature and delivers new capabilities within limits.
3. Evolutionary As indicated by the name, projects within this level of complexity and risk introduce change, new capabilities and may have a fairly extensive scope. Disciplined skills are required to successfully manage evolutionary projects. Scope frequently spans programs and may affect one or two other departments or agencies. There may be substantial change to business process, internal staff, external clients, and technology infrastructure. IM/IT components may represent a significant proportion of total project activity.
4. Transformational At this level, projects require extensive capabilities and may have a dramatic impact on the organization and potentially other organizations. Horizontal (i.e. multi-departmental, multi-agency, or multi-jurisdictional) projects are transformational in nature. Risks associated with these projects often have serious consequences, such as restructuring the organization, change in senior management, and/or loss of public reputation.

Source: Project Complexity and Risk Assessment Tool

Appendix E - Description of the Agency's project management lifecycle

 

Appendix E
Text equivalent for Agency's project management lifecycle

The image illustrates the Public Health Agency of Canada's Project Management Lifecycle.

A series of boxes are separated into three rows entitled: "Stages", "Key Deliverables" and "Approvals/Gates".

The first row entitled "Stages" contains the five stages of the project management lifecycle: "Initiating", "Planning", "Executing" "Monitoring and Control" and "Closure".

The second row entitled "Key Deliverables" illustrates what key deliverables relate with which stage of the project management lifecycle. The key deliverables relating to the "Initiating" stage are the following: initial risk assessment, concept document, business case, project charter and project complexity and risk assessment. The key deliverable relating to the "Planning" phase is the project management plan which includes the budget and the schedule. The key deliverable relating to the "Executing" and "Monitoring & Controls" phases is the status report which includes the project risk and project change requests. Finally, the key deliverable relating to the "Closure" phase is the close-out report.

The last row entitled "Approvals/Gates" demonstrates the three levels of approval that occur during each stage of the project management lifecycle. These are: "Go/No Go Approvals", "Oversight and Reporting" and "Project Acceptance". The "Initiating" and "Planning" phases fall under the "Go/No Go Approval" level. The "Executing" and "Monitoring & Control" phases fall under the "Oversight and Reporting" level. Finally, the "Closure" phase falls under the "Project Acceptance" level.

It should be noted that a PCRA is completed only for large projects that are part of the Investment Plan, in collaboration with CPMO.

Page details

Date modified: