Audit of the Management Control Framework for the Canada 150 Federal Secretariat

December 2016

Cette publication est également disponible en français.
© Her Majesty the Queen in Right of Canada 2017
Cat. No. CH6-56/2017E-PDF
ISBN 978-0-660-07082-7

Table of contents

• Executive summary
• 1.0 Background
• 2.0 About the audit
  • 2.1 Project authority
  • 2.2 Objective and scope
  • 2.3 Approach and methodology
• 3.0 Findings and recommendations
  • 3.1 Planning
  • 3.2 Reporting on Canada 150 activities
    • 3.2.1  Reporting requirements
    • 3.2.2 Data collection and analysis
  • 3.3 Risk management
  • 3.4 Canada 150 project selection, approval and monitoring
    • 3.4.1  Project selection and approval
    • 3.4.2 Project monitoring
• 4.0 Conclusion
• Appendix A – Audit criteria
• Appendix B – Management action plan

2.1 Project authority

The authority for this audit is derived from the Multi-Year Risk-Based Audit Plan for 2016-2017 to 2018-2019 that was recommended by the Departmental Audit Committee and approved by the Deputy Minister in April 2016.

2.2 Objective and scope

The objective of this audit is to provide assurance on the effectiveness of the management control framework for Canada 150 Federal Secretariat and the appropriate management of the Canada 150 Fund.

The engagement included all Canada 150 activities covering fiscal year 2015-2016 to the substantial completion of audit work in September 2016. The audit is future oriented and aims to support management in implementing a reasonable level of controls to ensure the successful delivery of the Initiative as the Canada 150 activities are ongoing with the expectation to be completed by the end of 2017.

2.3 Approach and methodology

The audit was conducted in accordance with the Treasury Board Policy on Internal Audit and the Internal Auditing Standards for the Government of Canada.

The audit methodology included:

• reviewing the organization’s documentation, guidelines and procedures, Celebration and Commemoration Program Terms and Conditions, policies and legislation relevant to the organization;
• collecting data through interviews and observations to examine processes, procedures and practices;
• conducting walkthroughs of key controls and processes to demonstrate the organization structure, processes and accountability obligations;
• reviewing and testing program files; and
• analyzing financial and non-financial information.

As most of the Canada 150 Fund is delivered through the Celebration and Commemoration Program (CCP), the audit team worked with the knowledge, observations, and conclusions developed in the April 2016 Audit of the Celebration and Commemoration Program. As such, efficiencies were gained, duplication of work was avoided, and the impact on the CCP program staff was limited.

3.0 Findings and recommendations

The audit findings are based on a combination of the evidence gathered through the audit methodologies applied for each criteria. Appendix A provides a summary of all findings and conclusions for each of the criteria assessed during the audit. The findings of lesser materiality, risk or impact have been communicated with the auditee either verbally or in the management letter.

3.1 Planning

The purpose of project planning is to act as a tool to provide value for money and demonstrate sound stewardship in program delivery. A comprehensive approach to program planning may increase the likelihood of project delivery as it considers project complexity, appropriate controls, risk assessment and risk responses. Therefore, the development of an integrated plan would increase the probability that Canada 150 objectives would be met.

The audit team reviewed the Treasury Board policy and standards on project management and industry standards on project planning. The audit team also interviewed management staff within the Federal Secretariat to determine whether an integrated plan is in place for the delivery of Canada 150 and examined existing planning tools that are used by the Federal Secretariat.

Specific planning mechanisms are in place within the Federal Secretariat to ensure delivery of the Canada 150 objectives. These include a governance framework which outlines the objectives, coordinating committees, and the performance measurement framework for Canada 150. Roles and responsibilities have been documented for key stakeholders in the delivery of Canada 150 for the Federal Secretariat, Canadian Heritage programs, and other federal institutions. Other planning tools have been developed for the implementation of Canada 150 such as marketing and outreach strategies.

While specific planning mechanisms enable the Federal Secretariat to manage its day to day operations, integrated planning supports strategic decision-making by offering a broader view of resources, risks and deliverables to achieve objectives in a timely manner. It was noted that there was an absence of an integrated plan. Having an integrated plan would allow the Federal Secretariat to build upon its current planning mechanisms and to develop a systematic framework which facilitates planning across the Secretariat to ensure timely delivery of the Initiative.  Without an integrated plan, the Federal Secretariat faced challenges such as unclear priorities, duplicate roles, responsibilities and work, resource planning, and reduced efficiency in delivery of the Canada 150 activities.

Recommendation:

1. It is recommended the senior executive director of Canada 150 Federal Secretariat develop an integrated plan that will further support and monitor the delivery, performance, and project management of Canada 150 activities.

3.2 Reporting on Canada 150 activities

3.2.1 Reporting requirements

Clear reporting requirements facilitates the ability to meet accountability obligations to deliver and provide oversight of activities.

The Federal Secretariat was established to implement a pan-governmental approach and provide oversight for Canada 150 and is responsible for the reporting of Canada 150 activities undertaken by all federal institutions.  Requirements for reporting must be clear in order for both Canadian Heritage programs and federal institutions to provide accurate results on Canada 150 activities.

The audit team reviewed the Canada 150 governance framework to determine the reporting requirements for Canadian Heritage programs and other federal institutions. In particular, the team examined the performance measurement framework developed by the Federal Secretariat to report on progress to internal and external stakeholders and whether Canada 150 objectives have been reached.

The team found that the performance measurement framework includes clear reporting requirements for Canadian Heritage programs and other federal institutions. The performance measurement framework is a three-tier structure which include strategies to assess if the outcomes and objectives of Canada 150 have been met and the extent to which the Federal Secretariat has ensured a whole of government approach to the delivery of activities.

3.2.2 Data collection and analysis

Collecting and analyzing information on Canada 150 activities is necessary to facilitate decision-making and to provide accurate information and results on the delivery of Canada 150.

The audit team reviewed the information collected and analyzed by the Federal Secretariat that is used to monitor the delivery of Canada 150 activities. In addition, the audit team reviewed the tools that are used by the Federal Secretariat to report on the results of Canada 150.

The team found that the Federal Secretariat is in the early phase of collecting and analyzing information for Canada 150 projects from other federal institutions and Canadian Heritage programs. The Federal Secretariat has developed a web-based tool for federal institutions to collect information on each Canada 150 project and includes information which is aligned with the requirements of the performance measurement framework. The Federal Secretariat has access to information from Canadian Heritage programs as it can use the information management system used to administer grants and contributions.  Canada 150 results are reported externally through the corporate reporting mechanisms, including a dedicated horizontal table in the departmental Report on Plans and Priorities.

An area of improvement was noted for internal reporting on Canada 150 activities to the Department’s senior management. The Federal Secretariat has not developed a tool or dashboard to provide senior management with information on the progress of Canada 150 activities. Developing a tool or dashboard would facilitate information dissemination, decision-making, and highlight areas where increased management attention should be directed. In addition, such a consistent tool would provide senior management with the necessary information reducing the reliance on verbal briefings and facilitate trends and change analysis.

Recommendation:

2. It is recommended the Senior Executive Director of the Canada 150 Federal Secretariat develop a tool to facilitate internal reporting to senior management on the delivery of Canada 150 activities.

3.3 Risk management

Risk management involves a systematic approach to setting the best course of action by identifying, assessing, mitigating, and communicating risk issues. Risk management facilitates decision-making and prepares senior management to develop strategies in order to respond proactively. 

The audit team gathered evidence and held interviews with staff from the Federal Secretariat and Canadian Heritage programs to determine whether risks to the delivery of Canada 150 were assessed, identified, mitigated, and monitored. 

The team found that the Federal Secretariat has identified key risks to the delivery of Canada 150. There were risk identification sessions held for the Federal Secretariat and Canadian Heritage Programs between September 2015 and January 2016. During these sessions, risks were identified and were ranked according to impact and likelihood. Summary reports were prepared on identified risks and recommended areas where mitigation strategies should be focused. In addition, a risk assessment session was held during the summer 2016 with other governmental departments to obtain their views on the key risks to the delivery of Canada 150.

Appropriate mitigation and monitoring strategies with assigned accountabilities will improve the ability of the Federal Secretariat to identify controls to address risks and take a proactive approach in delivering Canada 150. The audit team noted that although key risks were identified, the Secretariat has not sufficiently demonstrated how these risks will be mitigated and continuously monitored.

Recommendation:

3. It is recommended the Senior Executive Director of the Canada 150 Federal Secretariat strengthen the risk management practices, in particular mitigation and monitoring of identified risks.

3.4 Canada 150 project selection, approval and monitoring

3.4.1 Project selection and approval

The objective of the Treasury Board Policy on Transfer Payments is to ensure that transfer payments are managed in a manner that is sensitive to risks and address government priorities. The Department needs to ensure that projects are only recommended for funding that meet objectives and are approved according to a risk-based approach. With Canada 150 being a government priority and having time-sensitive milestones, there is a heightened importance on due diligence and ensuring there are as little delays as possible to the project approval process.

The audit team examined 47 (36%) grants and contributions out of 132 approved projects (as of August 2016) sampled among Major Events, Signature Initiatives, and Community projects. Documentation in the Department’s Grants and Contribution Information Management System was reviewed and interviews were held with program management and staff who provided further insight into the understanding of the approved projects.

Through file review and testing, it was found that all approved sampled projects met Canada 150 objectives. For each project, an assessment was conducted by program staff which consisted of a set of eight questions with a score on how well the applicant and project met the objectives of Canada 150. These questions also included the capacity of the recipient and whether the proposed approach would allow for the successful timely delivery of the project.

Approved Canada 150 Fund projects may receive up to 100% of funding for eligible expenses. The audit team found that there was no correlation between the percentage of funding received by recipients from Canadian Heritage and project risk assessments. Out of the 47 projects reviewed, 25 of the 34 low risk projects (74%) received 50% or more of project funding from Canada 150.

The majority of the approved projects were assessed as low risk. Table 2 provides the distribution of risk ratings for approved Canada 150 projects.

Figure 1. Applicants risk level (from total approved projects*)

Opportunities for improvement was noted for timely approval of Canada 150 projects and signing of agreements. Understanding the delays in announcing the Canada 150 initiative, the subsequent establishment of the Federal Secretariat, and the effects of the 2015 federal election, there is heightened importance to ensuring that projects are approved in a timely manner.  Delays in project approvals may impact on delivery of Canada 150 objectives as it may lead to application withdrawals, re-scope of proposed applications, and impediment to projects’ capacity to meet expected outcomes.

The audit team found that half of the contribution agreements in the sample did not have a signed agreement in place as of August 2016. For contribution agreements which had been signed, it took an average of 54 days from the date the project was approved to a signed contribution agreement. In order for recipients to receive their funding in time to deliver projects and to avoid continuing delays, the timely signing of contribution agreements becomes increasingly important.

Recommendation:

4. a.) It is recommended the Senior Executive Director of the Canada 150 Federal Secretariat facilitate the timely approval of projects.
   b.) It is recommended the Director General, Major Events, Commemorations and Capital Experience facilitate the timely signing of agreements. 

3.4.2 Project monitoring

An effective approach to monitoring contribution agreements ensures that projects are delivered according to milestones and deliverables included in the signed agreements and that objectives are met. Insufficient monitoring and reporting increases the likelihood that early detection of projects risks won’t be identified in advance of a payment request which may lead to the potential misuse of public funds.

The audit team found that the current practice is to include monitoring and reporting requirements in the contribution and grant agreements. These include activity reports, cash flows, and audited financial statements. All funded recipients are also required to submit a final activity report once their project is complete. Additional monitoring efforts (e.g., increased client contact and/or site visits) are based on the level of risk identified.

It was noted that there is no strategic, risk based plan to monitor the delivery and performance of the Canada 150 projects throughout the country. The majority of these projects will be delivered in 2017 with the average approved amount being above $780,000. In addition, 25 out of the 34 (74%) low risk sampled projects received 50% or more of project funding from Canada 150, which represents a significant contribution to project funding and is not captured in the risk assessments. Given the significance and complexity of Canada 150, there is a need to have a strategic approach to monitor these projects.

Having a strategic monitoring plan will facilitate the allocation of resources, increase the likelihood of projects meeting milestones and deliverables and ensuring public funds are used as intended. 

Recommendation:

5. It is recommended the Senior Executive Director, Federal Secretariat in consultation with Director General, Major Events, Commemorations and Capital Experience, develop a strategic monitoring plan which considers the number of projects and their respective risks, timelines and available resources.

4.0 Conclusion

The objective of the audit was to provide assurance on the effectiveness of the management control framework for Canada 150 Federal Secretariat and the appropriate management of the Canada 150 Fund.  Based on the audit observations, it was found that there are effective elements of management control framework for the Canada 150 Federal Secretariat and the management of the Canada 150 Fund. Areas of improvement were noted for integrated planning, reporting to senior management, mitigating and monitoring risks, project approvals and signing of agreements, and developing a strategic approach to monitoring Canada 150 projects. Management’s response and action plans are deemed to be sufficient to address the recommendations found in the report.

Appendix A – Audit criteria

(The X indicates the areas of coverage for the identified issues.)

Appendix B – Management action plan