Quality Assessment of CRA Internal Audit

Corporate Audit and Evaluation Branch
July 2006

• Introduction
• Objective
• Scope
• Methodology
• Reporting
• Findings and Action Plans
  • A) Attribute Standards
    • Due Professional Care
    • Independence
    • Continuing Professional Development
    • Internal Assessments
  • B) Performance Standards
    • Reporting to the Board and Senior Management
    • Planning Considerations
    • Quality of Communications
  • C) IIA Code of Ethics
• Conclusion
• Appendix A - Assessment Rating
• Appendix B - Principal Elements of the Independend Validator's Role
• Appendix C - Independent Validation Statement

Introduction

In January of 2002, the Institute of Internal Audit (IIA) published new/upgraded standards reflective of the changing role of internal audit. The Office of the Comptroller General (OCG) of the Government of Canada adopted these new standards.

The new standards require the establishment of a quality assurance and improvement program for the IA activity. More specifically, Standard 1312 requires that an external assessment be performed at least once every five years to obtain an objective appraisal of the activities' compliance with the IIA standards and to improve the organization's operations. This Standard, the first assessment for which must be completed by January 1, 2007, can be met through either an external review, or a self-assessment confirmed by independent validation.

To meet the Standard, CAEB proceeded with a self-assessment for independent validation. This process was initiated with an internal risk assessment based on the IIA standards that resulted in identifying areas where improvements were required, and establishing action plans to address all issues. The implementation of those action plans has been closely monitored and their present status is reflected in the results reported here. The independent validation statement is attached as Appendix C.

Objective

The objective of the self-assessment was to provide assurance to the CRA's Board of Management (BoM) and Internal Audit and Program Evaluation Committee (IAPEC) that the IA activity of CAEB is conforming to the established professional standards as published in the IIA's “The Professional Practices Framework”.

Scope

The self-assessment was based on, and covered, the IIA's International Standards for the Professional Practice of Internal Auditing, and the CRA's Code of Ethics. Assessment of the IA activity was based on CAEB's existing policies and procedures and the results of quality assurance work performed during the 2005-2006 year. It also included monitoring the progress of actions plans established following an internal risk assessment exercise.

CAEB did not assess the elements of the standards related to CRA senior management or committees. The independent IIA certified quality assessment consultant assessed these elements as part of the external validation.

Methodology

The audit steps and criteria for the self-assessment were based on the IIA's Quality Assessment Manual (4^th edition) and CAEB's own quality assurance audit programs. The Corporate Services and Professional Practices teams prepared and/or utilized:

• results of quality reviews of completed audit products;
• analyses of results of informal quality reviews on audit deliverables;
• results of client satisfaction surveys;
• reviews of policies and other documentation (internal and external);
• interviews and discussion with CAEB managers and staff; and
• benchmarking where information was readily available.

The external consultant validated the self-assessment outputs based on the statement of work detailed in the attached Appendix B (external assessment).

Reporting

This report was prepared for the IAPEC and Audit Committee of BoM on the results of CAEB's self-assessment of its' compliance with the standards, and outlines any action plans for improvement.

The external validator reviewed the self-assessment work, interviewed members of Agency senior management and the Audit Committee of BoM, and prepared a report for Agency senior management (Appendix C) on the CRA's level of compliance with the standards. That report concurs with this reports conclusion of “generally conforms”. The validator's observations have been incorporated into the findings and management actions plans.

Findings, recommendations and action plans

Overall, CAEB's Internal Audit activity “generally conforms” to the Professional Standards as set by the Institute of Internal Auditors. This is the highest rating achievable and supports the views of peers in other Canadian government organizations that the CRA IA activity is a leader in its field. In the IIA lexicon, “generally conforms” means that an internal audit activity has a charter, policies, and processes that are judged to be in accordance with the Standards, with some opportunities for improvement.

The standards are subdivided into three main categories: Attribute, Performance and IIA Code of Ethics. The results of each category are detailed below.

A) Attribute Standards

The attribute standards detail four main areas: Purpose, authority, and responsibility; independence and objectivity; proficiency and due professional care; and quality assurance/improvement program.

The IA Policy and Audit Committee Charter meet all the major elements of the IIA's model. The IA Policy is currently under revision for a fall 2006 approval. The quality assurance program being used is a best practice and has been used by other government institutions in the design of their own programs. Our assessment determined that internal audit generally conformed to all individual attribute standards within these four categories, with the exception of a partially complies for due professional care. There are also recommendations related to organizational independence, continuing professional development and internal assessments.

Due Professional Care

Internal auditors should apply the care and skill expected of a reasonably prudent and competent internal auditor by considering the extent of work to achieve engagement objectives; the relative complexity, materiality or significance of matters to which assurance procedures are applied; the adequacy and effectiveness of risk management, control and governance processes; the probability of significant errors; and the cost of providing assurance relative to potential benefits. Ongoing quality review and quality assurance activities in the first half of the 2005-2006 fiscal year showed that additional work was required in the planning phase of engagements to better understand the area under review, better assess risks and ensure that objectives were meaningful and attainable within resources available.

As a result, several actions were taken in 2005-2006 to improve this situation. Information on improving planning phase work was communicated to all staff. Program profiles are now required for almost all audits and increased review of audit programs is in effect. The Quality Review Steering Committee reinforces these measures through detailed and strategic questioning.

An advanced training course was piloted and is being evaluated. Staff has been strongly encouraged and are being supported in the achievement of professional audit designations to further strengthen capabilities.

In addition, early this fiscal year, a Professional Practices Division was established and has been staffed to provide further guidance and support. Results to date from these initiatives have been positive.

Recommendation

The actions listed above should be continued and evaluated to ensure compliance with the due professional care standard.

Action Plan

The Internal Audit Directors, and the managers of Corporate Services and Professional Practices will continue to provide the DG CAEB with quarterly updates on the status of detailed actions as noted above. In addition, Professional Practices will perform quality assurance reviews in the fall of 2006 and provide the DG CAEB with an assessment of compliance with this standard as at December 31, 2006.

Independence

The IIA standards require that the Chief Audit Executive (CAE) report to a level within the organization that allows the IA activity to fulfill its responsibilities. The key principle in meeting this standard is that the reporting relationship be such that the independence, objectivity and competence of the CAE are clearly established and recognized throughout the organization and are supported by the structure.

The IIA's literature on this topic and information collected through benchmarking speaks to three different organizational models. The IIA's ideal model has functional reporting of the CAE to the Audit Committee and administrative reporting to the CEO. In the other two models the CAE reports for all purposes to the CEO or to another member of senior management, most frequently the CFO. Our benchmarking showed that all three models are being used effectively. The IIA recognizes in practice advisory 1110-2, that other reporting relationships (outside their ideal model) can be effective if there are clear distinctions between the functional and administrative reporting lines.

The CRA model is reflective of a reporting relationship as described in IIA practice advisory 1110-2 that is recognized in the CRA as being effective. The DG CAEB reports both administratively and functionally to the Commissioner on all matters and functionally to the Audit Committee of BoM on matters that fall within the ambit of its responsibility as described in the CRA Act. The DG CAEB further demonstrates independence through in-camera meetings with the Audit Committee of BoM, which are initiated by both parties.

There are two bodies that govern IA work in the CRA, the IAPEC and the Audit Committee of BoM. The title of the IAPEC does not clearly distinguish it from the Audit Committee of the BoM. This has the potential to create confusion to both internal and external stakeholders.

The roles and responsibilities of the IAPEC are described in the IA policy, including the review and approval of both governance documents and final reports. The responsibilities and duties of the Audit Committee of BoM are described in its' Charter, including the approval of IA governance documents and review of final reports that fall within the authority of BoM. This reflects the unique governance structure of the Agency. However, members of the Audit Committee of BoM have recently initiated discussions on the extent of their responsibilities and duties. Our assessment confirms the need for further discussion in this regard.

Recommendations

The word “Management” should be added to the title of IAPEC to better distinguish its role from the Audit Committee of BoM.

Discussions should be held with the Commissioner and BoM to confirm roles and responsibilities.

Action Plan

The word “Management” will be added to the title of the IAPEC.

The DG CAEB will initiate discussions with the Commissioner and the Audit Committee of BoM, for decision by December 2006, to confirm the roles and responsibilities of the respective committees.

Continuing Professional Development

The IA activity strongly supports continuing professional development through both human and financial resource means. In addition, all staff prepare an individual learning plan as part of their annual performance appraisal exercise. In the 2005-2006 year, particular support and participation was evident in second language training. The activities in this regard could be enhanced through the development of longer term (3-5 year) staff development plans detailing training activities, work assignments, language training, etc. that would meet both personal and workplace goals and needs. More effort in this area would also facilitate better succession planning, an area where staff surveys noted a weakness.

Recommendation

The IA activity should prepare long-term (3-5 year) staff development plans to define training activities, work assignments and language training. These plans should form the base of a more structured and documented succession plan.

Action Plan

Learning Profiles detailing potential career paths and required skills were released on the CAEB InfoZone site on June 19, 2006. Feedback over the next 6 months on the use of this tool will be combined with other tools and further research into staff development plans by Professional Practices Division, to further define the scope of formal long-term staff development plans. This work will start immediately and will be implemented starting this fall. Full implementation will be reflected in the 2007-2008 performance appraisal cycle (September 2007).

Branch Management Committee members will use this information and the results of performance appraisal to devise and document a succession plan during this same time frame.

Internal Assessments

Internal assessments include the ongoing review of quality through supervision. While the audit work reviewed was of good quality, the evidence on file in most cases to show the extent of supervision was not sufficient. The IA activity has responded to that in the interim by enforcing the use of tools intended for that purpose.

CAEB has purchased a state-of-the-art electronic working paper product known as TeamMate and is in the process of having this product certified for use, expecting a fall 2006 pilot and full implementation by April 2007. This product is in use by both public and private sector internal audit activities around the world and will fully meet the standards for evidence of supervisory review.

B) Performance Standards

The Performance Standards detail seven main areas: managing the internal audit activity; nature of work; engagement planning; performing the engagement; communicating results; monitoring progress; and management's acceptance of risk.

The performance standards detail the expectations of the CAE in managing the day-to- day activities of internal audit to add value to the organization. These include preparing a risk-based plan, performing proper risk assessment, managing resources efficiently and effectively, providing manuals and instructions and having timely and effective communications.

The IA activity has a number of excellent tools in support of the performance standards. There is a thorough manual, detailing all aspects of the IA activity. A number of templates are in use to ensure consistency and quality. An in-house introductory training course covers all key topics and is delivered to all new staff. An enhanced course has been piloted and is being assessed.

A follow-up process is in place that meets the standards but that the IA activity is reconsidering to improve the rigor and timeliness of the process and the accountability of management.

Our assessment determined that internal audit generally conformed to all individual standards within these seven categories with the exception of partially complying with the reporting to the Board and senior management sub component of the managing the IA activity standard, the planning considerations sub component of the engagement planning standard, and the quality of communications sub component of the communicating results standard.

Reporting to the Board and Senior Management

The standards require that the CAE report to the Board and senior management on the status of IA's performance relative to the annual business plan. The word report is used in a very broad context here to include the IA charter/policy, IA annual business plan, progress to the annual business plan, Human Resource issues, budgets and actuals, timeliness, significant risks, exposures and control issues, corporate governance issues and any other matters requested by the board.

Internal Audit reports to the IAPEC and the Audit Committee of BoM in a number of ways. The IA Policy is presented to both bodies and is reviewed at least every three years. The IA Business Plan and Annual Report are also discussed with both committees. The Annual Report speaks to the use of financial resources as well as providing examples of the value added by the IA function in assisting the Agency in obtaining its objectives. In addition, it includes comments on progress achieved related to key activities aimed at improving IA effectiveness. In year, quarterly progress to business plan reports are provided to the Audit Committee of BoM, detailing the status of items in the plan and all Assignment Planning Memoranda and Final Reports are presented to IAPEC. Annual summary follow-up reports detail the number and status of action plans.

While many performance items are presented to the audit committees, they are somewhat piecemeal and not all potential categories are included. For instance, detailed information on budget to actual timelines is not provided at this time nor has any information on staff skills and qualifications formally communicated. Summaries of Client Satisfaction surveys have not been communicated to the committees.

Recommendation

Internal Audit should continue to refine the performance information provided to the IAPEC and the Audit Committee of BoM.

Action Plan

Internal Audit will review the performance information provided to the committees and adjust as determined. Benchmarking will occur where possible and the DG CAEB will discuss with the chairs of the committees to determine if their needs are being met. Decisions taken on performance information requirements will be progressively implemented starting in the 2006-2007 fiscal year.

Planning Considerations

Planning considerations should include the following: gaining an understanding of the entity to be audited including its objectives, system of control and significant risks; developing engagement objectives and scope that are reflective of the risks identified; determining client expectations and respective responsibilities.

Ongoing quality review and quality assurance activities in the first half of the 2005-2006 fiscal year showed that additional work was required in the planning phase of engagements to better understand the area under review, better assess risks and ensure that objectives were meaningful and attainable within resources available.

This finding is directly linked to the due professional care standard discussed earlier. Actions already underway, recommendations and action plans are one and the same and are not repeated here.

Quality of Communications

The standards require that communications be accurate, objective, clear, concise, constructive, complete and timely.

A best practice noted during the assessment of the quality of communications is the publishing of IA reports on the CRA external website. This practice fully supports the government-wide objective of public reporting and full transparency.

Based on our self-assessment work all components are met with the exception of timeliness.

This standard speaks to the existence and effectiveness of tools to monitor the timeliness of engagements. CAEB IA does have the tools and variances are explained, however, timeliness of reporting the final results has been an issue in some cases. Significant analysis and a timeliness study completed in the fall of 2005 resulted in recommendations and action plans that are now being implemented. While it is too early to conclude if the actions have been fully successful, timeliness has improved and remains a priority.

Comments were received during the self-assessment related to the level of communication with management in the CRA on the roles and responsibilities of Internal Audit, Program Evaluation and the external auditor (OAG). It was felt that better informing management of the roles and responsibilities of these three groups would facilitate the work of all these groups.

Recommendations

The effectiveness of the timeliness study action plans should continue to be monitored.

CAEB should develop a strategy and prepare and distribute the necessary communication products to improve CRA management's understanding of the roles and responsibilities of Internal Audit, Program Evaluation and the external auditor (OAG).

Action Plans

The effectiveness of the timeliness study action plans will continue to be monitored through the DG's status report and the quality assurance program. In addition, timeliness expectations have been or will be built into Director, Account Manager and Team Leader performance expectations during their next cycle. Professional Practices will monitor timeliness and provide the DG with a status report on compliance with the standard as of December 31, 2006.

Communication to management of the roles of IA, PE and the external auditor is the responsibility of all IA staff. All staff will be reminded immediately of the expectation for them to take every opportunity to explain the roles of these three groups. To facilitate this, Professional Practices will add an explanation to the CAEB InfoZone site this summer. The MG champion for CAEB and the Professional Practices Division will work jointly with the CRA MG Focal Point group and Management Development Program representatives to determine the most effective means to communicate review function roles to the management community. The targeted completion date will be March 31, 2007.

C) IIA Code Of Ethics

The IA activity of the CRA generally conforms with this standard. The standard consists of four main elements: integrity, objectivity, confidentiality and competency. These are consistent with the CRA's values and are ingrained through corporate culture and the CRA's Code of Ethics. While it is not a requirement of the standard, many IA organizations have their staff sign a conflict of interest statement on an annual basis. This serves to have staff read the statement and remind themselves of their professional responsibilities in this regard.

Recommendation

It is recommended that internal audit staff sign a conflict of interest statement on an annual basis.

Action Plan

Effective for the employee performance cycle beginning September 1, 2006, all managers and staff will be required to sign the conflict of interest statement as part of the annual performance expectations process.

Conclusion

Based on the risk assessment work completed in the 2005-2006 fiscal year including the resultant actions from that assessment, the activity's ongoing quality assurance program and the self-assessment work recently completed, the CRA's Internal Audit activity generally conforms with the IIA's international standards for internal audit, their highest rating for an IA activity. In areas where improvements could further increase the level of compliance, internal audit has already taken actions or has identified plans for implementation.

Appendix A - Assessment Rating

GC - "Generally Conforms"
PC - "Partially Conforms"
DNC - "Does Not Conform"

General Guidelines for Evaluation of Conformity to the Standards and Code of Ethics:

Consider each individual Standard, including the relevant Implementation Standards (which give additional guidance on assurance and consulting services), and conclude as to the degree of conformity by the activity to each one.

Consider each section of the Standards and conclude as to the degree of conformity by the activity to each section taken as a whole, based on conclusions reached for the related individual standards in the section and on other relevant observations made during the Quality Assessment.

On the same basis as for sections of the Standards, conclude as to the degree of conformity by the activity to the major categories of the Standards; then make an overall evaluation as to the activity's conformity to the Standards as a whole.

Consider the four principles and related rules of conduct in the Code of Ethics and conclude whether or not the activity's management and staff uphold each of the principles and apply the related rules of conduct.

GC — “Generally Conforms” means the evaluator has concluded that the relevant structures, policies, and procedures of the activity, as well as the processes by which they are applied, comply with the requirements of the individual Standard or element of the Code of Ethics in all material respects. For the sections and major categories, this means that there is general conformity to a majority of the individual Standards or elements of the Code of Ethics, and partial conformity to the others, within the section/category. There may be significant opportunities for improvement, but these should not represent situations where the activity has not implemented the Standards or the Code of Ethics, is not applying them effectively, or is not achieving their stated objectives.

PC — “Partially Conforms” means the evaluator has concluded that the activity is making good faith efforts to comply with the requirements of the individual Standard or element of the Code of Ethics, section, or major category, but has fallen short of achieving some of their major objectives. These will usually represent some significant opportunities for improvement in effectively applying the Standards or Code of Ethics and/or achieving their objectives. Some of the deficiencies may be beyond the control of the activity and may result in recommendations to senior management or the board of the organization.

DNC — “Does Not Conform” means the evaluator has concluded that the activity is not aware of, is not making good-faith efforts to comply with, or is failing to achieve many/all of the objectives of the individual Standard or element of the Code of Ethics, section, or major category. These deficiencies will usually have a significant negative impact on the activity's effectiveness and its potential to add value to the organization. They may also represent significant opportunities for improvement, including actions by senior management or the board.

Appendix B

The principal elements of the Independent Validator's role included:

• Review the self-study and other advance materials and note items for follow-up on site visits;
• Identify potential best practices and opportunities for improvement;
• On-site, review the documentation of the self assessment;
• Perform limited tests, by reference to source documents, the self assessment work and the validity of the information submitted;
• Review the evaluation summary regarding conformity to the Standards and discuss the basis of conclusions with the Project Leader and Director General (DG);
• Conduct brief interviews with the head of the Agency, the Chair of the Audit Committee, other appropriate senior officials and one or two clients of Corporate Audit services;
• Review the DG's communication with the Audit Committee on the results of the self-assessment;
• Prepare a memorandum to serve as the basis of a closing briefing with the self assessment team and the DG; and
• Complete the formal validation report for transmittal to the audit committee by January 2007.

Appendix C

[Appendix C is also available in a pdf version.]

Independent Validation Statement

I, David Rattray, FCGA, CIA, an Associate Partner with the Centre for Public Management Inc., Ottawa, Ontario, was engaged to conduct an independent validation of Canada Revenue Agency's (CRA) Internal Audit Division's self-assessment process and report. The primary objective of the validation was to verify the assertions made in the attached self-assessment report, concerning adequate fulfillment of the Agencys' basic expectations of the internal audit activity and its conformity to The Institute of Internal Auditor's (IIA) Standards for the Professional Practice of Internal Auditing (Standards). Other matters that might have been covered in the requirements for a full independent assessment, such as an in-depth analysis of best practices, governance, consulting services, and use of advanced technology, were excluded (as per IIA practice) from the scope of this independent validation by agreement with the Director General Audit and Evaluation (CAE).

In acting as validator, in a capacity independent of the organization, I have the necessary knowledge and skills to undertake this engagement. The validation, conducted during the period May and June, 2006 consisted primarily of a review and testing of the procedures and results of the internal audit self-assessment. In addition, I reviewed CRA supplied documentation, conducted audit working paper file reviews, carried out internal audit staff focus groups and conducted structured interviews with the Commissioner, the Chief Financial Officer and Assistant Commissioner of F&A, the Assistant Commissioner Atlantic, the Director General Audit and Evaluation (CAE) as well as the past and current Board of Management Audit Committee Chairs.

Any observations or recommendations which I have made as a result of this validation assignment have been discussed with the Chief Audit Executive and the CRA Self Assessment Team and have been incorporated into the attached Self-Assessment Report.

I concur fully with CRA Internal Audit's conclusions in the self-assessment report attached.

__________________________________

Original Signed by:
David Rattray, FCGA, CIA
Independent Validator
Centre for Public Management Inc.