Integrated Revenue Collections Pre-implementation Audit

Final Report

Corporate Audit and Evaluation Branch
January 2009

Table of Contents

• Executive Summary
• Introduction
• Focus of the Audit
• Findings and Recommendations
  • Conceptual Design
  • Expected Benefits
  • Governance and Project Management
• Conclusion

Executive Summary

Background: The Integrated Revenue Collections Project (Project) is a systems development project in partnership between the Taxpayer Services and Debt Management Branch (TSDMB) and the Information Technology Branch (ITB). The Project objective is to develop technological tools, collectively referred to as Integrated Revenue Collections (IRC), to support business transformation undertaken by the TSDMB. TSDMB is responsible for the collection of all taxes, levies, duties and non-tax accounts administered by the Canada Revenue Agency. As of March 31, 2008, uncollected taxes, interest, and penalties owed to the CRA was $22.4 billion. The overarching goal of this business transformation is to move from a reactive collection focus to a proactive debt management focus, using risk profiling, enhanced data and dynamic strategies, to more effectively manage tax debt levels and maximize tax debt collections.

The Project team plans to deliver IRC through a series of phases each consisting of multiple releases. At the time of the audit, the Project was in its first phase and the next release for IRC is release two, scheduled for April 2009.

As of March 31, 2008, the Agency had spent approximately $67 million on the first phase of IRC and remaining development work for phase one is estimated to cost $38 million.

Objective: The objective of the audit was to assess whether the necessary design, governance, and project management controls were in place to support the Project in meeting its objectives.

The examination phase of the audit took place from October 2007 to July 2008, and primarily focused on work done by the Project team from April 2005 onward. The audit was conducted in accordance with the International Standards for the Professional Practice of Internal Auditing. The IT Governance Institute's Control Objectives for Information and related Technology (CObIT) and Val IT audit methodologies were used to conduct this audit.

Conclusion: There is good control over the construction of the technology changes planned for the current release of IRC (phase one release two). Overall, the Project team incorporated a number of useful controls to guide project progress and manage developmental risk in terms of on-time, on-budget implementation of functionality. Continuing the use of these controls, with some improvements, will help ensure the successful delivery of future releases and phases.

In order to fully achieve the Agency's goal of enhancing debt management through the use of enhanced information and technological tools, a mature end-state vision, including case management, for IRC as the enabler for TSDMB's business transformation will need to be implemented. This will entail the successful implementation of future phases and releases and will require the appropriate use of this new technology by operations to address previously identified risks associated with debt management. Due to the nature of the risks and the level of expenditure, it is imperative that the controls currently in place be continued, and improved based on lessons learned as future phases are implemented. In order to assist the Agency in verifying that the technology is being used as intended and providing the desired results, a post-implementation audit is recommended for the 2011-2012 fiscal year.

Introduction

The origins of the Integrated Revenue Collections Project (Project) began in 1992. The purpose of the Project was to support the integration of collections activities announced by the former Minister of National Revenue in November 1992. The Project's initial objective was to integrate revenue lines into the primary case management system, the Automated Collections and Source Deductions Enforcement System (ACSES) with the long-range goal of modifying or replacing ACSES with an improved case management system. The project was sponsored by the revenue collections area, which is currently part of the Taxpayer Services and Debt Management Branch (TSDMB), and was developed in partnership with the Information Technology Branch (ITB).

Following the Treasury Board Secretariat's Enhanced Framework for the Management of IT Projects , and consistent with generally accepted project management practice, the Project was separated into a series of phases, each phase comprised of a series of releases. The goal of a phased approach is to break down the Project into manageable portions that can be implemented individually and provide immediate benefits. This approach also reduces risk exposure if the Project were to be prematurely stopped, and allows for the Project team to build on successes and apply lessons-learned from previous phases to future phases of the Project. The initial phase (pre-business transformation) of the Project began implementation in 1995, continuing through to 1997.

TSDMB began a business transformation initiative in 1997. Evolving since its conception, the goal of this transformation was to move from a reactive accounts receivable focus (i.e. collecting taxpayer debts) to a proactive debt management focus (i.e. using risk profiling, enhanced data and strategies to more effectively manage tax debt levels). The business transformation focuses on five business directions:

• integrated service and debt management approach;
• new and enhanced technological platforms;
• legislative amendments;
• TSDMB employees; and
• strategic client relationships.

In January 2001, the objective for phase one of the Project was updated to provide technological solutions (referred to as "IRC") to act as enablers for the other components of the business transformation. The tools being developed for IRC relate to risk scoring, case management, and performance data.

A business case was initially prepared in September 2001 proposing the IRC solution for phase one. The first phase was further divided into multiple releases. Release one of the first phase, referred to as T1NB (T1 non-business) Proof of Concept, was implemented in April 2006. The purpose of this release was to implement a pilot project to test technology tools to ensure the software would interact with current systems as intended.

An updated business case was subsequently presented to Resource and Investment Management Committee (RIMC) in November 2006 to provide further details on release two. The current release (release two), referred to as T1 Integration, was in development at the time of this audit and is scheduled for implementation in April 2009.

The primary focus of IRC for the current release under development is to improve the information available to operations to manage the tax debt of individuals who do not have business income. To achieve this outcome, the Project team plans to deliver the following technologies for headquarters:

• Business rules engine;
• Data mining software; and
• Taxpayer Folder.

These tools are designed to assist in the management of taxpayer debts on which collection actions can be taken and which have not yet been assigned to collections officers in a Tax Service Office (TSO). These accounts are currently managed at the headquarters level by routine, automated processes such as computer generated letters and/or are worked by the National Collections Call Centre (NCCC). This is internally referred to as the "front-end" of the collections continuum.

In addition, the Project team is planning the following tools for collections agents and managers in TSOs:

• Taxpayer View; and
• All Revenues Table.

These enhanced inventory management tools are designed to help collection agents and managers in field offices better manage collections inventories, through more holistic client views for agents, and better information for managers to enhance decision-making on taxpayer debt accounts that have been assigned to a TSO. This is internally referred to as the "back-end" of the collection continuum.

In 2007, the Project team undertook a scope review exercise and decided to reduce the scope of the current release to focus on risk management and performance data, deferring case management and workload distribution tools to a future release. The business case was subsequently updated and presented to RIMC in September 2007.

At the time of this audit, another release (Release 3) was being considered that would implement the case management and workload distribution tools that were removed from the current release. Detailed plans for other future phases and releases were not present, although the business case for the current release did reference future phases that would implement case management and risk management tools for business and for individual non-tax accounts.

As of March 31, 2008, the Agency had spent approximately $67 million on the first phase of IRC, $28 million of which was funded prior to the establishment of RIMC and $39 million of which was funded with approval by RIMC. The remaining development work on the current release is expected to cost about $38 million.

As of March 31, 2008, total taxpayer debt including penalties and interest owed to the Agency was $22.4 billion. This amount represents the cumulative book value of all outstanding taxes owed to the CRA for all previous fiscal years. During the 2007-2008 fiscal year, Agency tax revenues exceeded $300 billion and Debt Management collected over $11.9 billion in arrears.

The Project was audited by the Office of the Auditor General (OAG) as part of its audit of Large Information Technology Projects, presented in Chapter 3 of its November 2006 Report. The audit reviewed certain project management controls, noting that improvements had been made during the course of the audit. Collection of income tax debts was also audited by the OAG and reported in Chapter 29 of its 1994 Annual Report and followed up in Chapter 8 of its 2006 Status Report. Taxpayer debt was also the subject of the Corporate Audit and Evaluation Branch's (CAEB) Review of the Management of Accounts Receivable in 2005. These reviews looked at debt management controls and noted that enhanced controls, processes, and tools were needed to better understand and manage outstanding taxpayer debt. The Project is designed to provide enhanced information and tools to address recommendations raised in these previous reports.

Focus of the Audit

The objective of the audit was to assess whether the necessary design, governance, and project management controls were in place to support the Project in meeting its objectives. This internal audit was conducted based on the following lines of enquiry:

Design
The audit assessed the planning conducted to develop the business and technological concepts and to estimate the benefits and costs to be realized from the deliverables.

Governance
The audit assessed the organizational structure and communication controls that were in place.

Project Management
The audit determined whether key project management controls were in place.

The audit looked at phase one of IRC, focusing on controls for the current release (release two), as well as the controls that would apply to future releases and phases. The examination phase of the audit took place from October 2007 to July 2008, and primarily focused on work done by the Project team since April 2005. The audit was conducted in accordance with the International Standards for the Professional Practice of Internal Auditing. The IT Governance Institute's Control Objectives for Information and related Technology (CObIT) and Val IT audit methodologies were used to conduct this audit.

Findings and Recommendations

Overall, the controls for release two of phase one of the Project were good. The conceptual design was sufficiently planned by the end of the audit to guide development of the solution for the release. Governance controls were effective in guiding the Project team in making decisions on scope, timelines, and progress. Many project management tools were in place and being refined by the end of the audit. Recommendations for improvement were minor and were often implemented during the course of the audit.

Conceptual Design

The Project team researched trends in collections and best practices in other tax agencies in managing debt. A conceptual systems design for the current phase of IRC was prepared based on this information. At the end of the documentation review period for this audit (January 2008), there was not sufficient evidence of work done on a high-level conceptual design for the overall project. However, during the reporting phase, additional evidence was available that more work on conceptual design was in progress. With the inclusion of this additional information, the high-level conceptual design was determined to be more sufficiently elaborated and documented.

However, for the current release, it would have been more helpful to have completed a high-level conceptual design (such as a solution architecture) covering all phases prior to completing detailed business requirements for the current release. Additional work on the conceptual design, particularly relating to operational processes to be affected by IRC, would have more clearly illustrated the planned path towards achieving the target vision for the project.

The Project team mapped deliverables to TSDMB action plans prepared to respond to the 2006 OAG report and tracked progress of each deliverable towards the completion of action plans. By doing this, the Project team demonstrated that IRC would generally meet the individual recommendations made in the audit report once all phases were fully implemented. Similarly, the Project team demonstrated that the deliverables would generally meet many of the TSDMB commitments made in the 2005 CAEB review of accounts receivable. For example, the Taxpayer Folder (a database tool intended to capture additional information on debt and on agent activities) should be able to collect the data necessary to analyze components of the outstanding debt and to evaluate the effectiveness of use of legislative tools.

The Project's business and information technology teams worked together in defining business requirements, researching potential solutions, and selecting technology tools that would meet business objectives for the current release. The T1NB pilot project and the non-filer pilot project demonstrated that the technology tools would meet business objectives and were compatible with existing systems. The technology tools purchased to use for IRC were subjected to stress tests and to simulations as part of pilot projects. The technology was also tested in the Agency testing environments. Situations where the technology did not perform as intended were quickly identified and resolved without affecting the production environment.

Expected Benefits

The 2007 business case states that "it is expected that the Release two functionality will generate benefits of approximately $200M per year" - $90M in accounts receivable productivity gains and $110M in additional revenues from increased compliance (e.g. data mining). The $110 million in additional revenues was based on an estimate of expected returns from non-filing taxpayers who had been considered by current systems to be low-potential accounts. Reported results from a non-filer pilot project in 2007 indicated that projected assessments were in line with the $110 million estimate reported in the business case. At the time of the audit, more work was being done by the Project team to validate the $90 million in productivity gains.

In 2007, the Project team, with assistance from the Program Evaluation Division of CAEB, developed a Results-based Management and Accountability Framework (RMAF) for the current release in order to meet Agency business case requirements. An RMAF is intended to demonstrate a rationale for the project and the objectives it intends to achieve. The Program Evaluation Division of CAEB plans to measure baseline information related to the "taxpayer view" macro, the "enhanced reporting" feature, and the risk-based strategies in the 2008-2009 fiscal year. The Program Evaluation Division is then planning to compare the baseline measurements against a post implementation follow-up in 2014.

In respect to benefits measurement related to the current release, it is important to note that some deliverables will be implemented in April 2009 (release 2.0) while others will be further implemented in October 2009 (release 2.1). TSDMB estimates that these tools will have to be used for approximately 18 months before significant differences can be detected, while the end-state benefit outcomes derived from these enhancements will not be realizable for several years. The current release represents only the start of an evolutionary process for the IRC vision and is not an end-state. In order to ensure that the project continues to focus on solutions that contribute to tangible results for improving debt management, measurable expected benefits will need to be clearly articulated for each subsequent release of IRC along with a plan to measure and report back on the realization of those benefits.

Governance and Project Management

An appropriate governance structure was in place between TSDMB and ITB. Senior management steering committees, involving representatives from both branches, provided clear direction to the Project team. The committees met regularly, were provided regular progress updates, and made decisions on significant issues.

Committees at the operational levels were also in place and working to provide day-to-day management of Project activities and progress. Evidence was found that RIMC, with the support of the Resource Management Directorate, provided a challenge function to ensure that the Project's objectives and assumptions for release two were documented.

With the creation of the Business Technology Solutions Directorate in April 2005, the Project team created a formal project management office and implemented project management tools for progress management, risk management, and resource management. Standard project controls such as business cases, risk assessments, status updates, and change and issue management were in place. Continuing the use of this formal process and continuing the enhancement of tools in use would improve the likelihood of effective IRC development in future releases and phases. Employing the use of lessons learned reviews, by the business team in addition to the IT team, would further assist the Project team in improving its processes.

Recommendations

Overall, the Project team demonstrated good use of controls in managing the current release of the Project. At the time of the audit, indications were that the Project team should be able to deliver the scope identified for the current release in the September 2007 business case, within the identified cost and time constraints.

At this point in the project, the audit does not have any recommendations for management response and action. However, the continued use and improvement of previously identified controls will be necessary in order to maximize the likelihood of the solution meeting the technology needs of TSDMB's business transformation initiative. Also, for the technology to be effective, as the project matures, TSDMB will have to develop clear plans for the operational use of the technology in order to realize the target vision for the initiative.

Given the phased, iterative, and complex nature of the project, and the fact that the current release essentially represents the start of an evolutionary process in debt management, the Agency should continue to provide strong oversight through existing mechanisms such as RIMC and OPS, to ensure the real risks of any large and costly business transformation are mitigated.

To assist management in monitoring Project progress and to provide value-added assurance services to the Agency, CAEB plans to conduct an audit two years after the implementation of the current release. A post-implementation audit will provide assurance that the controls guiding the use of the technology by operations are working as intended. The audit is tentatively scheduled for the 2011-2012 fiscal year.

Conclusion

There are two principal risks related to IRC. The first risk is the possibility that the Project team would not be able to deliver the required solution on time and on budget. Large-scale projects have an inherently high risk of failure to fully meet their objectives. The key to managing this risk, to the extent possible, is to incorporate sufficient controls into the project phases to allow management to make informed decisions on scope, progress, and cost.

Overall, the Project team incorporated a number of useful controls into the project management process to guide project progress and costs. The Project team also incorporated controls to reduce risks, such as the release one pilot project that tested technology tools used in release two. Management was able to make informed decisions on the scope of the Project, such as the decision to reduce the scope of release two. Continuing the use and improvement of these controls should improve the likelihood of successful delivery of future releases and phases.

The second risk relates to IRC's ability to meet its objective, to be the technology enabler of TSDMB's business transformation. As identified in previous audits, such as the OAG's 2006 Collection of Tax Debts and CAEB's 2005 Review of the Management of Accounts Receivable, and consistent with the experience of other tax administrations around the world, tax debt inventories within the Agency continue to grow. These reviews noted that the Agency did not have the information it needed to understand and effectively manage this growth. As part of the TSDMB business transformation initiative, IRC is intended to significantly contribute to solving this issue.

To ensure the Project stays on track through future releases and phases, controls used during the current phase of the Project will have to be continued, and in some cases enhanced where appropriate, based on future lessons learned. Project-level controls, such as high-level conceptual designs, measurable benefits, and the use of pilots as proofs of concept, will help ensure design and development work are focused on value-added solutions and that the technology is optimized by the operations area.