Audit of Horizontal Case Management

Final Report

Corporate Audit and Evaluation Branch
April 2009


Table of Contents

Executive Summary

Background: Workload management systems, often called case management systems, were developed to enhance the Agency’s ability to deliver its core business by providing the tools necessary to effectively capture, store, and share information. Timely and detailed information from case management systems allow local and headquarters management to monitor transactions, track program progress, and find ways to improve program efficiency and effectiveness.

The Canada Revenue Agency (CRA) currently uses 18 case management systems to manage workloads across its various programs. Most of these systems were designed at a time when the standard industry practice was to develop single entity, “stove-piped” solutions. While this method was effective in maintaining control and accountability in a single program area, management has since recognized the value of integrating business intelligence from multiple program areas and aligning systems to take full advantage of program interdependencies.

Case management systems that are integrated at the enterprise level (i.e. horizontal case management systems) are able to achieve degrees of horizontality in both business and technological dimensions:

Although technological horizontality offers potential efficiency gains and cost savings for CRA, business horizontality is the key to enhancing the Agency’s capacity for program excellence. The Val IT™ ISACA governance framework states that IT investments should be driven by business needs, rather than the technology itself. Therefore, while technology is important to achieving horizontality, it is only an enabler for businesses to become more horizontal, not a solution on its own.

The Corporate Case system was developed in the late 1990s and was the Agency’s first step towards achieving technological horizontality. It is used in a number of program areas, such as Canada Pension Plan and Employment Insurance (CPP/EI) Rulings, Standardized Accounting (SA), and T2 Assessing, and interacts/coexists with legacy systems to provide case management functions, such as inventory tracking, file transfers, and a notepad.

Globus, one of the newer iterations of Corporate Case, was the next evolutionary step towards achieving technological horizontality, and also represented the Agency’s first attempt at creating a case management system with a focus on achieving business horizontality. Globus was developed in 2001 as a joint effort between the Appeals, Assessment and Benefit Services (ABS), and Information Technology (IT) branches. According to the Project Charter, the system was envisioned to be a common platform that could serve different workloads from multiple program areas.

Objective: The objective of this audit was to determine whether an appropriate and effective governance structure was in place to support the Globus system, and that controls were in place to ensure that the system design met business needs.

For this report, Globus will serve as a case study to identify opportunities for improvement and lessons learned to assist senior management in the development of future horizontal case management systems.

Conclusion: The audit found that Globus made progress in advancing technological horizontality, and provided a relatively low-cost solution that is generally meeting program-specific needs of the two areas that originally implemented it (i.e. Appeals and T1/Benefits in ABSB). However, Globus did not meet its original vision of becoming an enterprise-wide case management system. Since April 2007, Globus has evolved into two separate service areas called “Appeals” and “T1/Benefits” in order to better accommodate program needs specific to the Appeals and ABS branches.

The audit found that in the absence of certain key strategic controls, such as a documented strategy, to drive an appropriate level of horizontality, there was a tendency for managers to optimize systems to meet their own program needs. In order to ensure a focus on the strategic objective of increasing the horizontality in case management systems within CRA, governance controls that encourage horizontal relationships and activities need to be enhanced.

A focus on governance and strategic controls is particularly relevant as the Agency is moving forward with plans to provide new horizontal case management functionality for its Compliance Systems Redesign (CSR) project. This new case management functionality, which was originally called the Enterprise Work Management (EWM) system, will use an integrated IT architecture system and intends to achieve business and technological horizontality within Compliance Program Branch (CPB) by connecting over 40 programs. Key IT components and functionalities that are being built to achieve this horizontality within CPB are eventually expected to be leveraged and reusable for other workloads within the Agency.

The Case Governance Steering Committee (CGSC) was established to ensure that the Agency’s case management systems were developed in a horizontal manner. However, the CGSC's activities were not tightly coordinated or supported by a detailed business strategy. There was also a lack of clarity among key stakeholders and senior management over the CGSC’s roles and responsibilities, and a formal reporting structure was not in place to help resolve these issues.

Overall, the CRA continues to make progress in developing horizontal case management systems. The evolution from the initial Corporate Case systems to Globus (and eventually to the new case management functionality used in CSR) revealed incremental advancements in technological horizontality. In addition, the lessons learned on Globus helped contribute to the new case management vision, which is articulated in the IT Service and Case Management Strategy.

To help realize its vision for horizontal case management systems, the Agency should ensure that the following strategic controls are in place:

Action Plan: All recommendations within this report have been agreed with by the CGSC and appropriate action plans have been submitted. These action plans include:

The CGSC, in conjunction with ITB, will develop a planning framework to assist program branches identify their horizontal case business requirements. The planning framework will be developed by Fall 2009, and be implemented over the course of 2009-2010.

The Appeals Branch and ABSB, in conjunction with ITB, will produce preliminary multi-year plans for their respective Service Areas of Globus, to be reviewed by the CGSC at its September 2009 meeting. Other program branches that use Corporate Case (LPRAB, CPB, and the remaining program areas within ABSB), in conjunction with ITB will produce preliminary multi-year plans for their respective Service Areas of Corporate Case. ITB agrees to review the plans against the IT Service and Case Management Strategy, with CGSC input, and to provide its recommendations to the business owners.

The CGSC will update its terms of reference to ensure that it is aligned with all audit recommendations, and present this update to all CGSC members by September 2009.

The CGSC, in collaboration with Finance and Administration (F&A), will ensure reviews of all case-related proposals to be submitted to OPC and RIMC. The basis for reviews will be the IT Service and Case Management Strategy and the Multi-Year Plans.

The CGSC agrees to provide a forum for horizontal discussion on the definition of the common data fields that are needed to meet horizontal case requirements that involve stakeholders as needed (e.g. Data Stewardship Working Group, Corporate Information Management Steering Committee, Corporate Audit and Evaluation Branch).

Introduction

Workload management systems, often called case[Footnote 1] management systems, were developed to enhance the Agency’s ability to deliver its core business by providing the tools necessary to effectively capture, store, and share information. Timely and detailed information from case management systems allow local and headquarters management to monitor transactions, track program progress, and find ways to improve program efficiency and effectiveness.

The Canada Revenue Agency (CRA) currently uses 18 case management systems to manage workloads across its various programs. Most of these systems were designed at a time when the standard industry practice was to develop single entity, “stove-piped” solutions. While this method was effective in maintaining control and accountability in a single program area, management has since recognized the value of integrating business intelligence from multiple program areas and aligning systems to take full advantage of program interdependencies.

Case management systems that are integrated at the enterprise level (i.e. horizontal case management systems) are able to achieve degrees of horizontality in both business and technological dimensions:

The Corporate Case system was developed in the late 1990s and was the Agency’s first step towards achieving technological horizontality. It is used in a number of program areas, such as Canada Pension Plan and Employment Insurance (CPP/EI) Rulings, Standardized Accounting (SA), and T2 Assessing, and interacts/coexists with legacy systems to provide case management functions, such as inventory tracking, file transfers, and a notepad.

The concept behind Corporate Case was to develop a generic, centrally managed case management system that could be customized to meet the needs of various business areas. Up to eighty percent of the program code in each of the iterations of Corporate Case is common. This common code made it possible to develop new case management systems without having to start from “scratch.” Since 1998, eight different iterations of Corporate Case have been created in this manner, including SA Case, T2 Case, and Globus.

Globus, one of the newer iterations of Corporate Case, was the next evolutionary step towards achieving technological horizontality, and also represented the Agency’s first attempt at creating a case management system with a significant focus on achieving business horizontality. Globus was developed in 2001 as a joint effort between the Appeals, Assessment and Benefit Services (ABS), and Information Technology (IT) branches. According to the Project Charter, the system was envisioned to be a common platform that could serve different workloads from multiple program areas.

The Agency is now moving forward with plans to develop new horizontal case management functionality for its Compliance Systems Redesign (CSR) project. This new case management functionality, which was originally called the Enterprise Work Management (EWM) system, will use a modern integrated IT architecture system and intends to achieve business and technological horizontality within CPB by connecting over 40 programs. The IT components and functionalities that are being built to achieve this horizontality within CPB are eventually expected to be leveraged and reusable for other workloads within the Agency.

Focus of the Audit

The objective of this audit was to determine whether an appropriate and effective governance structure was in place to support the Globus Case System, and that controls were in place to ensure that the system design met business needs. As such, this audit included a review of overall governance strategies, committees and other elements.

Audit examinations were conducted in the Appeals Branch, ABSB, and ITB in Headquarters, as well as selected offices in the Pacific, Prairie, and Ontario regions from July to December 2007.

For this report, Globus will serve as a case study to identify opportunities for improvement and lessons learned to assist senior management in the development of future horizontal case management systems.

The audit was conducted using a combination of the following methodologies:

The audit was conducted in accordance with the International Standards for the Professional Practice of Internal Auditing and Information Systems Audit and Control Association (ISACA).

Findings, Recommendations and Action Plans

1.0 Governance

CRA’s Project Management Policy and Resource and Investment Management Committee (RIMC) Administration Procedures and Document Guidelines require that all CRA projects have a governance structure in accordance with best practices of project management. In the context of case management, it includes, but is not limited to, an approved strategy for horizontal case management systems and an established decision-making body with appropriate representation, authority, and support.

1.1 Strategy

1.1.1 CRA Case Management Strategy

A review of documentation and interviews with key stakeholders identified that the Agency does have a vision to improve horizontal management. The audit also identified a number of documents that contained elements to transform the vision into more concrete terms and objectives. These include the Globus Project Charter (2002); Future Directions Interim Report (2004); Future Case - Concept of Operations (2004); EWM Business Case (2006); Future Case Study Report (2006); ITB Architecture (2007); and the IT Service and Agency Case Management Strategy (2007).

The Case Governance Steering Committee (CGSC) was established in 2003 with a mandate to “oversee the development of the Agency's various case management systems, both current and future.” Since the start of this audit, CGSC has taken steps towards transforming its vision for horizontal case management systems into a formal strategy. CGSC was involved in establishing the IT Service and Agency Case Management Strategy, which ITB presented to the Operations Committee in November of 2007. This strategy intends to establish IT architectural standards that can be used to build new program applications and serves as the foundation for technological horizontality.

Although technological horizontality offers potential efficiency gains and cost savings for CRA, business horizontality is the key to enhancing the Agency’s capacity for program excellence. The Val IT™ ISACA governance framework states that IT investments should be driven by business needs, rather than the technology itself. Therefore, while technology is important to achieving horizontality, it is only an enabler for businesses to become more horizontal, not a solution on its own.

Most of CGSC’s efforts to date have been directed towards addressing the technology challenges. As a result, key issues relating to business horizontality, such as assessing the need for horizontality between program areas, establishing the necessary linkages across business lines, clarifying roles and responsibilities, and agreeing on standard terminology for common business processes have not been addressed.

Recommendation

The CGSC should establish a planning framework to assist program branches identify and prioritize their short, medium, and long term horizontal business needs with respect to case management functionality. The planning framework should ensure that these business needs are documented in a form that is compatible with the IT Service and Case Management Strategy.

Action Plan

The CGSC, in conjunction with ITB, will develop a planning framework to assist program branches identify their horizontal case business requirements. The planning framework will be developed by Fall 2009, and be implemented over the course of 2009-2010.

1.1.2 Globus Development Plan

The 2002 Globus Project Charter (draft) included plans and processes to identify key business requirements, but only for the program areas initially involved. Implicitly, the strategy for Globus was to focus on the Appeals and ABS branches first, and then expand at a later date. Subsequent interim reports and updated strategies identified challenges with respect to horizontal business requirements, such as establishing standard definitions for common data elements and keeping business rules separate from the core rules. However, there was no evidence of any follow up activity to resolve these issues.

The benefits expected from Globus included: improving processing times by reducing non-value added activities; improved client service through a unified taxpayer view and enhanced inventory tracking; providing a single source of statistical information to enhance managers' ability to achieve service standards; and providing a common user interface to reduce training costs.

As differences between the Appeals and ABSB’s T1/Benefits programs became evident, the two program areas expressed a need for specific functionality. This led to requests for and the implementation of changes that were less generic and more program specific, eventually resulting in Globus being split into two separate service areas: one for Appeals programs and one for T1/Benefits programs.

According to Globus project documentation and interviews with key stakeholders, the decision to move certain program areas to the Globus system was based primarily on short term needs, such as an immediate need for a new case management system or enhancements to existing case management systems. The audit did not find any plans that took into consideration the program areas that would benefit most from sharing a horizontal case management system.

The four program areas that initially migrated to Globus (i.e. Appeals, Supplementary Assessing System (SAS), Taxpayer Requested Reassessment (CRR), and Disability Tax Credit (DTC)) were not dependent on each other with respect to inputs and outputs. As a result, there were very few opportunities to assess the horizontal capabilities of the system. Selecting interdependent and horizontally linked program areas (e.g. SAS has strong links with T1 Error Correction) to test a horizontal case management system may have forced design conflicts to be resolved prior to implementation and provided a test of the costs and benefits.

The Globus experience demonstrates that a well-defined strategy to address business horizontality should be in place to ensure short-term needs do not take precedence over the long-term vision. This would ensure that design choices are carefully considered before systems are built, rather than afterwards.

Recommendations

The Appeals Branch and ABSB should produce multi-year plans to guide decisions on future developments for their respective Globus service areas, including updated features for these systems.

The CGSC and ITB should review these plans to ensure consistency with the IT Service and Case Management Strategy, as well as the planning framework recommended above.

Action Plans

The Appeals Branch and ABSB, in conjunction with ITB, will produce preliminary multi-year plans for their respective Service Areas of Globus, to be reviewed by the CGSC at its September 2009 meeting.

Other program branches that use Corporate Case (LPRAB, CPB, and the remaining program areas within ABSB), in conjunction with ITB, will produce preliminary multi-year plans for their respective Service Areas of Corporate Case.

ITB agrees to review the plans against the IT Service and Case Management Strategy, with CGSC input, and to provide its recommendations to the business owners.

1.2 Organization, Roles and Responsibilities

1.2.1 CRA Case Management Coordination

The authority for development and maintenance decisions relating to case management systems rests within the individual program branches. Accountability is determined based on which branch has ownership over the affected workload or which branch controls the necessary resources.

There is no formal mechanism in place to clearly assign accountability over shared assets such as horizontal case management systems; however, the CRA does have structures in place to address issues that are horizontal in nature. Agency-level committees, such as the Operations and Strategic Directions committees, provide a forum to share information and discuss issues that affect multiple stakeholders at the strategic level. There are also inter-branch committees and working groups in place to manage horizontal issues at the tactical level, both between program areas and across functional boundaries. The CGSC is an example of a tactical-level, cross-functional committee that has a specific mandate to oversee the development of the Agency’s case management systems.

The CGSC is co-chaired by the Director General, Revenue and Accounting Systems Directorate (RASD) of ITB and the Director General, Research, Risk and Business Management Directorate of the Compliance Programs Branch (CPB), and includes representatives from the ABS, Appeals, IT, CPB, Taxpayer Services and Debt Management (TSDM), and Corporate Strategies and Business Development (CSBD) branches. Prior to this audit, the CGSC was not holding official meetings on a regular basis. However, on February 18, 2008, the CGSC reconvened and reaffirmed its purpose with respect to overseeing horizontal case management systems development for the Agency.

The CGSC has the appropriate representation to oversee the development of horizontal case management systems. However, there was a lack of clarity among key stakeholders and senior management regarding the Committee’s roles, responsibilities, and its reporting structure. The CGSC was not formally accountable to any other governing body, and as a result, these issues had not been resolved.

Recommendation

The program branches should discuss and agree on the CGSC’s roles and responsibilities to ensure alignment with the Agency’s vision for horizontal case management systems.

Action Plans

The CGSC will update its terms of reference to ensure its alignment with all audit recommendations and present this update to all CGSC members by September 2009.

The CGSC, in collaboration with F&A, will ensure reviews of all case-related proposals to be submitted to OPS and RIMC. The basis for reviews will be the IT Service and CASE Management Strategy and the Multi-Year Plans.

1.2.2 Globus Coordination

For Globus, there was one section within the Appeals Branch, eight sections within ABSB, and three sections in ITB responsible for day-to-day decision-making. In addition to these work sections, the Case Client Forum, consisting of managers from the various program branches using Corporate Case, was created in 2001 to share plans and discuss priorities for the various iterations of Corporate Case.

Although the Case Client Forum stopped meeting between March of 2005 to January of 2008, documentation reviews and interviews with Headquarters and regional stakeholders showed that effective communication was still occurring. Individual branch managers and regional coordinators responsible for Globus communicated regularly with each other. Issues were identified, documented, and prioritized; and action plans were approved and implemented as resources permitted.

The Globus experience indicates that using informal structures and communication channels for horizontal projects can be successful when there are relatively few business lines involved, or when the costs are relatively low. However, to meet strategic needs for large-scale horizontal initiatives, where there are more stakeholders involved and higher associated costs, formal structures, such as the Case Client Forum, may be required to help ensure that all system issues are identified and good practices are shared.

1.2.3 Standardization of Data Definitions and Common Processes

In order to create horizontal interfaces so that business lines can share data and communicate with each other, business lines need to agree on a standardized data structure to define current business processes. Standardized definitions for a common database design also need to be in place to achieve feasible horizontal linkages across all business lines and provide integrated reporting functionality.

The 2004 Corporate Case Future Directions Interim Report reinforced the importance of developing common data definitions so that all business lines could describe their business processes using a single common language. The report also states that this is the most important feature for horizontality, and the most challenging issue to deal with.

This issue was also reviewed in the Internal Audit of the Business Intelligence Decision Support (BIDS) Program (2008). The audit found that many of the clients felt that a key issue impacting BIDS was the resolution of Agency Data Warehouse (ADW) data definitions. One data element could potentially have many definitions by business users. CRA is now in the process of building additional horizontal case management systems, yet there is still no plan to identify and prioritize common data elements needed to achieve horizontal linkages between programs. The issue of common data definitions (referred to as Data Stewardship) is currently under the responsibility of the Data Stewardship Working Group, which reports to the Corporate Information Management Steering Committee (CIMSC).

Recommendation

The program branches, through the CGSC, should determine which data fields need to be standardized in order to achieve horizontal linkages between program areas. The CGSC should work with key stakeholders, including the Data Stewardship Working Group, to facilitate the prioritization and definition of the common data fields to be used in CRA case management systems.

Action Plan

The CGSC agrees to establish a horizontal forum to assist in prioritizing and defining the common data fields that are needed to meet horizontal case requirements that involve other stakeholders as needed (e.g. Data Stewardship Working Group, Corporate Information Management Steering Committee, and Corporate Audit and Evaluation Branch).

2.0 Design (for Globus)

For horizontal case management systems, design controls are needed to ensure that systems are developed in a manner that is consistent with the vision for horizontality, both with respect to IT architecture and business requirements. Key controls include:

2.1 Business Case

Although a business case was planned for Globus, one was not completed. Globus was developed at a time when a predecessor to RIMC called the Resource and Project Review Committee (RPRC) provided oversight. A project charter for Globus was prepared and incorporated most of the requirements of a formal business case.

Although it was never officially approved, the Globus Project Charter stated that Globus was being designed to meet the case management needs of both the Appeals and ABS branches, and later, all of the Agency's future case management requirements. Key issues, such as end user requirements, direct and indirect benefits to stakeholders, project scope, communications strategies, and the various roles and responsibilities for each participating branch were addressed.

The key area where the project charter could have been improved was in regard to measuring results. Although expected direct and indirect benefits were documented, there was no plan to track or measure whether the expected benefits were ever realized. The audit found no evidence that management had undertaken formal or informal efforts to assess results against the goals and objectives identified in the project charter.

The audit also found that there was no Post Implementation Review (PIR) conducted on Globus as per CRA's Project Management Policy requirement. The purpose of the PIR is to measure the project’s results against its predefined goals and objectives. The absence of a PIR or ongoing reporting against expected benefits means that lessons learned throughout the Globus experience were not formally documented for future reference. Establishing performance measurements and conducting PIRs are now required elements under the RIMC guidelines for Business Cases and Close-Out Reports.

2.2 Issue Management

The Globus Project Charter mentioned that the proposed approach for designing Globus was "breaking new ground" and identified a risk that there could be problems with the Globus technology. Employing new technology or using novel approaches to systems design carry inherent risks. Therefore, in accordance with CRA’s Project Management Policy, a formal documented risk assessment should have been prepared to identify potential risks and communicate mitigation strategies.

The project charter did contain mitigation strategies to manage potential problems associated with the Globus technology, but the listed strategies were more reactive than proactive. The audit did not find evidence of a preventative mitigation strategy, such as a proof of concept (PoC), to test whether the technology was mature enough to achieve the Globus vision.

A PoC is recognized in private industry as an essential step when introducing emerging technologies, especially in enterprise-level applications. A PoC is important because it can identify unknown risks beforehand, and is an effective tool for determining realistic and attainable objectives prior to the design stage.

A PoC may have identified the limitations of Corporate Case as an enterprise-wide platform before Globus was developed as a pilot. It would have also helped to establish more realistic objectives for Globus as a case management system for the Appeals and ABS branches only.

During the course of this audit, CPB management provided documents confirming that PoCs were performed on the CSR project.

2.3 Reporting

The audit team conducted interviews and observed employees and managers in Appeals offices in three regions, as well as the DTC, SAS, and CRR programs in two Tax Centres. Users indicated that they were satisfied with the system and appropriate feedback mechanisms, such as regional coordinators, change requests, and program updates were in place and working. Headquarters officials responsible for Globus validated the feedback from the regions. Initially, reporting functions for the Globus system employed the Cognos™ suite of tools, which were considered industry-leading, yet complex applications.

In 2006, a newer Cognos™ tool known as ReportNet was implemented, and interviews with staff and managers in the regions indicated that the new reporting tool was easier to use. No major concerns have been expressed over the presentation, accuracy, or accessibility of statistics.

Conclusion

Overall, Globus made progress in advancing technological horizontality, and provided a relatively low-cost solution that is meeting program-specific needs of the two areas that originally implemented it (i.e. Appeals and T1/Benefits in ABSB). With regular updates and improvements being made to both the system and its reporting functions, most end users and managers consider the system to be a success. As a result, the Appeals and ABS branches have ongoing plans to bring additional program workloads onto Globus. Also, neither branch has expressed a need to migrate away from Globus to take advantage of newer technology.

However, Globus did not live up to its original vision of becoming an enterprise-wide case management system. Since April of 2007, Globus has evolved into two separate service areas called “Appeals” and “T1/Benefits” in order to better accommodate program needs specific to the Appeals and ABS branches.

CRA’s experience with Globus has shown that horizontal case management systems are complex to design and present additional risks compared to conventional "stove-piped" systems. These risks include:

CRA has made progress in its attempts to develop horizontal case management systems. The evolution from the earlier iterations of Corporate Case to Globus (and eventually to the new case management functionality used in CSR) showed incremental advancements in technological horizontality, as well as a clearer vision for business horizontality. During this period, the Agency also created a horizontal working group called the Case Governance Steering Committee (CGSC) to oversee case management systems development on behalf of the Agency.

However, the Agency's efforts to date have placed more emphasis on achieving technological horizontality rather than business horizontality, and as a result, a number of the above risks remain unresolved.

In order to improve horizontality in case management systems within CRA, governance controls that encourage horizontal relationships and activities need to be enhanced. The current organizational structure in CRA still follows the traditional hierarchical model that emphasizes control and accountability, and as a result, individual managers tend to optimize activities within their own work areas, instead of focusing on long term, horizontal benefits to the Agency.

The CGSC was established to ensure that the Agency’s case management systems were developed in a horizontal manner. However, the audit found that the CGSC's activities were not fully coordinated or supported by a detailed strategy.

To help realize its vision for horizontal case management systems, the Agency should ensure that the following strategic controls are in place:


Footnotes

[Footnote 1]
A case is a unit of work that requires further action. Case management systems record and track work performed on a case.

Page details

Date modified: