Review of the Efficiency and Effectiveness of Monitoring and Oversight Processes

August 2013

(Supersedes April 2012)

7050-54 (CRS)

Reviewed by CRS in accordance with the Access to Information Act (AIA). Information UNCLASSIFIED.

Acronyms and Abbreviations

Results in Brief

Monitoring and oversight (M&O) of financial management activities is a common requirement in most Financial Administration Manual (FAM) chapters. The FAM serves as the primary source of departmental direction regarding financial management and internal controls over financial reporting (ICAFR) requirements. The M&O requirements related to the FAM chapters has been primarily delegated to the Regional Departmental Accounting Office (RDAO). This review examined the effectiveness of the monitoring activities.Some current M&O activities are not as effective as they could be for the following reasons:

• There are challenges with regards to the financial monitoring delegation structure, specifically that the Assistant Deputy Minister (Finance and Corporate Services) (ADM(Fin CS)) has no direct authority over the individuals performing the RDAO activities. In fact these individuals are performing the RDAO duties only as one part of their many duties with most of their time spent on Level One (L1) chain of command requirements.
• The RDAOs do not have sufficient resources to meet the monitoring requirements included in the FAM chapters.
• There is currently no guidance or oversight being provided by the Central Departmental Accounting Office (CDAO).
• The M&O that is being done is generally not risk-based.
• The reporting of monitoring results is inconsistent and the associated follow-up activities lack rigour.

The monitoring processes related to financial activities need to be reviewed, especially in terms of the reliance on RDAOs to perform the majority of monitoring activities. An overall financial management M&O strategy is required so that a realistic level of monitoring is assigned to all parties having responsibility for financial transactions. This should include a review of the main FAM chapters and consideration of the following:

• An evaluation of M&O requirements and a realistic and achievable distribution of M&O responsibility amongst all stakeholders, including the CDAO, L1 comptrollers and the various RDAOs.
• An analysis of the risks to financial management and ICAFR, and the development of a risk-based approach to managing M&O activities, with some risks being managed locally, some centrally and possibly accepting that some minor risks would not be managed through monitoring.
• More assistance provided by the CDAO to the RDAOs, including sharing of best practices, developing standard checklists and reporting procedures, consolidating results for departmental reporting for the Chief Financial Officer (CFO), identifying training requirements and preparing a framework for a localized approach to risk-based monitoring activities.
• An understanding by all stakeholders of the responsibilities and authorities of the CFO with regard to departmental financial management and ICAFR.

Note: For a more detailed list of Chief Review Services (CRS) recommendations and management response, please refer to Annex A—Management Action Plan.

Introduction

Background

FAM chapters cover a wide area of financial management, and ICAFR and M&O activities included in these chapters are a key component of the management control framework. Two of the most used M&O activities are staff inspection visits (SIV) and staff assistance visits (SAV), but there are other similar activities. SIVs and SAVs are generally performed by the over 30 RDAOs. RDAO is a title usually given to the senior-most comptroller in a geographical region. RDAO responsibilities are typically added to the comptrollers’ regular L1 comptrollership duties. Other types of M&O activities include command comptroller inspection (CCI) and logistic compliance inspection (LCI). A list of the most prevalent M&O activities, including the general scope of each, is included in Annex C.

SIVs and SAVs are not only considered as a compliance measurement but also as a way to improve processes, enhance controls and serve as a teaching tool for new employees.

Several factors contributed to the inclusion of this review in the risk-based audit plan. Previous CRS observations regarding the adequacy of monitoring and oversight, the initiation of financial and materiel attestation requirements in fiscal year (FY) 2008/09 and the Policy on Internal Controls were important factors. The Prime Minister’s Advisory Committee report about the cost and the time requirements related to oversight activities was also an important consideration, particularly in light of the current fiscal situation.

Objectives

The objectives of this review were as follows:

• determine if compliance monitoring is effective, efficient and provides a risk-based approach to the oversight of processes and controls related to financial management and financial reporting; and
• determine whether a governance structure is in place that enables efficiencies and promotes effective compliance monitoring (Annex B outlines the review criteria assessed in this review).

Scope

The review focused on the M&O processes in place from April 2008 to December 2011, with emphasis on compliance to the FAM chapters. Transactions initiated by special operations and transactions surrounding capital assets were excluded from the scope. Also, FAM chapters that had recently been the subject of a specific audit were removed from scope. Annex D containes a listing of the 63 FAM chapters that were in scope. The review originally included some areas of supply management in its scope, namely contracting and procurement, inventory stocktaking and food services. However, because time constraints limited the depth to which this area was reviewed, there will be no reference to the supply management area in this report.

Methodology

To complete the review, two procedures were adopted by the audit team.

Full-scope coverage procedures:

• interviewed Maritime Forces Atlantic (MARLANT) and Maritime Forces Pacific (Navy), Land Force Atlantic Area (LFAA) (Army) and 1 Canadian Air Division (Air Force) Commanding Officers for Finance (J8) and Logistics (J4), managers of compliance sections for Finance and Logistics, staff conducting finance and logistics compliance reviews, managers and/or working-level staff subject to finance and logistics compliance reviews and RDAOs; and
• reviewed documentation on compliance policies and procedures by functional authorities and respective L1 and L2, clients’ working papers, checklists and reports for sampled compliance reviews, schedules of compliance reviews, and statistical information maintained by clients.

Partial scope coverage procedures:

• interviewed Managers of Compliance Sections for Finance and Logistics and Staff conducting finance and logistics compliance reviews within the Army (Land Force Central Area, Land Force Western Area, and Land Force Quebec Area), the Air Force (3 Wing – Bagotville, 12 Wing – Shearwater, 19 Wing – Comox), Assistant Deputy Minister (Science and Technology) and Chief Military Personnel;
• interviewed ADM(Fin CS), Assistant Deputy Minister (Human Resources – Civilian), and ADM(Mat) staff during planning phase; and
• met with CDAO to discuss its role and responsibilities.

Findings and Recommendations

Monitoring and Oversight Process

The current financial management M&O activities are not effective and are potentially creating a false sense of security in terms of being effective risk-mitigation tools.

Financial Management Structure

The complexity of the Department’s organizational structure requires that roles and responsibilities for M&O be clearly defined so as to avoid gaps or overlap in subject matter and unit coverage. ADM(Fin CS), as the functional authority for financial management and as CFO for the Department, has overriding authority over financial management in the Department of National Defence and the Canadian Armed Forces (DND/CAF). This includes the responsibility for developing a financial management framework^[1] that includes M&O activities for DND/CAF financial activities.

ADM(Fin CS) has provided direction in the following manner:

• creating the CDAO, located in the National Capital Region;
• establishing RDAOs across Canada, which are supported by the CDAO; and
• defining compliance monitoring responsibilities within each FAM chapter, the majority of which are primarily assigned to the RDAOs but also, at times, shared with comptrollers and responsibility centre (RC) managers.

During the CRS review, both concerns and positive developments regarding the impact of the current management framework were noted.

One of the positive developments was the recent establishment of the CDAO. Although the CDAO was noted as a key player in the financial management framework, it was only created in FY 2011/12, and as of January 2012, its terms of reference were still being finalized. This is a key step towards the development of an effective financial management framework.

One of the areas of concern is the decentralized nature of financial operations at DND/CAF and the impact this has on assigning accountability for associated activities. Although the CFO is effectively responsible for financial management and its oversight, the various FAM chapters delegate the majority of the M&O activities to the RDAOs.

The RDAOs are located in the various regions and they have a local chain of command, but they also have a dotted line reporting relationship with ADM(Fin CS) through the CDAO that requires them to perform some financial management duties. With dotted line relationships come the usual concerns.

The individuals who perform the RDAO role have many other responsibilities. This creates a situation where establishing priorities can be challenging. In some areas, there was evidence that responsibilities related to the RDAO were given a lower priority.

RDAO Responsibilities

The functional authority structure (i.e., the CDAO and RDAOs) is embedded within the chain of command structure. An L2 or L3 comptroller for a given L1 may also be an RDAO comptroller for a particular geographical area which may include lodger units that belong to other environments or L1s. As an example, the MARLANT Comptroller is responsible for financial management for the Navy as well as functioning as an RDAO comptroller for the Halifax area which also includes amongst others, 12 Wing at Shearwater and LFAA headquarters.

The RDAOs are responsible for processing accounts payable and accounts receivable (A/R) for a geographic area. They are also responsible for monitoring these units for compliance with FAM chapters applicable to the transaction types being processed.

RDAOs have been assigned compliance monitoring responsibility in the majority of the FAMs selected for this review but surprisingly, the organization from which they receive direction, the CDAO, was only assigned responsibility in a fraction of those same FAM chapters. One of the expected roles of the CDAO is to oversee the RDAOs; therefore, a more direct correlation of responsibility assignment would have been expected within those chapters.

The fact that an RDAO may be responsible for monitoring activities for units in its geographical area but outside the chain of command could put them in a sensitive position. Although this structure appears to have worked thus far for enforcement of FAM chapters regarding the Financial Administration Act (FAA) Sections 32, 33 and 34 and the delegation of authorities, if RDAOs were to exercise the full extent of their FAM responsibilities they may very well face greater resistance from the units not under their command. The extent of the RDAOs’ authority and responsibility will have to be more clearly defined to gain widespread acceptance under this structure.

The RDAOs do not appear to have sufficient resources to carry out all of their compliance-monitoring responsibilities included in the FAMs. Occasionally, other departmental comptrollers perform some of the M&O functions. However, there were FAM chapters that were not being monitored for compliance by the RDAOs, or by the comptrollers. For example, only one of the seven organizations were reviewing FAM chapters related to civilian pay and that organization was monitoring only one of the four applicable chapters. Pre-payments, inventory, repairables, valuation and capitalization of equipment and weapon systems and works were not being reviewed by any of the organizations subject to the review. Often the RDAOs believed that the functional authorities or the L1 were monitoring for compliance.

CDAO Role

Since the CDAO had not been staffed until very recently, there were some concerns surrounding the role that would be expected of a corporate or centralized accounting office. That being said, the review occurred at an opportune time as the terms of reference for the CDAO were still being developed.

One of the roles that the CDAO could play is that of being the key information provider to the CFO regarding the quality of M&O activities being performed throughout the DND/CAF. It was observed that there has been no direct communication between the RDAOs and ADM(Fin CS) on the results of M&O activities.

Since the majority of M&O activities are the responsibility of the RDAOs, ADM(Fin CS) requires a means to quickly and directly disseminate instructions or guidelines to those who perform M&O duties on his behalf. The CDAO should be responsible for this and it should also be communicating the Department’s risk strategy and its risk-tolerance levels.

Another important role that should be considered for the CDAO is that of advisor and facilitator to the RDAOs. At this moment, most of the approximately 33 RDAOs are developing their own checklists, defining what constitutes an error in testing, determining reporting and follow-up standards and developing their own risk-based frameworks. These are some examples of the areas that are creating duplication of effort within DND/CAF. In light of the current fiscal constraints, efficiency in processes should be maximized. The CDAO should be assisting in the development and sharing of standard monitoring tools and guidelines. Finally, the CDAO is in a unique position to identify overall training requirements for M&O activities.

Risk-Based Monitoring

Given that the significant number of FAM chapters make it impractical to perform M&O activities to a substantive extent in all areas of financial management and ICAFR, it is critical that a risk-based approach to monitoring be used.

The risk-based approach should consider at the very least the inherent risks associated with the financial activities being monitored as well as the units being subjected to M&O. Some of the risk assessments can be made centrally (for example, the threshold for what is considered a high-dollar-value invoice) while others would lend themselves more to a local risk assessment (for example, there is a new accounts payable clerk with no previous accounting experience who now works on the base).

There was some evidence that a risk-based approach was being used in some limited areas. ADM(Fin CS) has set a level of $250,000 as a high-risk transaction. Invoices over this amount must undergo a pre-payment verification in addition to the standard post-payment verification (PPV). Sampling is used to select and review the low-risk transactions as part of the PPV process. As another example, there is a unit that is using a download of all acquisition card transactions as part of its review. Instead of spending a lot of time vouching acquisition card items to the invoice, given the low-risk nature (low-dollar value) of these transactions, they skim through the transactions looking for items that look unusual, including evidence of contract splitting. This is a very good example of a risk-based strategy. It also is very efficient because it allows the RDAO to perform this analysis from its own offices as opposed to incurring the cost to travel to the unit’s location and losing productivity during the travel period. Unfortunately, these two risk-based approaches were more the exception than the norm. Most of the RDAOs reviewed were not using any type of risk-based approach. They would conduct annual site visits to all units and use the same checklist each year.

Although ADM(Fin CS) has established $250,000 as the threshold level for high-risk transactions, some organizations have decreased the amount to $100,000 requiring them to pre-verify more transactions. As well, it appears that the number of low-risk transactions selected for sampling seems very high. The samples are generated directly by the main departmental accounting system and despite our numerous requests, no one was able to provide us with any documentation or useful information regarding how the parameters used by the application to determine the sample size were developed and decided upon. The amount of time spent on reviewing a large number of low-risk transactions could be used for M&O of other FAM chapters that may pose a higher risk to financial management and ICAFR.

Reporting and Follow-Up

Reporting on M&O activities and follow-up on the recommendations included in the various reports was generally found to be weak.

There is little or no standardization of reporting activities, so it was virtually impossible to consolidate the results at the departmental or even L1 level, and it was generally very difficult to even compare them from one unit to another. Reports were not tracked in any database so no trending analysis could be performed. Thus, the results of M&O activities did not easily identify systemic issues nor could they be easily used to facilitate a risk-based approach to identifying M&O requirements.

In some cases local reports would provide the unit with all of the errors found but the number of items sampled was not provided, resulting in management not knowing the extent of the errors observed or if the error rate was acceptable. For example, finding two errors out of a sample of five would lead to a different conclusion than finding two errors in a sample of 100.

Some units were reporting only negative test results leading to a biased assessment. Several reports did not provide an overall assessment of the level of compliance and strengths and weaknesses of the organizations while other reports were identifying very minor issues. These approaches appeared to lead to a lowering of employee morale and the view that the M&O process was a form of punishment for the staff of the unit being reviewed.

It was also noted that recommendations included in the report dealt with specific errors and rarely addressed systemic issues. When reports did identify systemic issues, the recommendations were often noted as a general observation and did not always require that the reviewer be provided with notification that corrective action had been taken. Easy to implement recommendations, such as recovery of a claims overpayment, were more likely to be followed up on. Treating systemic issues in this manner could lead those subject to the review to believe that the non-action items were a lower priority than the action-required items, when the opposite should be the case.

When the matter of systemic issues not being followed up was discussed with the RDAOs reviewed, inexperience and high turnover were often identified as the cause of non-compliance and the belief was that nothing can be done to address these issues. No consideration had been given to implementing preventive actions such as improving the knowledge transfer period between departing and new employees (when possible), increasing supervision of employees or improving new employee training as risk-management strategies to prevent such recurring errors.

Annex A—Management Action Plan

Management Action

A – Issue new FAM Chapters on CDAO, RDAO and Financial Governance that will articulate and communicate the financial management framework and the role of the CFO.

OPI: DG Fin Ops/DFPP

Target Date: December 2012

B – Communicate expectations through financial management governance bodies, including DCC and FEC.

OPI: DG Fin Mgmt

Target Date: January 2013

Management Action

CDAO will work with RDAOs to identify work scope and articulate their resource requirements for the next business planning cycle.

OPI: DG Fin Ops

Target Date: August 2012

Management Action

Develop and communicate to L1s and RDAOs, the CDAO Concept of Operations that describes the terms of reference for the CDAO functions and responsibilities.

OPI: DG Fin Ops

Target Date: April 2012

Management Action

A risk-based, standardized oversight framework will be implemented and used for CDAO M&O activities.

OPI: DG Fin Ops

Target Date: August 2012

Management Action

CDAO, in conjunction with RDAOs, will produce on a cyclical basis standard CDAO reporting at a departmental level for RDAO M&O activities.

OPI: DG Fin Ops

Target Date: October 2012

Annex B—Review Criteria

Objective

1. Compliance monitoring is effective, efficient and provides a risk-based approach to the oversight of processes and controls related to financial management.

Criteria

• Compliance monitoring is conducted, identifies compliance issues and provides adequate coverage of high-risk areas; and
• Compliance monitoring recommends appropriate corrective action, is reported to the appropriate level of management, followed up on to ensure corrective action is taken and documented in a manner which allows for measurement of compliance over time and against similar organizations.

Objective

2. A governance structure is in place that enables efficiencies and promotes effective compliance monitoring.

Criteria

• Roles and responsibilities of compliance monitoring are documented, clear and organized to maximize efficiencies and effectiveness of the compliance monitoring process.

Annex C—M&O Activities and Descriptions

Table 1. M&O Activities and Descriptions. This table lists the various M&O activities being conducted in the organizations reviewed during this audit and provides a brief description of each activity.

Annex D—FAM Chapters in Scope

Listed below are all the FAM chapters selected for review. The FAM chapters were categorized by the CRS reviewers to highlight the different subject areas. Although the “Total FAMs in Scope” is counted as 55, there were 63 FAM chapters in scope during this review. Certain FAM chapters that were very similar in subject matter were grouped as one in this review’s count.

Financial Statement Items (20 FAM Chapters in Scope)

• FAM 1020-2 – Accrual Accounting Principles for A/R
• FAM 1020-3 – Prepayments
• FAM 1020-4 – Capital Assets
• FAM 1020-4-1 – Land
• FAM 1020-4-2 – Buildings
• FAM 1020-4-3 – Works
• FAM 1020-4-4 – Capital Work In Progress (WIP)
• FAM 1020-4-5 – Informatics Hardware and Software
• FAM 1020-4-6 – Valuation and Capitalization of Equipment and Weapon Systems
• FAM 1020-4-7 – Leases and Leasehold Improvements
• FAM 1020-5 – Inventory
• FAM 1020-5-1 – Repairable
• FAM 1020-6 – Loans, Investments and Advances
• FAM 1021-1 – Liabilities
• FAM 1021-4 – Contingent Liabilities
• FAM 48 – Payables at Year End (PAYE)
• FAM 1018-1 – Managing Public Revenue
• FAM 1018-2 – Accounting and Control of A/R and Public Revenue
• FAM 1019-2 – Accounting for the Disposal of Surplus Assets and the Return of Proceeds from the Disposal of Surplus Assets
• FAM 87 – Write-off of Debts

Payment Method (9 FAM Chapters in Scope)

• FAM 1016-7 – Departmental Credit Cards
• FAM 1016-7-1 – Acquisition Cards/FAM 62 – Corporate Credit Cards – Issue and Control
• FAM 1016-7-2 – Departmental Travel Account
• FAM 1016-7-3 – Individual Travel Card
• FAM 1016-8 – Standard Payment System – Online Receiver General of Canada Sites
• FAM 1016-9 – Accountable Advances/FAM 1016-9-4 – Accountable Advances – Petty Cash/FAM 74 – Issue and Control of Accountable AdvancesM
• FAM 1016-10 – Managing Expense Claims
• FAM 1016-1 – Transfer Payments
• FAM 1016-6 – Interdepartmental Settlements/FAM 1016-6-1 – Suspense Accounts – Other Government Departments

Pay – Civilian (4 FAM Chapters in Scope)

• FFAM 1023-1 – Financial Administration of Civilian Pay
• FAM 1023-1-2 – Emergency Salary Advances – Civilian Employees
• FAM 85 – Secondment Agreements
• FAM 58 – The Public Service Regional Pay System/FAM 52 – Custody and Distribution of Civilian Employee Pay Cheques

Pay – Military (10 FAM Chapters in Scope)

• FAM 100 – Pay Allotments
• FAM 103 – Accounting Procedures for Military Foreign Service Regulations, 1979
• FAM 104 – Federal Income Tax, Quebec Income Tax, Unemployment Insurance Premiums, and Canada Pension Plan Contributions
• FAM 106 – Trainees in Canada Under the Military Training Assistance Program
• FAM 108 – Procedures for Compulsory Payments of Pay and Allowances
• FAM 94 – Pay Accounting Administration
• FAM 97 – Pay Accounting Records, Documents, and Vouchers
• FAM 98 – Issue of Pay and Allowances and Acceptance of Funds for Credit to Pay Accounts or for Remittance
• FAM 102 – Enrolments, Releases, Members Deceased, or Members Reported Missing, Prisoners of War, Interned, or Detained by a Foreign Power
• FAM 1025-14 – Mandatory Payment of Annual Leave – Regular Forces

Types of Expenditures (3 FAM Chapters in Scope)

• FAM 1017-6 – Hospitality – Official Mementos/FAM 1017-1 – Hospitality Inside Canada
• FAM 50 – Provincial & Territorial Sales & Fuel Taxes
• FAM 1019-8 – Reporting and Recovery of Public Support to Shared Activities of Personnel Support Program (PSP)

Cash Management (6 FAM Chapters in Scope)

• FAM 1020-1 – Cash
• FAM 1018-3 – Acceptance of Credit and Debit Cards
• FAM 68 – Duties and Responsibilities of Accounting Officers, Authorized Forms, and Disposal of Obsolete Documents
• FAM 70 – The Working Capital Fund Account
• FAM 71 – Canadian Forces Unit Working Capital Ledger
• FAM 73 – Identification & Control of Disbursements

Financial Controls (3 FAM Chapters in Scope)

• FAM 7 – Financial Systems and Controls
• FAM 1016-2 – Expenditure Planning and Initiation – FAA Section 32/FAM 1016-3 – Account Verification – FAA Section 34/FAM 1016-4 – Payment – FAA Section 33
• FAM 1014-4-1 – Control of Financial Signing Authorities

Footnote 1 As per DAOD - 1000