Audit of Account Verification Quality Assurance Processes

Audit objectives

The first objective of this audit was to assess whether quality assurance (QA) activities over the account verification are adequate to demonstrate due diligence per the FAA and associated TB Directive.

The second objective of this audit was to assess the overall implementation of the A/P process controls in the SAP environment from expenditure initiation to issuance of payment.

Summary of key findings

As a result of the A/P implementation, input processing time has increased slightly but this is offset by a decrease in processing time for payment, review and query responses.

Supporting documentation is not consistently accessible or uploaded in electronic format. SAP and the Vendor Invoice Management (VIM) system are not used to the fullest extent to manage A/P documentation.

Business rules were defined and worked as automated controls to block payments relating to invoice errors or to flag high risk characteristics for review. Error types and severity have been identified and form the basis for QA reporting. A set of desk procedures is maintained by the A/P processing centres to guide officers in reviewing various types of transactions. The controls in place were observed to be working effectively.

QA samples were drawn from a pool labelled low risk that included both low and medium risk transactions which tends to skew the results in favour of the low risk transaction. One class of low risk transactions was removed in August and reviewed separately, which partially corrects the skewing. Further refinements to the sampling plan could be considered.

There are significant delays observed in obtaining supporting documentation related to acquisition card purchases. This causes untimely completion of QA reviews and reports to management. There are also delays in obtaining copies of contract terms and conditions to verify price, quantity and quality. Observed error rates are within established tolerances. QA review results for low risk transactions are reported in the SAP system. Results for high risk pre-verification reviews are not captured but are corrected prior to payment.

Audit conclusion

In the opinion of the audit team, the design and implementation of the A/P system in SAP, including processes for continuous improvement, are adequate to manage the Department's A/P relating to operations and maintenance and interdepartmental settlements. There are opportunities to improve the Department's practices with respect to the use of electronic documents.

The sampling and gating protocols implemented by the Department are also adequate to demonstrate compliance to the Directive on Account Verification. Some improvements have been suggested for consideration.

The conduct and management of the QA review process is adequate. There is an opportunity to improve the documentation of QA review results which will strengthen the quality of reports on the adequacy of account verification in the Department.

Recommendations

1. It is recommended that the Chief Financial Officer (CFO) issue clear guidance on what supporting documents for A/P should be uploaded to the system and who is responsible for this work.
2. It is recommended that the CFO develop a methodology for capturing the results of QA reviews of high risk transactions.

1.1 Context

On April 1, 2014, Employment and Skills Development Canada (the Department) began to use the SAP enterprise management system as the system of record for financial transactions. The migration from the Corporate Management System to SAP necessitated a change in the business processes used in the Department.

In particular, the A/P implementation has changed the processes used for initiation and approval of invoices and payments. At the same time, greater functionality embedded in SAP changed the way that QA selected transactions for pre-approval and post-audit.

Good design principles in an A/P system will ensure that segregation of duties is part of the design, that delegated limits over transaction types are respected and that audit trails are maintained so that management can hold delegated officers accountable for their decisions. These design principles are also embedded in the FAA and the TB Directive on Account Verification (the Directive) making the implementation of a good design also a matter of compliance to legislation and policy.

The Directive mandates that departments implement a system of QA over account verification. Account verification is commonly referred to as Section 34 of the FAA and is the certification that goods and services were received as ordered or that the payee is eligible to receive the payment. The Department's QA program contains a detailed listing of error tests that satisfy the requirements of Section 34 and related policies. ^{Footnote 1}

1.2 Audit objectives

The first objective of this audit was to assess whether QA activities over the account verification are adequate to demonstrate due diligence per the FAA and associated TB Directive.

The second objective of this audit was to assess the overall implementation of the A/P process controls in the SAP environment from expenditure initiation to issuance of payment.

1.3 Scope

The scope of this audit includes:

• Transactions relating to the period June 1, 2014 to August 31, 2014 selected for QA review or gated for pre-verification from the SAP A/P system and all related documentation.
• Design documents relating to the implementation of the A/P modules in SAP and the QA review processes.
• All systems involved in the processing of A/P or in the QA review process.

Transactions processed through the Common System for Grants and Contributions (CSGC) are not part of the scope of the audit.

Segregation of duties is a key control in the design of accounting systems. Internal Audit Services is conducting a separate audit of the implementation of delegation of financial authorities within the SAP system. Therefore, this audit does not examine segregation of duties in detail.

A major upgrade to SAP was underway during the conduct phase of the audit which limited the testing of automated controls. Additional tests are scheduled for February 2015 and any significant findings will be reported under separate cover.

1.4 Methodology

This audit used a number of methodologies including: document review, interviews, on-site observations, walkthroughs, as well as sampling and testing.

Representatives from Chief Financial Officer Branch (CFOB) at National Headquarters (NHQ) and at the regional processing centres in Montreal and Winnipeg were interviewed in order to have a comprehensive view of the operational environment. Travel to regional offices took place in August and November 2014.

2.1 Accounts payable process is adequately designed

The Department has implemented a standard SAP A/P module with the minimum necessary customization. This customization is primarily in the area of approval controls which follows the delegation of authority regimes mandated by the FAA and other federal legislation.

The major programs - Employment Insurance, Canada Pension Plan (CPP), Old Age Security, Canada Student Loans, and Canada Education Savings Grant - issue payments to Canadians and other payees through their own systems. QA over payments in these systems has been or will be addressed in other audits. The SAP A/P system is used to issue payments relating to operations and maintenance transactions and grant and contribution payments. Approvals for grant and contribution payments are handled through CSGC and have a separate QA process.

The audit team conducted walkthroughs of A/P transactions as part of the planning and testing phases of the audit. The walkthroughs showed that there are eight main streams that lead to issuing a payment. Purchases may be initiated by a formal purchase order (PO) and contract or they may be ordered without a purchase order (NPO). The invoice may be either paper or electronic which leads to four combinations or streams - PO with electronic invoices; PO with paper invoices; NPO with electronic invoices; and NPO with paper invoices. The fifth stream consists of employee travel which is initiated and approved through the SAP travel module. High volume low dollar paper based transactions are another stream. Payments made through an acquisition card are the seventh stream; and grant and contribution payments are the eighth.

In the first five streams, the SAP module creates a work flow notification that requires a delegated manager to electronically certify the transaction under Section 34 of the FAA. The high volume low dollar transactions have the Section 34 certification signature directly on the invoice, which is confirmed at the input stage. Acquisition card transactions are reconciled using a separate set of procedures but are subject to QA verification.

Paper invoices are scanned into the VIM and an optical character recognition subsystem, the Invoice Capture Center (ICC), attempts to capture the essential invoice information such as payee, invoice date, amount, PO number and due date. ICC can be programmed to recognize each vendor's invoice style to improve the data capture results.

The scanned documents are accessible on-line to delegated managers and financial officers for review, follow up and QA. The audit team was informed that although there was a small increase in the work needed to input the invoices to the system, the availability of the scanned documents has resulted in improvements in the time needed to follow up queries from vendors. Additional backup documentation, such as email correspondence or journal voucher worksheets, can be attached directly to the transaction in SAP.

In the transactions reviewed by the audit team, many of them did not have scanned backup documents attached. Further, the audit team could not find any formal guidance on which documents should be uploaded into the system. In the opinion of the audit team, the benefits of having scanned supporting documents, such as packing slips, variance reports, contracts and requests for proposals available are numerous. There is an opportunity to improve the efficiency of follow up and review by ensuring that a complete set of supporting documents for each transaction are available electronically.

Business units which receive goods or services pursuant to a PO create a receiving report to acknowledge what was received from the supplier. This receiving report is matched to the PO and the invoice. Any discrepancies are flagged for follow up by the A/P processing unit.

SAP has error trapping routines built in to the travel and A/P modules which will block transactions with incomplete information or that exceed committed budget or that are duplicate payments. Transactions that exhibit pre-defined high risk characteristics are gated for QA review prior to payment. The workflow audit trails capture the automated error tests and results. The audit team and CFOB are aware that there are some potentially high risk transaction characteristics that did not trigger a pre-payment review at the time of the audit. These will be reviewed and implemented as part of CFOB's continuous improvement process.

The audit team was not able to complete detailed testing of the error trapping routines embedded in the SAP system in time for this report. A major SAP system upgrade was planned for late January which affected the working environment. Testing will be conducted in February 2015 and any significant findings will be reported separately.

The audit team concludes that the A/P module implementation with the VIM subsystem provides some benefits for the management of A/P. The controls embedded in the system do block errors for correction and high risk transactions are gated for pre-payment review.

2.2 Gating and sampling design for quality assurance is adequate

QA officers review transactions in three circumstances: transactions rejected by the A/P error routines; high risk transactions flagged by the A/P gating routines for pre-verification by QA prior to releasing payment; and transactions sampled from low risk transactions for quality control purposes.

As part of the implementation of the A/P module, two sets of business rules were created. One set defines the set of errors which will cause a transaction to be rejected for rework and block the payment. A second set defines parameters for high risk transactions that will be reviewed in detail by QA personnel. In both cases, QA personnel verify the request for payment prior to releasing the system block. Any errors discovered are categorized by type and severity as specified in the review checklist.

The audit team observed gated transactions from most of the high risk categories established in the business rules, which indicate that the system applies the rules as specified. The audit team was informed that some questionable transactions were not gated for pre-verification. These potential high risk characteristics were not part of the initial implementation but have been flagged for consideration. A/P management has a process in place to modify the business rules. The audit team regards this as part of the continuous improvement of the QA process that is mandated by the Directive. The details of these transaction characteristics have been discussed with A/P management.

During the period from April to July 2014, all transactions that were not gated for high risk characteristics were eligible for sampling. Monthly samples were drawn from the low risk population for QA review. The samples in this period were skewed because the highest volume of transactions processed through the A/P module is for fees for medical examinations relating to CPP Disability claims. Beginning in August 2014, the CPP Disability medical fees are treated as a separate pool of transactions from the general low risk pool. This allows for better information about both pools of transactions.

The Directive specifies different review requirements for high, medium and low risk transactions. The Department has only two levels of risk, high and low. From a sampling design perspective, bundling low and medium risks into a single pool will tend to over-represent low risk transactions and under-represent medium risk transactions. The segregation of the CPP Disability medical fee payments into a separate population has mitigated this but the remaining low risk sample is still skewed.

One issue with the creation of the low risk samples is that acquisition card transactions are included in the sample. These transactions are processed in a different manner than other payables because the statements are paid in full first then reconciled, verified and attributed to the correct fund center. As discussed above, the documentation for these credit card transactions is often not readily available to the officers who perform either the reconciliations or QA. Missing or delayed documentation affects the timing and quality of the QA reports. CFOB may wish to consider creating a separate sample of acquisition card transactions and embedding the QA process in the reconciliation process.

The original intention was to produce monthly QA reports on low risk transactions, which was a good decision immediately post-implementation. To date, the documentation issues noted above have delayed the production of the reports. As the Department becomes more experienced in the use of the SAP enterprise management system, CFOB may wish to consider reducing the frequency of low risk sampling by basing the samples on a quarterly population rather than a monthly population. The resources freed up could then be assigned to more thorough reviews of medium risk transactions such as travel and hospitality.

The audit team concludes that the sampling and gating plan was adequate based on information available during the design phase. Additionally, there are processes and plans in place for regular review and refinement of the system parameters for gating and sampling which satisfy the requirements of the Directive.

2.3 Quality assurance processes are adequate but can be improved

QA officers use a checklist as an aid to guide their review. The checklist provided by NHQ is based on the requirements of the Directive and lists errors by type and severity. However, the checklist does not provide guidance on how to perform the review rather it is a comprehensive listing of error reporting codes. This is mitigated by the experience and professional qualifications of the QA officers, regular team meetings, and conference calls between the A/P processing centres. Additionally, the Winnipeg processing centre maintains a set of desk procedures to guide QA officers. This is an evergreen document that is updated when new issues are discovered or when required by system changes.

The internal audit team verified the QA officer's decisions from two samples of transactions: high risk transactions released for payment and low risk transactions selected for QA review. Overall the error rate in the low risk population, as reported by the QA officers, is within the 5% tolerable error rate. The audit team's review indicates that the QA officers' error rate is also within the tolerable limit. The audit team concludes that the QA process is performed competently by the QA review officers.

The audit team observed that the QA review personnel are diligent in their review of their assigned transactions and occasionally go further than expected. One of the officers interviewed was concerned about duplicate payments to one-time vendors, the majority of whom are Canadian passport applicants who are reimbursed for the cost of replacement documents and photos damaged during passport processing. He developed an ad hoc routine to extract all instances where the same person was paid more than once and discovered that there were four duplicate payments during the first six months of the fiscal year out of approximately sixteen hundred payments, leading to the conclusion that the risk of duplicate payments in the one-time vendor process is very low.

One area of concern is the delay in confirming the validity of prices where there is a contract in place for goods and services. The scanned documents will often have time sheets or packing slips attached to the invoice which can substantiate the hours worked or the quantity of goods received. When a delegated manager certifies a payment for Section 34, the manager is confirming that all contract terms and conditions have been met, including price, quantity and quality. The QA officers are responsible to verify this is correct. However, the contract, or contract text, is not readily accessible in electronic format to the QA officers to verify the per diem rates or the contracted price for various goods. This leads to delays in both pre-verification and QA review while copies of the relevant contracts or pages of the contract are located and forwarded to the reviewer. CFOB should consider that contracts are part of the suite of supporting documents for A/P and ensure that QA officers have access to them.

The requirements of the Directive are not fully met with respect to documenting QA decisions. The SAP system has a pop-up data collection screen available for the low risk transactions that allows the officers to enter the type of errors, if any, and add two lines of notes approximately 80 characters each. Neither the pop-up screen nor the system has a way to track the identity of the review officer, which would be useful to management when following up issues. Otherwise, the pop-up screen is sufficient to capture the essential information about the decision. The officer can record a more detailed explanation by attaching a document to the transaction file or by using the permanent note feature in the transaction record.

However, the transactions gated for high risk pre-verification do not have a data collection feature available in the system. Section 6.3.3 of the Directive requires that QA review practices enable reporting on results to demonstrate the adequacy of account verification. Currently there is no formal method for capturing errors discovered and corrected during pre-verification. The audit team was informed that there is usually some correspondence between the QA officer and the business unit to resolve issues that block release of payment to the supplier but this is not routinely attached to the transaction file. High risk transaction errors are corrected prior to payment, therefore the residual risk to the Department is more about preventing future errors and the cost of correcting them than in issuing an invalid payment. Because there is no formal way to capture error reports, the information needed to make decisions about system improvements, training or other corrective actions is ad hoc in nature and may lead to sub-optimal decisions.

The audit team concludes that the QA processes in place are adequate to meet the requirements of the Directive, except with respect to error reporting for high risk transactions.